Question 400 A job has many stages with executor CPU near idle and high network shuffle time. Which tuning lever is more plausible than adding CPU cores blindly? Reduce unnecessary exchanges or improve join and aggregation partitioning Increase executor CPU indefinitely regardless of plan Increase executor heap without inspecting remote shuffle transfer and skew metrics Revoke dataset SELECT grants Explanation Correct Answer A:When network movement dominates, more CPU may not address the bottleneck. Incorrect Answers B:Additional cores cannot remove excessive data exchange by themselves. C:More heap might not address a network-bound shuffle stage; the diagnosis should start from the observed fetch waits and partition distribution. D:Reducing permissions would not make the authorized query execute faster.
Question 401 A production pipeline processes 1 TB but fails with a driver result-size error after attempting to print all rows. What modification addresses the cause? Increase executor parallelism but retain the full DataFrame collect on the driver Increase only the number of worker nodes while retaining a full driver-side print Stop materializing the complete distributed result on the driver Increase task retries without changing collect Explanation Correct Answer C:Distributed transformations should write or aggregate results instead of collecting 1 TB to driver memory. Incorrect Answers A:Additional executors do not remove the driver-side materialization that causes the result-size failure. B:More distributed workers do not avoid collecting the entire result back to one driver process. D:Repeated driver overflows will remain likely.
Question 402 An ETL job works until a dependency upgrade changes the behavior of a date parsing function. Which operational practice helps isolate the regression? Compare pinned library/runtime versions against the last successful run Alter every source event timestamp before investigation Disable all job history retention Change the dashboard refresh schedule without comparing the dependency versions used by the failing notebook Explanation Correct Answer A:A version delta linked to the symptom narrows root cause and supports a controlled rollback. Incorrect Answers B:Blind data changes hide whether the dependency caused the behavior shift. C:Removing history discards the version and failure evidence. D:Dashboard scheduling cannot explain a method removed or changed by an upgraded library in the job runtime.
Question 403 A job fails intermittently when many workloads start together, and cluster event logs mention a cloud quota limit. What should the operator do? Delete Delta file history Change every notebook SQL join to CROSS JOIN Grant analysts extra SELECT permissions Check workspace/cloud quotas and capacity provisioning against concurrent demand Explanation Correct Answer D:Cloud instance or compute quotas can block startup when multiple jobs request resources at once. Incorrect Answers A:Storage history cleanup does not allocate new worker instances. B:Join logic does not resolve resource provisioning quota denial. C:Data-read grants do not increase cloud instance quotas.
Question 404 A heavy groupBy stage has almost equal input partitions but one reducer remains slow because one group key appears in nearly half the records. What concept explains it? Automatic permission inheritance Notebook cell ordering Aggregation key skew after the shuffle Uniform aggregation key frequencies Explanation Correct Answer C:Many records sharing one key can concentrate into one reducer regardless of earlier balanced inputs. Incorrect Answers A:Catalog grants do not control aggregation group sizes. B:Cell numbering does not determine group key distributions. D:Uniform keys would not create the described single hot reducer.
Question 405 A team enabled Liquid Clustering but queries use entirely different filters from the chosen clustering keys. What should be assessed? Increase table size without examining which columns queries filter on Assume any clustering key always optimizes all columns Whether the clustering key strategy reflects real query predicates Delete indexes from unrelated operational databases Explanation Correct Answer C:A layout optimized for unrelated attributes may not provide useful file skipping for actual filters. Incorrect Answers A:More data does not align clustering keys with the actual predicates that determine file-skipping effectiveness. B:Physical layout benefits depend on data distribution and query access patterns. D:Those sources do not control the Delta table's clustering strategy.
Question 406 A Spark task OOMs because a single customer has 90% of input records and a join groups them together. What should be considered? Address data skew and task-level memory pressure using a suitable join/partition strategy Repartition only by the same highly skewed customer key without addressing the hot key Increase driver memory while retaining the per-customer skew in executor tasks Increase every job retry count without revising the skewed join strategy Explanation Correct Answer A:A hot key can create one oversized partition even if total cluster memory is large. Incorrect Answers B:Hash partitioning by the same hot customer ID can concentrate its records in one oversized partition and preserve the OOM. C:The hot-key shuffle pressure is distributed; a larger driver does not remove the oversized executor partition. D:Retries repeat the same heavy partition and cannot resolve an underlying skew/OOM condition.
Question 407 A SQL query suddenly scans most of a large Delta table despite filtering a narrow subset. What should the engineer inspect? Query filters, file pruning statistics and the table's current layout Check only the total table row count without examining the query plan Only the number of workspace folders Increase SQL warehouse capacity before examining the query plan and pruning Explanation Correct Answer A:Unexpected full scans can reflect ineffective predicates, unsuitable clustering or stale data layout. Incorrect Answers B:Row count cannot explain why a selective predicate failed to prune data files or which filter was applied. C:Folder count cannot reveal missing file skipping. D:Additional compute can hide symptoms but does not identify why a selective filter now scans many files.
Question 408 A data engineer sees dozens of failed task retries followed by eventual success, but the overall nightly job duration doubled. Which evidence is useful? Only the number of result columns Task attempt history, retry count and time spent in failed attempts Look only at the final successful job state, ignoring failed task attempts Only the catalog display name Explanation Correct Answer B:Retries add wall time, so attempt-level evidence helps explain the duration increase. Incorrect Answers A:Schema width alone cannot explain repeated failures and retries. C:A green eventual success can hide recurrent retries; run-level success alone cannot explain the underlying transient failures. D:Namespace spelling cannot show attempt timing.
Question 409 An executor throws OutOfMemoryError during a large cross join introduced by a code change. What is the best first correction? Increase the cluster to maximum size while keeping the unintended cross join Change the join alias and assume it eliminates an unintended Cartesian product Delete all source tables Restore a valid selective join predicate or remove the unintended Cartesian expansion Explanation Correct Answer D:The cross join can multiply data volume dramatically; repairing semantics is more direct than only adding memory. Incorrect Answers A:More resources may not make the incorrect unbounded plan acceptable. B:Alias changes do not create a missing join predicate or prevent cross-join cardinality explosion. C:Destructive data deletion is not necessary to fix the join plan.
Question 410 A long-running job is failing during dependency installation because package versions conflict. Which logs and state matter? Retry the same job without examining the declared dependency versions Inspect only the job run result and omit the library installation errors Library installation logs and the declared dependency graph Upgrade every installed package blindly to its latest available version Explanation Correct Answer C:Version constraints and installation errors reveal incompatible package combinations. Incorrect Answers A:A retry may encounter the identical package-resolution conflict; library installation evidence is needed to identify incompatible version constraints. B:A failed overall run status can show when installation failed but not which dependency constraints or installed artifacts caused the conflict. D:Updating all packages without checking compatibility constraints can introduce new conflicts instead of resolving the reported dependency error.