Question 380 A large join stage shuffles 400 GB of data and spends most of its time exchanging partitions. Which optimization should be investigated? Granting every analyst write permissions Join strategy, filters, and partitioning that can reduce the shuffle Increase the number of notebooks in the workspace without changing the join plan Increase the shuffle partition count without inspecting skew, data reduction or the current join strategy Explanation Correct Answer B:Broadcasting a small side, filtering early, or improving partition distribution may reduce exchange cost. Incorrect Answers A:Authorization scope is unrelated to shuffling efficiency. C:Notebook inventory does not reduce network shuffle data exchanged during the large join stage. D:Adding partitions can change task size but cannot by itself remove unnecessary exchanges; diagnose join distribution and filtering first.
Question 381 A Spark job fails with an executor out-of-memory error while the driver is healthy. Which setting or design should be examined? Inspect only the source table namespace and assume this explains executor memory exhaustion Increase only driver memory without checking executor input partitions Executor memory use, per-task data sizes, and skew/shuffle behavior Increase only driver memory even though Spark reports the failing allocation on an executor Explanation Correct Answer C:Executor OOM arises in distributed tasks and may require better partitioning or appropriately sized executors. Incorrect Answers A:The catalog name cannot establish task-level memory requirements or the oversized input/shuffle responsible for an executor OOM. B:The failure is on executors, so driver display history and its memory setting do not address per-task executor pressure. D:Driver memory does not directly resolve executor heap pressure. Diagnose per-task skew and memory on the process that failed.
Question 382 A Python notebook calls collect() on tens of millions of rows and its driver crashes with out-of-memory. What correction is most direct? Avoid bringing the full distributed dataset into driver memory Raise the executor count but keep collecting the complete dataset on the driver Increase all-purpose compute workers but keep collecting tens of millions of rows into driver memory Add more executor cores while keeping the same unlimited collect Explanation Correct Answer A:Collect materializes all selected rows on the driver; distributed computation or bounded samples are safer. Incorrect Answers B:More executors do not remove the single-driver memory limit that collect() can exhaust. C:More workers do not remove the driver-side collect operation that materializes the entire result on one process. D:More executors do not prevent the driver receiving an oversized result.
Question 383 A Databricks job fails before its notebook begins because no worker nodes are provisioned. Which diagnostic area is most relevant? Check only target row counts without examining cluster provisioning errors Increase notebook task timeout but leave the cluster unable to provision worker nodes Cluster startup events, compute policy, quota and cloud capacity errors Tune the gold SQL query plan without reviewing the failed cluster creation request Explanation Correct Answer C:A pre-execution failure points to resource provisioning rather than transformation SQL logic. Incorrect Answers A:A job that fails before notebook startup has not executed its table write; provisioning events and cluster policy are the relevant evidence. B:A longer task deadline does not resolve cloud quota, capacity, startup failure or restrictive compute-policy errors before execution. D:Query optimization cannot start a cluster whose worker provisioning failed before any notebook statement ran.
Question 384 A job succeeds on development compute but fails in production with ClassNotFoundException for a library. What should be compared? Compare only Spark executor counts while ignoring library installation Only the number of job tags Installed library versions and runtime dependencies across compute environments Only the latest source table row count Explanation Correct Answer C:The class-not-found error suggests a missing or incompatible runtime dependency in production. Incorrect Answers A:Executor counts can affect performance but do not establish which dependency is missing in production. B:Tags do not install dependencies. D:Data count does not create missing Java/Python classes.
Question 385 After a library upgrade, a notebook fails with an API method that existed previously. What is the most targeted first action? Drop the source Delta table immediately Change the Spark shuffle partition count before checking dependency compatibility Compare dependency versions and check for a breaking API or transitive conflict Increase shuffle partitions Explanation Correct Answer C:A version upgrade can change method availability; pinning a compatible build may resolve it. Incorrect Answers A:Data deletion does not repair a library API mismatch. B:Shuffle tuning does not resolve a method removed or changed by an upgraded package. D:Partition parallelism cannot restore a removed method.
Question 386 A nightly Lakeflow Job usually runs 14 minutes but has taken 42 minutes for three consecutive days. What is a useful first measurement? Compare only the number of jobs configured in the workspace Inspect the current notebook source but not previous run metrics Increase job retries until execution returns to fourteen minutes Compare recent run-history durations and input volumes with historical baselines Explanation Correct Answer D:Run history confirms the regression and helps distinguish increased work from a compute or code change. Incorrect Answers A:The number of job definitions does not show whether input volume, run environment or task duration changed in this specific job. B:Source code inspection can help, but cannot establish the measured 14-to-42-minute regression without prior runs and workload sizes. C:Retries recover failures but are not a diagnosis of a consistent slow execution pattern.
Question 387 A job task repeatedly fails in the validation branch but all upstream ingestion tasks succeeded. How should an operator narrow the problem? Delete the validated table to hide the error Assume the ingestion tasks failed despite success status Upgrade every cluster immediately without error analysis Inspect the failing task logs and the data contract or assertions it evaluates Explanation Correct Answer D:A validation-only failure points to its own logic or the quality of ingested records. Incorrect Answers A:Removing evidence prevents root-cause review. B:There is no evidence that successful extraction tasks failed to execute. C:Unrelated changes can complicate diagnosis and do not target the validation failure.
Question 388 A Spark UI shows one reduce task processing 80% of a dataset while others are nearly empty. Which root cause is likely? A skewed key distribution concentrating rows into one shuffle partition An evenly balanced hash partitioning of the join key Uniform partition distribution Compare the job owner’s name while ignoring the task run-as data identity Explanation Correct Answer A:Uneven shuffle partition sizes indicate hot keys and imbalanced work. Incorrect Answers B:Uniform key distribution would not leave one reducer processing most rows; this is the opposite of the observed skew. C:Uniform distribution would not concentrate most rows into a single task. D:User naming does not establish effective object access; examine the run-as principal and its actual Unity Catalog grants.
Question 389 A join between a billion-row fact and a tiny lookup causes expensive shuffling. Which physical optimization should be evaluated? Cross-joining then filtering after materialization Rename join aliases but leave the large shuffle plan unchanged Broadcasting the small lookup if it is safe and supported Broadcasting the billion-row fact unconditionally Explanation Correct Answer C:Broadcasting the small relation can remove a large shuffle from an equijoin plan. Incorrect Answers A:Cartesian expansion increases intermediate data and processing. B:Changing aliases does not change how Spark distributes join records or whether the small lookup is broadcast. D:Huge broadcast payloads can exhaust memory and worsen the workload.
Question 390 A Spark stage shows high task scheduling overhead because a tiny dataset was split into tens of thousands of shuffle partitions. What can help? Drop all query predicates Change table owner privileges Reassess shuffle partition count and adaptive coalescing behavior Increase partition count without limit Explanation Correct Answer C:Excessively small tasks can cost more to schedule than to process; suitable partition sizing reduces overhead. Incorrect Answers A:Processing more rows will not solve too many tiny tasks. B:Permissions do not set physical shuffle task granularity. D:More tiny tasks can amplify scheduler overhead.