Question 160 A source publishes data as gzip-compressed CSV files. The ingestion strategy must correctly interpret the content. What does the design need to account for? Only the Delta table history retention policy Set the reader to parse uncompressed text and rely on notebook retries for the gzip payload Only whether SQL keywords are uppercase The file format and supported compression handling in the selected reader Explanation Correct Answer D:The reader must decode compressed CSV payloads before interpreting delimiters and typed values. Incorrect Answers A:History retention affects stored versions, not source decompression. B:Retries repeat the same incompatible parsing behavior; the reader needs support for the actual compression and file format. C:SQL casing does not decode gzip data.
Question 161 A cloud-object source can deliver a new file while an earlier one is still being uploaded. The loader should not ingest incomplete content. What should the source contract address? Ingest every object as soon as its name first appears irrespective of completeness Atomic delivery or readiness signaling before discovery/consumption Only changing output table column names Increase the file-discovery polling frequency without requiring an upload-complete signal Explanation Correct Answer B:A producer-side completion convention prevents readers from consuming partially written files. Incorrect Answers A:Premature ingestion can parse incomplete records. C:Target naming cannot establish source-object readiness. D:More frequent discovery raises the chance of reading an incompletely delivered object; readiness must be explicit.
Question 162 A stream consumes object files into a governed bronze table but leaves no checkpoint metadata. What does it risk during driver restart? Changing the actual source file encryption key Automatically acquiring new Unity Catalog privileges Removing data from every gold table automatically Losing durable progress needed to resume the incremental stream safely Explanation Correct Answer D:Without stable checkpoint state, recovery can reprocess or skip files depending on restart design. Incorrect Answers A:Missing stream progress state does not by itself rotate source encryption. B:Restart behavior cannot grant unrelated catalog permissions. C:A restart does not inherently delete unrelated downstream tables.
Question 163 A silver pipeline has rows with price_txt values "12.50", "9.00", and "bad". Gold requires numeric revenue and visibility into invalid values. Which transformation is appropriate? Use a tolerant numeric cast with an explicit invalid-value quality rule Convert all malformed strings to 12.50 Sum the raw strings lexicographically Drop every row with any text-valued field Explanation Correct Answer A:A safe cast plus bad-record handling keeps valid prices numeric while separating malformed source values. Incorrect Answers B:Inventing a valid price would contaminate the gold revenue measure. C:String concatenation/order does not compute decimal revenue and cannot flag invalid numbers. D:All source prices are text encoded, so this would remove legitimate numeric records.
Question 164 A DataFrame has a nullable region column. A filter `col("region") == "EU"` is applied. Which rows remain? All rows except the ones marked EU Filter using `region IN ("EU")` and include rows whose region is NULL by assuming nulls compare equal Replace equality with `region.isNull()` but claim it selects European rows Only rows whose region compares equal to EU Explanation Correct Answer D:The predicate retains matching values; null comparison is not true and does not pass the filter. Incorrect Answers A:That is the opposite of the stated equality predicate. B:SQL three-valued logic means NULL comparisons do not evaluate to TRUE for this equality predicate. C:An isNull filter deliberately selects missing regions, not the records whose value matches EU.
Question 165 An order DataFrame must keep every order even when the customer dimension lacks a matching customer ID. Which join fits? Inner join from orders to customers Left semi join from orders to customers Left anti join from orders to customers Left outer join from orders to customers Explanation Correct Answer D:A left join retains all order-side rows and supplies null customer columns when unmatched. Incorrect Answers A:An inner join drops orders with no matching customer record. B:A semi join retains matching orders only and does not include customer columns. C:An anti join returns only orders lacking a customer match, not all orders.
Question 166 A customer table has two rows for ID 42, while the orders table has three rows for ID 42. How many rows can an inner equijoin on ID produce for that key? One joined row Two joined rows Six joined rows Three joined rows Explanation Correct Answer C:Each of the three order rows matches both customer rows, so the key produces 3 × 2 = 6 join combinations. Incorrect Answers A:An ordinary equijoin does not automatically deduplicate duplicate join keys. B:Two counts dimension duplicates but ignores three separate order matches. D:Three would assume only one customer record matches each order, but there are two.
Question 167 Two daily DataFrames have identical schemas and must be stacked while retaining all duplicates for later reconciliation. Which operation is appropriate? CROSS JOIN UNION DISTINCT with an implicit uniqueness requirement UNION ALL in SQL or DataFrame union with matching columns INNER JOIN on all fields Explanation Correct Answer C:Union-all-style stacking retains duplicates; Spark DataFrame union does not distinct rows automatically. Incorrect Answers A:A cross join multiplies combinations instead of stacking source datasets. B:Distinct union can discard repeated rows needed for reconciliation. D:Joining pairs records rather than appending one day beneath the other.
Question 168 Two DataFrames have the same column names but in different orders. The engineer wants to align fields by name before stacking. Which PySpark method is suited? union with no column-order review crossJoin groupBy without aggregates unionByName with compatible column types Explanation Correct Answer D:unionByName matches columns by name, avoiding positional mismatch from different orders. Incorrect Answers A:Standard union resolves columns by position and may misplace values when orders differ. B:A Cartesian join multiplies rows rather than stacking them. C:Grouping does not align and append rows from the two DataFrames.
Question 169 An inventory dataset contains a tags array for each product and requires one output row per product-tag pair. Which transformation is relevant? filter(tags IS NULL) Use size(tags) to compute the number of elements without producing separate rows explode(tags) while preserving the product identifier split(product_id) Explanation Correct Answer C:Explode turns each array element into its own row for downstream relational analysis. Incorrect Answers A:Filtering to null tags discards the records that need expansion. B:size reports array length per product; it does not expand the array into the required one-row-per-tag output. D:Splitting the identifier does not flatten the tags array.
Question 170 A raw transactions table may contain identical rows more than once. Before a distinct-order count, the engineer must remove duplicate order IDs while retaining one row per order. Which operation fits? crossJoin with the distinct IDs groupBy("order_id").count() as the final order detail table dropDuplicates(["order_id"]) after defining which row wins dropDuplicates() on every column without regard to ID Explanation Correct Answer C:Key-specific dedup retains a row per order, but business ordering is needed when duplicates conflict. Incorrect Answers A:A Cartesian join would inflate rather than deduplicate order records. B:A count aggregate removes non-key order attributes and is not itself a complete row record. D:Rows with same order ID but different timestamps would not be collapsed.