Question 640 An MLflow evaluation contains adjudicated expected answers, but a custom correctness scorer receives only the generated response because the answer column was not mapped into its inputs. Why does the current approach fail under these conditions? Citation syntax validity is weaker than provenance integrity; a fabricated identifier can still look well formed. A scorer can misjudge valid traces when its required argument mapping omits available reference data even though the dataset itself is complete. End-to-end generation failure can originate in tools, retrieval or prompts; one judge score cannot localize every cause. MLflow scorers measure distinct attributes; passing one risk dimension does not establish task correctness. ExplanationCorrect Answer B:Map the existing expected_answer column explicitly into the scorer invocation and validate its typed inputs before evaluation. Other Answers A:A scorer can misjudge valid traces when its required argument mapping omits available reference data even though the dataset itself is complete. C:That mechanism describes another problem: An agent trace includes a tool error, yet the evaluation dashboard attributes the bad final response only to an LLM judge score. It does not explain this case, where a scorer can misjudge valid traces when its required argument mapping omits available reference data even though the dataset itself is complete. D:A scorer can misjudge valid traces when its required argument mapping omits available reference data even though the dataset itself is complete.
Question 641 An MLflow evaluation contains adjudicated expected answers, but a custom correctness scorer receives only the generated response because the answer column was not mapped into its inputs. Which observation would establish that the requirement is now met? Evaluate one fabricated and one genuinely grounded citation and inspect scorer feedback per trace. Run traces with known incorrect totals and confirm the numeric scorer flags them even when safety passes. Score one known correct and one known incorrect trace and verify the reference values appear in scorer inputs and recorded feedback. Introduce a mocked failing tool call and verify its span/error is detected independently of the final-answer score. ExplanationCorrect Answer C:A scorer can misjudge valid traces when its required argument mapping omits available reference data even though the dataset itself is complete. Other Answers A:Score one known correct and one known incorrect trace and verify the reference values appear in scorer inputs and recorded feedback. B:Score one known correct and one known incorrect trace and verify the reference values appear in scorer inputs and recorded feedback. D:Score one known correct and one known incorrect trace and verify the reference values appear in scorer inputs and recorded feedback.
Question 642 An MLflow evaluation contains adjudicated expected answers, but a custom correctness scorer receives only the generated response because the answer column was not mapped into its inputs. What lasting control best prevents recurrence of the identified issue? Enforce scorer signature-to-dataset column mapping tests and typed evaluation fixtures on each run. Maintain provenance-aware scorers and negative citation fixtures across retrieval releases. Keep stage-specific structured checks alongside model-based evaluators. Maintain a scorer portfolio aligned to business success, correctness and safety requirements. ExplanationCorrect Answer A:A scorer can misjudge valid traces when its required argument mapping omits available reference data even though the dataset itself is complete. Other Answers B:This control addresses a different exposure: A citation judge passes responses with valid-looking IDs even when those IDs never appeared in the retrieved evidence. It would not prevent this failure, because a scorer can misjudge valid traces when its required argument mapping omits available reference data even though the dataset itself is complete. C:That a scorer can misjudge valid traces when its required argument mapping omits available reference data even though the dataset itself is complete. D:A scorer can misjudge valid traces when its required argument mapping omits available reference data even though the dataset itself is complete.
Question 643 An agent trace includes a tool error, yet the evaluation dashboard attributes the bad final response only to an LLM judge score. What is the most targeted corrective step for this observed failure? Inspect nested trace spans and add code-based checks for tool status and tool-output integrity in the evaluation. Map the existing expected_answer column explicitly into the scorer invocation and validate its typed inputs before evaluation. Inspect per-stage inference records for search timeouts and missing-context flags before changing prompt text. Add a task-specific numeric correctness scorer alongside the built-in safety scorer and inspect results per trace. ExplanationCorrect Answer A:The action targets the stated cause: End-to-end generation failure can originate in tools, retrieval or prompts; one judge score cannot localize every cause. Check the repair by: introduce a mocked failing tool call and verify its span/error is detected independently of the final-answer score. Other Answers B:End-to-end generation failure can originate in tools, retrieval or prompts; one judge score cannot localize every cause. C:End-to-end generation failure can originate in tools, retrieval or prompts; one judge score cannot localize every cause. D:End-to-end generation failure can originate in tools, retrieval or prompts; one judge score cannot localize every cause.
Question 644 User reports say a RAG bot cites the wrong policy, but only final answers are stored and engineers cannot see which version was retrieved. What should the implementation team change first to resolve the described defect? Inspect per-stage inference records for search timeouts and missing-context flags before changing prompt text. Log minimal redacted diagnostics and stable trace/source IDs rather than raw protected customer fields. Inspect nested trace spans and add code-based checks for tool status and tool-output integrity in the evaluation. Enable approved inference/trace logging with query, index revision, retrieved document IDs and citations while redacting sensitive values. ExplanationCorrect Answer D:The action targets the stated cause: Without evidence-version provenance, the team cannot separate retrieval mistakes from answer-generation mistakes. Check the repair by: reproduce a reported answer and find the exact document IDs and index revision in the recorded trace. Other Answers A:Without evidence-version provenance, the team cannot separate retrieval mistakes from answer-generation mistakes. B:Without evidence-version provenance, the team cannot separate retrieval mistakes from answer-generation mistakes. C:Without evidence-version provenance, the team cannot separate retrieval mistakes from answer-generation mistakes.
Question 645 User reports say a RAG bot cites the wrong policy, but only final answers are stored and engineers cannot see which version was retrieved. Why does the current approach fail under these conditions? Request totals alone cannot explain tail latency linked to prompt length, retries or endpoint errors. Observability pipelines retain data independently from serving and must meet the same access and privacy requirements. Without evidence-version provenance, the team cannot separate retrieval mistakes from answer-generation mistakes. A generator can complete successfully after an upstream retrieval failure, so final model status alone hides the incident. ExplanationCorrect Answer C:Enable approved inference/trace logging with query, index revision, retrieved document IDs and citations while redacting sensitive values. Other Answers A:Without evidence-version provenance, the team cannot separate retrieval mistakes from answer-generation mistakes. B:Without evidence-version provenance, the team cannot separate retrieval mistakes from answer-generation mistakes. D:That mechanism describes another problem: A production service intermittently returns empty context, yet model completion traces show a normal success status. It does not explain this case, where without evidence-version provenance, the team cannot separate retrieval mistakes from answer-generation mistakes.
Question 646 User reports say a RAG bot cites the wrong policy, but only final answers are stored and engineers cannot see which version was retrieved. Which observation would establish that the requirement is now met? Correlate slow trace IDs with tokens, stage timings and endpoint status across the hourly spike. Examine inference tables for synthetic sensitive identifiers and verify retrieval debugging remains possible using safe IDs. Reproduce a reported answer and find the exact document IDs and index revision in the recorded trace. Correlate empty-answer cases with retrieval latency and tool error events in the same trace ID. ExplanationCorrect Answer C:Without evidence-version provenance, the team cannot separate retrieval mistakes from answer-generation mistakes. Other Answers A:Reproduce a reported answer and find the exact document IDs and index revision in the recorded trace. B:Reproduce a reported answer and find the exact document IDs and index revision in the recorded trace. D:Reproduce a reported answer and find the exact document IDs and index revision in the recorded trace.
Question 647 User reports say a RAG bot cites the wrong policy, but only final answers are stored and engineers cannot see which version was retrieved. What lasting control best prevents recurrence of the identified issue? Monitor required context availability and define fallback behavior for upstream search errors. Set telemetry retention and completeness tests for supported endpoint observability. Apply field-level retention, access policies and redaction validation to inference telemetry. Require source-version metadata and sanitized retrieval-span logging for grounded applications. ExplanationCorrect Answer D:Without evidence-version provenance, the team cannot separate retrieval mistakes from answer-generation mistakes. Other Answers A:Without evidence-version provenance, the team cannot separate retrieval mistakes from answer-generation mistakes. B:This control addresses a different exposure: Production model serving latency spikes every hour, but the team records only the number of requests and has no payload-size or stage-level diagnostic signal. It would not prevent this failure, because without evidence-version provenance, the team cannot separate retrieval mistakes from answer-generation mistakes. C:That without evidence-version provenance, the team cannot separate retrieval mistakes from answer-generation mistakes.
Question 648 A production service intermittently returns empty context, yet model completion traces show a normal success status. What is the most targeted corrective step for this observed failure? Enable approved inference/trace logging with query, index revision, retrieved document IDs and citations while redacting sensitive values. Enable approved inference telemetry capturing timestamps, latency, token counts, status and trace IDs while respecting privacy constraints. Log minimal redacted diagnostics and stable trace/source IDs rather than raw protected customer fields. Inspect per-stage inference records for search timeouts and missing-context flags before changing prompt text. ExplanationCorrect Answer D:The action targets the stated cause: A generator can complete successfully after an upstream retrieval failure, so final model status alone hides the incident. Check the repair by: correlate empty-answer cases with retrieval latency and tool error events in the same trace ID. Other Answers A:A generator can complete successfully after an upstream retrieval failure, so final model status alone hides the incident. B:A generator can complete successfully after an upstream retrieval failure, so final model status alone hides the incident. C:A generator can complete successfully after an upstream retrieval failure, so final model status alone hides the incident.
Question 649 A production service intermittently returns empty context, yet model completion traces show a normal success status. What is the decisive mechanism behind the observed result? A generator can complete successfully after an upstream retrieval failure, so final model status alone hides the incident. End-to-end generation failure can originate in tools, retrieval or prompts; one judge score cannot localize every cause. Without evidence-version provenance, the team cannot separate retrieval mistakes from answer-generation mistakes. Observability pipelines retain data independently from serving and must meet the same access and privacy requirements. ExplanationCorrect Answer A:Inspect per-stage inference records for search timeouts and missing-context flags before changing prompt text. Other Answers B:A generator can complete successfully after an upstream retrieval failure, so final model status alone hides the incident. C:That mechanism describes another problem: User reports say a RAG bot cites the wrong policy, but only final answers are stored and engineers cannot see which version was retrieved. It does not explain this case, where a generator can complete successfully after an upstream retrieval failure, so final model status alone hides the incident. D:A generator can complete successfully after an upstream retrieval failure, so final model status alone hides the incident.
Question 650 A production service intermittently returns empty context, yet model completion traces show a normal success status. Which verification is strongest for this particular failure mode? Examine inference tables for synthetic sensitive identifiers and verify retrieval debugging remains possible using safe IDs. Correlate empty-answer cases with retrieval latency and tool error events in the same trace ID. Reproduce a reported answer and find the exact document IDs and index revision in the recorded trace. Introduce a mocked failing tool call and verify its span/error is detected independently of the final-answer score. ExplanationCorrect Answer B:A generator can complete successfully after an upstream retrieval failure, so final model status alone hides the incident. Other Answers A:Correlate empty-answer cases with retrieval latency and tool error events in the same trace ID. C:Correlate empty-answer cases with retrieval latency and tool error events in the same trace ID. D:Correlate empty-answer cases with retrieval latency and tool error events in the same trace ID.
