Question 320 An agent has a new response-format requirement: a valid JSON object. Which evaluation is necessary? Inspect only whether braces appear Machine-parse outputs and check required fields and types Ignore malformed outputs if answers seem correct Score the response as prose poetry Explanation Correct Answer B:A fluent-looking response may still break JSON consumers. Incorrect Answers A:Braces do not guarantee valid structured output. C:Invalid JSON can still break the application. D:Literary quality doesn't test schema compliance.
Question 321 An HR assistant is tested with outdated leave policy documents. What dataset flaw must be corrected? Refresh the authoritative test references to the effective policy version Ask the model to ignore all policy dates Increase temperature to compensate for stale references Remove all policy tests Explanation Correct Answer A:Expected answers must match the current source of truth. Incorrect Answers B:Undated references can validate wrong guidance. C:Randomness doesn't update the source. D:That would eliminate the important workload.
Question 322 An agent connects to several services. How can evaluation identify a particular broken dependency? Record tool calls, statuses and correlated traces for failed tests Suppress all tool invocation logs Assume every failure is caused by hallucination Check only whether the conversation starts and ends with a friendly message Explanation Correct Answer A:Linked observations distinguish model errors from API failures. Incorrect Answers B:Without traces attribution becomes difficult. C:Technical failures have distinct causes. D:Polite message boundaries don't identify the failing API dependency.
Question 323 A team sees a high average satisfaction score but severe failures on rare accessibility requests. What evaluation design helps? Segment metrics and include representative high-impact minority cases Only optimize the average score Remove minority scenarios from the test set Count emojis in satisfied users' messages Explanation Correct Answer A:Averages can conceal critical subgroup regressions. Incorrect Answers B:Global averages may not protect high-impact uses. C:That hides the failures rather than fixing them. D:Emoji presence doesn't measure accessibility correctness.
Question 324 An agent retrieves a correct policy but formats a calculation incorrectly. Which independent evaluation dimensions matter? Check factual retrieval and numerical/format correctness separately Skip format checks because policy is right Remove the calculation from the output silently Treat an accurate citation as proof of all calculations Explanation Correct Answer A:Good grounding doesn't guarantee correct arithmetic or formatting. Incorrect Answers B:Downstream readers may misunderstand malformed numbers. C:The user requested an actual amount. D:Evidence retrieval and numeric transformation are different.
Question 325 A quality team wants a measurable release threshold for correct tool selection. What should it define? A requirement to use every tool at least once A threshold based only on answer length A subjective rule that the agent feels smart A scored set of intents with expected tools and permitted alternatives Explanation Correct Answer D:A benchmark can measure routing behavior consistently. Incorrect Answers A:Excess calls aren't a sign of appropriate selection. B:Length does not establish correct routing. C:Feelings cannot quantify tool selection.
Question 326 A new model improves average helpfulness but fails more often on customer privacy. How should the release decision weigh this? Hide failed privacy tests in an appendix Approve solely because aggregate helpfulness improved Treat privacy failures as potentially blocking regardless of average gains Reduce the weight of all privacy tests to zero Explanation Correct Answer C:Critical safety criteria should not be traded away silently for average quality. Incorrect Answers A:Concealment doesn't resolve the defect. B:Averages can hide unacceptable privacy regressions. D:This undermines the stated security requirement.
Question 327 A support agent's behavior is evaluated only by its author. What limitation should the report state? Say every deployment is guaranteed correct Claim that Microsoft certified each answer The results are AI-assisted or author-reviewed rather than independent expert certification Omit all reviewer attribution Explanation Correct Answer C:Review provenance affects how confidently results can be represented. Incorrect Answers A:Tests cannot cover every possible environment. B:Issuer endorsement cannot be inferred from local tests. D:Missing attribution obscures assurance limits.
Question 328 An agent is asked to find a record that does not exist. Which evaluation expectation is appropriate? The last successful customer's record A forced success message without an ID A clear not-found response with no fabricated record details A plausible fabricated record to keep the user happy Explanation Correct Answer C:Negative cases should test whether the agent resists invention. Incorrect Answers A:That leaks another user's information. B:No authoritative match has been found. D:False records are not acceptable.
Question 329 An agent handles 90% of requests quickly but some take a minute. Which metric reveals tail latency? The 95th or 99th percentile response duration by scenario Only number of test cases Only minimum response time Only average message length Explanation Correct Answer A:Tail percentiles expose slow requests hidden by averages. Incorrect Answers B:Count cannot represent duration distribution. C:The fastest case doesn't describe slow outliers. D:Text length isn't an end-to-end latency measure.
Question 330 An evaluation identifies a tool that sometimes calls DELETE rather than GET. What is the severity? A cosmetic UI issue Treat the incorrect DELETE call as a formatting defect and skip side-effect checks A potentially critical action-selection and permission defect requiring a release block A harmless wording difference Explanation Correct Answer C:A wrong HTTP method can change or destroy records. Incorrect Answers A:The defect concerns tool execution, not style. B:An unintended DELETE can destroy external records regardless of message tone. D:DELETE may alter external state.