Question 310 An agent usually answers refund questions correctly but occasionally invents exceptions. Which quality dimension should be measured? Grounded factuality and policy consistency on adversarial refund cases Only response character count Only startup animation speed Only number of published channels Explanation Correct Answer A:The defect is unsupported policy content rather than UI rendering. Incorrect Answers B:Length does not determine policy accuracy. C:Visual latency cannot reveal unsupported exceptions. D:Channel count is unrelated to invented rules.
Question 311 A team wants to compare an agent before and after changing its prompt. Which testing design is valid? Judge the new build only by enthusiasm Delete the old results before comparing Use different easier questions for the new build Use the same versioned test cases and scoring rubric for both builds Explanation Correct Answer D:Controlled comparisons attribute changes to the prompt revision. Incorrect Answers A:A subjective feeling is not a stable metric. B:Without a baseline there is no meaningful before/after measurement. C:Changed difficulty biases the comparison.
Question 312 An evaluation set covers only one happy-path order lookup. What gap should be addressed? Remove all expected-result assertions Declare the agent production-safe from one example Add invalid identifiers, authorization denial and backend outage cases Increase the number of exact identical happy-path copies Explanation Correct Answer C:Robustness requires testing expected failures and edge conditions. Incorrect Answers A:Assertions are necessary to determine correctness. B:One success does not validate critical behavior. D:Duplicates do not add failure-mode coverage.
Question 313 A model-generated financial answer is fluent but uses a fabricated exchange rate. Which evaluation should catch this? Check only that the answer contains a currency symbol Score only grammar and politeness Check factual values against a trusted dated exchange-rate source Assume the newest model always knows today's rate Explanation Correct Answer C:Correctness requires evidence for time-sensitive financial quantities. Incorrect Answers A:A symbol does not verify the rate. B:Good prose can contain wrong numbers. D:Training data and live rates may differ.
Question 314 An agent reads confidential records correctly for an administrator. Which test is required before release? Only repeat the admin queries with different wording Test whether the UI hides sensitive rows from an administrator Share the administrator token with the test group Run the same queries as an unauthorized ordinary user and verify access denial Explanation Correct Answer D:Admin success cannot establish least-privilege behavior. Incorrect Answers A:Authorized tests do not reveal permission leaks. B:Hiding display elements does not prove backend access denial for ordinary users. C:That eliminates the permission contrast.
Question 315 An external API tool returns 503 during a test. What expected agent behavior should be asserted? Generate a plausible confirmation number Silently omit the error and say completed Repeat writes forever until an uncertain result appears Report dependency unavailability without claiming the action succeeded Explanation Correct Answer D:Failure messaging is part of functional correctness. Incorrect Answers A:A made-up identifier is not a completed transaction. B:This misrepresents the external state. C:Continuously repeating a purchase write after a timeout can submit multiple transactions unless an idempotency or reconciliation mechanism prevents duplicates.
Question 316 A multi-agent evaluation reveals the main agent always delegates trivial FAQ questions to a costly specialist. What metric can identify the problem? Unnecessary delegation rate and tool usage per intent category Count of characters in each agent's name Total size of the FAQ source files only Number of icons in the chat UI Explanation Correct Answer A:Over-orchestration increases cost and latency without adding value. Incorrect Answers B:Names do not indicate routing efficiency. C:File size alone does not explain specialist calls. D:Icons cannot measure tool selection.
Question 317 An agent produces citations, but the cited documents do not support the claims. What should the evaluation rubric include? Only presence of any hyperlink Only alphabetical order of citations Score citation length and position without checking if the cited page supports the claim Claim-to-source entailment and accessible citation verification Explanation Correct Answer D:A valid URL is not necessarily evidence for a particular claim. Incorrect Answers A:A link can be unrelated or inaccessible. B:Ordering does not establish support. C:Citation aesthetics do not establish evidentiary entailment.
Question 318 A customer agent fails on a question combining product return and damaged shipment. What test category is missing? A faster agent initialization test More duplicate one-intent greetings Add only more one-intent return requests without damaged-shipment context Multi-intent robustness cases with explicit expected handling of both issues Explanation Correct Answer D:Real users can express multiple tasks in one utterance. Incorrect Answers A:Startup speed does not assess multi-intent reasoning. B:Repetition doesn't add combined-intent coverage. C:The combined-intent routing defect would remain untested.
Question 319 A team wants to test an agent's behavior when retrieved pages contain hostile instructions. What scenario should be added? A case that rewards obeying source instructions A routine spelling quiz without external content Prompt-injection cases where untrusted content requests unauthorized tool actions An admin-only test with all policies disabled Explanation Correct Answer C:The agent must preserve trust boundaries under adversarial retrieval. Incorrect Answers A:That validates the unsafe behavior instead. B:That doesn't exercise injection handling. D:An unrestricted account conceals escalation risks.
Question 320 An agent has a new response-format requirement: a valid JSON object. Which evaluation is necessary? Inspect only whether braces appear Machine-parse outputs and check required fields and types Ignore malformed outputs if answers seem correct Score the response as prose poetry Explanation Correct Answer B:A fluent-looking response may still break JSON consumers. Incorrect Answers A:Braces do not guarantee valid structured output. C:Invalid JSON can still break the application. D:Literary quality doesn't test schema compliance.