Context, memory and multi-agent systems quiz

Choose one answer.

• Compress; the notes may drop a detail that only the full history contained.
• Write; notes the agent never reads again add cost and clutter without helping.
• Select; the agent may pull in the wrong notes and miss the fact that mattered to the task.
• Isolate; the notes need a hand-off step between two separate agents that would share them.

Choose one answer.

• The sub-agent should have received the lead's full history, so that it could judge what the lead needs.
• Nothing is wrong; a longer return is safer because the lead can then check each of the searches.
• It should have returned a short brief, since the saving comes from handing back conclusions only.
• The lead should have truncated the log to its first 500 words before reading it into its window.

Choose one answer.

• The full transcript of the shift, because anything left out might be the one detail the next agent turns out to need.
• One paragraph describing what happened overall, since a shorter summary keeps the next agent's window as clean as possible.
• A short note with open tasks, decisions and the must-keep rules word for word, plus a pointer to where the full record is stored.
• Only the last ten turns of the shift, since the most recent messages are the ones that describe the current state of the work.

Choose one answer.

• System rules and tools, then reference documents, then the conversation, with the new user message last.
• The new user message first, then the system rules and tools, so the freshest text is read before anything else.
• Tool definitions re-sorted per request by relevance to the question, then the system rules, then the conversation.
• Today's date and the user's name first, then the system rules and tool definitions, then the conversation.

Choose one answer.

• Caching stores finished answers, so a changed tool list means every stored answer is now considered out of date.
• Tool definitions open the prefix, so changing them invalidates that level and every level cached after it.
• Only the system prompt is cached, so the tool list should not matter and the drop must be a provider fault.
• Caching switches itself off once a prompt grows past a size limit, and the extra tool pushed it over.

Choose one answer.

• Working memory, kept in the context window so that it is always visible to the model.
• Episodic memory, stored as a timestamped summary of the chat where the user said it.
• Procedural memory, saved as an instruction file that tells the agent how to plan meals for the user.
• Semantic memory, kept as a profile or key-value entry that can later be updated or deleted.

Choose one answer.

• Raise the number of recalled memories, so that the newer address is also likely to appear in the prompt.
• Score recall by recency and importance as well as similarity, and keep timestamps to resolve conflicts.
• Lower the sampling temperature, so that the model stops picking an old address from the recalled memories.
• Switch to a larger embedding model, so that similarity can tell the old address from the new one.

Choose one answer.

• Key every record, summary and embedding to a user id so a single delete removes them all across stores.
• Tell the model in the next system prompt to forget the user, then keep the stored memories for audit.
• Give every memory a short expiry time, so the data disappears without the need for any per-user deletion.
• Delete the raw chat records, since summaries and embeddings are only derived copies that are safe to keep.

Choose one answer.

• Overflow; count truncated or rejected requests against the window limit on each step of the run.
• Distraction; compare accuracy on a fixed task set with and without the extra context added.
• Tool-output bloat; track the share of tokens that come from tool results on each step.
• Drift; ask a probe question about the pinned rule after each compaction and check the answer.

Choose one answer.

• Change the tool to return only the needed fields, then track the share of tokens that come from tool results.
• Move to a model with a larger window, since the pages then fit and nothing gets cut.
• Add a line to the system prompt telling the model to ignore the markup and read only the visible text of the page.
• Put the search tool last in the tool list, so that the cached prefix stays the same for other tools.

Choose one answer.

• Re-rank the retrieved text by length so shorter passages come first, and track answer accuracy on a fixed task set.
• Retrieve more passages per step so the agent finds answers sooner, and track retrieval precision for each query.
• Clear or compress old tool results as the run goes, and track tokens per step against the window limit.
• Pin the task goal verbatim outside the history, and run a probe question about it after each compaction.

Choose one answer.

• A single agent with tools, because it can decide for itself how many steps each email needs.
• A multi-agent team with a reader, an extractor and a router agent, so that each role stays specialised.
• A workflow chaining an extraction call and a routing call, since the steps and order are fixed.

Choose one answer.

• Researching forty independent companies in parallel and merging short briefs into one comparison report.
• Fixing one bug in one file, where each edit depends on the result of the previous edit and test run.
• Answering employees' questions from one short policy document that fits comfortably inside a single prompt.
• Reformatting the same invoice layout into a standard template that the finance team has already defined.

Choose one answer.

• Add a third reviewer agent that reads both agents' output the next morning and flags duplicate refunds for a person to fix.
• Give each task one owner through a task contract, and record claimed and finished work in shared state that both agents check first.
• Lower each agent's token budget so that neither has room to finish a refund before the other agent has had its turn.
• Give both agents the same long prompt describing every task, so that each one knows the whole job and chooses better.

Choose one answer.

• Expect about 90%, because a chain is as reliable as its weakest step, so the single autonomous design can ship.
• Expect about 43%, because 0.9 to the power 8 is 0.43, so shorten the chain, add checks or use a fixed workflow.
• Expect about 20%, because eight steps that each lose 10 points remove 80 points, so add more agents to share the load.

Choose one answer.

• Design X, because a rate near 0.98 per step shows the design is more capable overall, which outweighs the length of the chain.
• Neither can be compared, because real steps are not fully independent and the formula gives no guidance.
• Design Y, because 0.95 to the power 5 is about 0.77 and 0.98 to the power 20 is about 0.67.

Choose one answer.

• Ask an LLM judge whether the final message sounds confident and politely confirms the refund to the customer.
• Confirm that the agent called the refund tool during the run, as the trace records, with no error shown.
• Compare the final message with a reference answer written in advance by the team for this task.
• Query the payments database and check exactly one refund exists, for the right order and the right amount.

Choose one answer.

• That run B cost far more and took on more risk, which trajectory metrics such as steps and errors reveal.
• Little, since a matching final state means the two runs have the same quality for the product.
• That run B should be scored as a failure, because retried tool calls count against the agent.
• That trajectory checks should replace state checks, since the path reveals much more than the final state does.

Choose one answer.

• pass@3 is about 0.73 and pass^3 is about 0.999, because pass@3 requires the three runs to pass on this one task.
• pass@3 is about 0.999 and pass^3 is about 0.73, because one needs a single success and the other needs three.
• Both are about 0.9, since repeating a task does not change how likely it is to pass on a single run.

Choose one answer.

• Use pass@k for both, since a high number looks best and either product is able to report it fairly.
• Use pass^k for both, since the stricter number is the safest one to report for a product.
• Use pass@k for the refund agent and pass^k for the patch tool, matching the number to the stakes involved.
• Use pass^k for the refund agent and pass@k for the patch tool, matching each to how success is used.

Choose one answer.

• Both rise, because more runs give the agent more chances to show what it can do on that task.
• Both fall, because more runs expose more of the failures that a short test with few runs hid.
• pass@k rises or holds while pass^k falls, because one needs a single success and the other needs all.
• pass^k rises, because averaging over more runs smooths the noise and reveals the true underlying rate.

Choose one answer.

• Ship once the average score across the runs reaches 90% on tasks the team chose after studying the failures.
• Ship if it meets targets fixed before testing: a pass^k floor, a clean safety result and a cost cap.
• Ship once the agent has processed a thousand tasks in staging without a crash or timeout.
• Ship once an LLM judge's mean score tops 0.8, with no human reading of the sampled outputs.

Choose one answer.

• A person approves the refund before the step runs, with the order and amount shown for review.
• A per-run token budget that cuts off a long run before it spends more than the team had allowed.
• A kill switch the on-call engineer can flip once someone notices that the agent is misbehaving.
• A system-prompt line telling the model to double-check the amount carefully before it sends a refund.