LLM foundations quiz

Choose one answer.

• The logits are decoded straight into the full answer text, since the final layer already holds the whole sequence.
• Softmax gives probabilities, one token is chosen and appended, and the loop repeats until a stop token or a length limit.
• The prompt is re-tokenized with the logits as extra context, and the model runs once more to refine the whole answer in place.
• The highest-scoring token is always taken, because softmax only ranks tokens and never produces probabilities to sample from.

Choose one answer.

• They are handled by separate modules that run before the transformer sees the text.
• The model switches to a rules engine whenever it sees a tool call, then switches back to generating text.
• They are structure formatted into the same token sequence, so the same next-token loop handles them.

Choose one answer.

• The model drafted the whole reply internally first, and a final check pass missed the invented name before streaming began.
• The softmax step returned a probability for a token outside the vocabulary, and that token was rendered as an invented name.
• Each step scored the next token from the text so far and one was chosen, with nothing in the loop checking it against a source.
• The sampled token was swapped for a random one whenever the model's confidence for that position dropped below a fixed threshold.

Choose one answer.

• Attention uses fewer arithmetic operations per token than an RNN at every sequence length, so it is simply cheaper.
• RNNs cannot process text longer than a few dozen tokens, whereas attention has no length limit at all.
• Positions are processed at once in training, and any earlier token can be read directly.
• Attention removes the need for tokenization and embeddings, so the input pipeline is simpler.

Choose one answer.

• An RNN must process steps in order, so training cannot parallelize across the sequence, and far-back information is squeezed through a fixed-size state.
• RNNs cannot be trained with gradient descent.
• RNNs need a larger vocabulary than transformers.

Choose one answer.

• Every token attends to every other token, including ones that come later in the sequence.
• Each token attends to itself and earlier tokens only; later positions are masked out.
• Each token attends only to the immediately previous token, like an RNN.

Choose one answer.

• It stays about the same, because attention is parallel.
• It doubles.
• It roughly quadruples.
• It grows by a factor of eight.

Choose one answer.

• Query is compared with Values; the resulting weights blend the Keys.
• Query is compared with the previous hidden state; the weights blend the embeddings.
• Query is compared with Keys; the weights blend the Values.

Choose all that apply.

• It stores each past token's Keys and Values so they are not recomputed on every decode step.
• It returns a stored answer when the same question is asked again.
• Its memory grows with context length and with the number of concurrent requests.
• On a brand-new prompt with no cached prefix, it speeds up prefill by skipping the prompt tokens.

Choose one answer.

• 256 MiB
• 128 KiB
• 256 GiB
• 1 GiB

Choose one answer.

• Decode, because each new token needs far more computation than the one before it, so the delay builds up at the start.
• Prefill, where the whole prompt is processed before the first token can be produced.
• Sampling, because longer prompts make top-p filtering slower for the first token.

Choose one answer.

• It replaces the word with a single unknown-word token, because the vocabulary only contains words seen in training, so the meaning is lost.
• It adds the new word to the vocabulary on the fly so the next request tokenizes it as one piece.
• It splits the word into smaller known pieces, down to single bytes if needed.

Choose one answer.

• By repeatedly merging the most frequent adjacent pair of symbols in a training corpus.
• By listing the most frequent whole words in a large dictionary and giving each an ID.
• By having the transformer discover good splits while it trains on next-token prediction.
• By splitting on whitespace and punctuation only, then lower-casing the pieces.

Choose one answer.

• Merges learned from frequent pairs in training text, so common pairs became units and rare strings fell back to pieces.
• A dictionary of whole words that the tokenizer looks up first, cutting at the longest entry and then splitting the remainder by syllable.
• The attention layers, which read the whole name in context at run time and decide where each unit of meaning ends.
• A fixed rule that cuts every word after a set number of characters, so each piece is the same length whatever the text.

Choose one answer.

• None, since tokenizers treat all languages alike and the ratio carries over.
• The estimate will be too high, since non-English text is shorter once translated.
• It may be far too low, since some languages need many more tokens.

Choose one answer.

• Embedding vectors cannot represent digits, because digits are not part of any vocabulary.
• Softmax rounds probabilities, and that rounding breaks arithmetic on the input.
• Tokens are subword pieces, not letters or digits.

Choose one answer.

• Nothing, because the provider charges per call as well.
• Your cost per call rises with token count while your revenue stays flat, so margins can turn negative; price or cap by tokens instead.
• Only latency changes, because cost depends on the number of words and not on tokens.

Choose one answer.

• One hand-designed vector per word, defined by linguists and frozen.
• One learned vector per token, trained jointly with the rest of the model.
• One vector per character, computed from the character's Unicode value.

Choose one answer.

• Sequences get longer, but the model gets smaller.
• Nothing changes except the vocabulary file, since embeddings are looked up lazily.
• Fewer tokens per text, but a larger embedding table and output layer.

Choose one answer.

• The count is only an estimate; the model's own tokenizer gives the number you are limited by.
• The counts are identical because all modern tokenizers use BPE and so share the same merges.
• The OpenAI count is always an upper bound, because its vocabulary is larger, so it is safe.

Choose one answer.

• Keep the existing token counts, since the text did not change.
• Multiply old counts by the ratio of the two vocabulary sizes, which predicts how many tokens the new model will use.
• Re-measure real traffic with the new tokenizer and update budgets.

Choose one answer.

• It becomes flatter, so lower-ranked tokens get chosen more often.
• It becomes sharper, so the top token is chosen more often.
• The ranking of tokens changes, so a different token becomes the most likely.

Choose one answer.

• All three, because top-p never removes tokens.
• The first two, since top-p always keeps at least two tokens.
• Only the first token, because it alone already exceeds 0.9.

Choose one answer.

• Top-k keeps a fixed count no matter how the probabilities look, while top-p keeps however many tokens are needed to reach the probability mass.
• Top-p changes the temperature automatically.
• Top-p can pick tokens outside the model's vocabulary.

Choose one answer.

• The provider changed the model's weights between the calls, since a fixed set of weights would return identical text for identical input.
• The server is dropping part of the prompt under load, so each call answers a slightly different question from the one you sent.
• The KV-cache is returning stored answers that belong to earlier users, which explains why the wording changes from call to call.
• Decoding samples from a distribution over next tokens, so identical prompts can diverge, and a lower temperature makes this rarer.

Choose one answer.

• High temperature for both, so the model explores more phrasings and is more thorough on the extraction too.
• Low temperature for extraction; higher temperature for subject lines.
• Low temperature for both, since lower is always better and variety only adds risk.
• Low temperature for subject lines and high for extraction, since extraction is harder.

Choose one answer.

• Fine; more controls always give finer control.
• Always set top-k to 1.
• Change one control at a time and measure on a fixed evaluation set.

Choose one answer.

• Temperature 0 is only a hint that the model may ignore when it is uncertain about the next token.
• The model's weights are updated slightly after each request as it learns from live traffic.
• The provider adds random noise to outputs on purpose, to make the model harder to copy.
• Greedy decoding is repeatable in theory, but server-side factors such as request batching can still change results.

Choose all that apply.

• Store the exact output that was shown to the user instead of expecting to regenerate it.
• Rely on temperature 0 alone so audits can replay the call later.
• Constrain the output with a schema and validate it, and compare extracted fields instead of exact prose.
• Assume that setting a seed always makes the output identical.

Choose one answer.

• Choose open-weight, because it is always cheaper once you count GPU hours against per-token API prices.
• Choose the API, because API-only models are always better and the team is too small to run anything itself.
• List criteria, then run both options on a small evaluation set of real examples.

Choose one answer.

• You can download the weights and run or fine-tune them yourself, but you take on serving and operations.
• Open-weight always means open source with no usage restrictions.
• They cannot be fine-tuned.

Choose one answer.

• The only downside is per-token cost; models use every position in a long input equally well.
• Retrieval is only needed when the knowledge base is larger than the window, so one that fits can be pasted in.
• Every request pays for all those tokens, and long inputs are used unevenly, so retrieval still helps.