Logit

Logits are the raw, unnormalized scores output by a neural network's final layer before softmax converts them into probabilities. For language models, the output layer produces one logit per token in the vocabulary, where higher logits indicate higher model confidence that the corresponding token should come next. The term 'logit' comes from statistics, referring to the log-odds of an event. In neural networks, logits are the pre-activation values that softmax exponentiates and normalizes. Converting logits to probabilities: p(token_i) = exp(logit_i) / sum(exp(all_logits)). The relative differences between logits matter more than absolute values since softmax normalizes everything. Logit manipulation enables fine-grained control over model behavior. Classifier-free guidance in diffusion models works by scaling logits. Constitutional AI modifies logits to discourage harmful outputs. Logit bias in API calls increases or decreases the probability of specific tokens. Temperature scaling divides logits before softmax: higher temperature flattens the distribution (more randomness), lower temperature sharpens it (more determinism). Understanding logits is essential for advanced prompting techniques, model interpretability, and controlling generation behavior beyond simple sampling parameters.