Softmax function

Softmax is a function that turns a set of scores into probabilities. In an LLM, the model calculates a score for every possible next token. These scores are called logits. A higher score means the…