Token Tokens and Embeddings, Explained Simply A model doesn't read words. It reads tokens: pieces of text that can be a character, part of a word, a whole word, or punctuation. Spaces matter too, so "red", "Red", and " red" can all become different tokens (OpenAI, n.d.-a)
