Andrej Karpathy's explaination of Bigram Language Model explained Bigram LMs, though simple, it provides powerful insight into the inner mechanics of how tokens are processed in language models. This is a pre-amble for what's next: GPT, which is the 2nd part of the series The Bigram model here incorporates: Tokenization, Vocabulary, Negative Loss Function, Cross Entropy, Logits, SoftMax, Optimizer, and AdamW These are essential ingredients to understand in order to build our knowledge on how LLM