Andrej Karpathy's explaination of Bigram Language Model explained Bigram LMs, though simple, it provides powerful insight into the inner mechanics of how tokens are processed in language models. This is a pre-amble for what's next: GPT, which is the 2nd part of the series The Bigram model here incorporates: Tokenization, Vocabulary, Negative Loss Function, Cross Entropy, Logits, SoftMax, Optimizer, and AdamW These are essential ingredients to understand in order to build our knowledge on how LLM
8:23
Andrej Karpathy's explaination of Bigram Language Model explained Bigram LMs, though simple, it provides powerful insight into the inner mechanics of how tokens are processed in language models. This is a pre-amble for what's next: GPT, which is the 2nd part of the series The Bigram model here incorporates: Tokenization, Vocabulary, Negative Loss Function, Cross Entropy, Logits, SoftMax, Optimizer, and AdamW These are essential ingredients to understand in order to build our knowledge on how LLM
149.7K views1 month ago
TikTokcalebwritescode
See more
Static thumbnail place holder