NanoEuler
An open-source GPT-2-style language model written from scratch in C/CUDA, with a BPE tokenizer, FlashAttention, pretraining and SFT, useful for learning how LLMs work.
Open sourceCoding & Development
Visit website(opens in a new tab)
An open-source GPT-2-style language model written from scratch in C/CUDA, with a BPE tokenizer, FlashAttention, pretraining and SFT, useful for learning how LLMs work.