Menu

GitHub - angelos-p/llm-from-scratch
πŸ“°
0

GitHub - angelos-p/llm-from-scratch

GitHubΒ·angelos-pΒ·28 days ago
#UsjkHxQc
Reading 0:00
15s threshold

Train Your Own LLM From Scratch A hands-on workshop where you write every piece of a GPT training pipeline yourself, understanding what each component does and why. Andrej Karpathy's nanoGPT was my first real exposure to LLMs and transformers. Seeing how a working language model could be built in a few hundred lines of PyTorch completely changed how I thought about AI and inspired me to go deeper into the space. This workshop is my attempt to give others that same experience. nanoGPT targets reproducing GPT-2 (124M params) and covers a lot of ground. This project strips it down to the essentials and scales it to a ~10M param model that trains on a laptop in under an hour β€” designed to be completed in a single workshop session. No black-box libraries. No model = AutoModel.from_pretrained() . You build it all. What You'll Build A working GPT model trained from scratch on your MacBook, capable of generating Shakespeare-like text.…

Continue reading β€” create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.

Read More