Running on CPU Upgrade 219 The Synthetic Data Playbook: Generating Trillions of the Finest Tokens π 219 Explore synthetic data experiments on a virtual bookshelf
view article Article The 1 Billion Token Challenge: Finding the Perfect Pre-training Mix Nov 3, 2025 β’ 65
Running Featured 1.33k FineWeb: decanting the web for the finest text data at scale π· 1.33k Read a detailed overview of the FineWeb webβscale text dataset
Running 3.78k The Ultra-Scale Playbook π 3.78k The ultimate guide to training LLM on large GPU Clusters
The Instruction Gap: LLMs get lost in Following Instruction Paper β’ 2601.03269 β’ Published Dec 19, 2025 β’ 8
Running on CPU Upgrade Featured 3.1k The Smol Training Playbook π 3.1k The secrets to building world-class LLMs
view reply You don't really have to clone the repo. The FastAPI code is just there for demonstration, and you can code the way you like. The main takeaway is the Dockerfile.
view article Article How to generate text: using different decoding methods for language generation with Transformers Mar 1, 2020 β’ 294
Komodo: A Linguistic Expedition into Indonesia's Regional Languages Paper β’ 2403.09362 β’ Published Mar 14, 2024 β’ 11