Traves/MicroGPT-Go

MicroGPT - 1500 Step Checkpoint

This is a local training checkpoint for MicroGPT (Go Edition). It features a compact transformer architecture trained on a specific dataset using the microgpt Go runtime.

Model Details

• Architecture: Transformer
• Layers: 2
• Embedding Dimensions: 80
• Heads: 4
• Block Size: 128
• Tokenization: BPE (cl100k_base)
• Training Steps: 1500
• Validation Loss: 4.4379

Usage

This model is designed to be run with the MicroGPT Go Edition engine.

Loading the Model

1. Download latest_checkpoint.json from this repository.
2. Place it in your go/models/ directory.
3. Use the MicroGPT CLI or TUI to load the model:

go run . chat-once models/latest_checkpoint.json "Your prompt here"

Training Data

The model was trained using the databricks-dolly-15k.jsonl dataset (or variant) as configured in the training hub.

Attribution

Built on the foundational ideas of Andrej Karpathy.