NanoChat Alpaca (Partial Fine-tuning)
Partially fine-tuned version of nanochat-speedrun-001 on the Alpaca instruction dataset.
Model Details
- Base Model: royam0820/nanochat-speedrun-001
- Fine-tuning Dataset: tatsu-lab/alpaca (52,002 examples)
- Parameters: 561M
- Vocab Size: 65,536
Training
- Epochs: 3
- Learning Rate: 1e-5
- Loss: 3.83 → 3.24 → 2.95
Status
⚠️ Partially trained - produces repetitive outputs. Needs more training.
Files
- checkpoint_1.pt, checkpoint_2.pt, checkpoint_3.pt
- tokenizer.pkl, token_bytes.pt