← Back to Benchmarks

Lattice Mini

42M parameters, trained from scratch. We don't run formal benchmark suites on it — and that's a deliberate choice.

What we measure instead

~42M

params — 12-layer GPT decoder, 512-token context

~8k

custom BPE vocab, trained on the corpus

2-stage

WikiText-2 pretraining → Alpaca instruction tuning

Mini exists to prove the full from-scratch pipeline works — custom tokenizer, pretraining, instruction tuning — on hardware that fits in a backpack. At 42M params, formal suites like MMLU don't tell you anything useful; the honest metric is "does it hold a conversation, and is it honest about the limits?"

Expect wrong answers. Mini's chat is a research demo. Repetition and confident nonsense are features at this scale — they're what you see when a model has too few weights and too little data. Lower temperature (0.3–0.5) helps.

Try it: Lattice Mini HF Space · Model weights · Read the from-scratch writeup