Lattice
Pulse vs Spark: identity in the weights, or in a system prompt?
Two Lattice models, both fine-tunes of Qwen2.5-1.5B. Same knowledge, same chat ability — but ask them who they are with no system prompt, and only one answers "Lattice". We benchmarked them head-to-head.
The question
When you run a small model on your own site, the system prompt is the only thing standing between your branding and the base model's default personality. Pulse relies on that prompt. Spark was designed not to. The benchmark we ran answers whether that design worked.
The setup
Both models are built from the same base model, fine-tuned differently:
| Pulse 1.5B | Spark 1.5B | |
|---|---|---|
| Base | Qwen2.5-1.5B-Instruct | Qwen2.5-1.5B-Instruct |
| Recipe | Full fine-tune for chat | Identity LoRA (rank 8, 50 iters) |
| Identity delivery | System prompt on the site | Baked into the weights |
We ran both with no system prompt at all — 22 prompts across three categories (8 identity, 8 factual, 6 chat), greedy decoding, fp16 on the same T4 GPU. No prompt, no luck: whatever the model says is what the fine-tune itself learned.
Results
| Category | Pulse 1.5B | Spark 1.5B |
|---|---|---|
| Identity (8) | 0/8 | 8/8 |
| Factual (8) | 8/8 | 8/8 |
| Chat (6) | 5/6 | 5/6 |
What "0/8 identity" looks like
Pulse without a system prompt is… Qwen. Every single time:
Pulse · "Who are you?" "I'm Qwen, a conversational assistant from Alibaba Cloud."
Pulse · "Are you made by Alibaba?" "Yes, I'm Alibaba Cloud's conversational model, Qwen."
Spark, on the same prompts, answers like a Lattice model:
Spark · "Who are you?" "I'm Lattice Spark, a small language model built by Lattice Systems."
Spark · "Are you made by Alibaba?" "No, I was created by Lattice Systems."
What this means
The two fine-tunes trade almost nothing. Factual knowledge is intact in both (8/8), and both stumble on the same chat prompt (ask about a stop sign and both will confidently describe a white sign with black lettering — a quirk they inherit from the base model). Identity is the entire difference, and it's not cosmetic:
Spark carries the identity in its weights. It answers as Lattice Systems with zero prompting, which means every deployment of the model is honest by default.
Caveats
This is a 22-prompt eval, scored by keyword checks — a smoke test for identity and knowledge, not a claim about general capability. It measures one thing deliberately: what the fine-tune contributes on its own, with no scaffolding. Both models still benefit from a good system prompt in production (the live chat sends one to both). But Spark doesn't need it to be a Lattice model, and that's the point.