The LLM

Neuron LLM

A language model whose weights live in a Solana account and whose every token is computed by a Solana program (neuron_llm). The math is integer-only, so every validator that runs it gets the same answer, bit for bit.

On Solana devnet. The model is locked in a devnet account and the program has written a first story there, identical to the off-chain run (details). Mainnet is next.

Measured

From rollup/sequencer/tests/llm.rs, generating from an empty prompt with greedy decoding. In every run the on-chain tokens are identical to the off-chain integer reference, and a lockstep test compares the whole session account with the host after every layer.

BuildStoryCompute per tokenTransactions per token
Plain bytecode (runs on stock Solana)127 tokens2.09M CU1.91
sol_infer (Neuron rollup)255 tokens0.36M CU0.27

The build deployed on devnet can't read its remaining compute (that syscall is off on devnet and mainnet), so it budgets each step with a conservative estimate. Measured over a full 255-token story, it takes 2.85 transactions per token: the figure on the home page.

Each transaction uses up to 1.4M compute units, Solana's per-transaction limit. Uploading the 313,751-byte model takes 327 transactions, each under the 1,232-byte packet limit. The story it writes from nothing begins: "Once upon a time, there was a little girl named Lily. She loved to play outside in the park."

Speed

A Step transaction doesn't name a position: it advances the session from wherever it is. So clients can send many at once and the chain runs them back to back, several in one block. npm run llm:live does exactly that against a running neuron-node: it uploads the model, starts a story, sends a full block of Step transactions every block and prints tokens as they land.

RunStoryBlocksSpeed
Live node over RPC, 34 Step transactions per block, sampled251 tokens3~230 tokens/s
Live node over RPC, 34 per block, greedy (byte-identical to the host reference)251 tokens2~370 tokens/s
Sequencer harness, 8 per block (llm_many_steps_per_block)255 tokens930 to 31 tokens per block

A full 48M CU block holds about 125 tokens. The sequencer executes about 525 tokens per second of CPU time on an Apple M4 Pro, where the SVM runs interpreted (Agave's JIT is x86-only). On Solana mainnet, a session is a single writable account, and Agave caps those at 12M compute units per block: about 5 tokens per block (~12 tokens/s) for one story, if a leader packs it fully. Separate sessions run in parallel.

Where the compute goes, per token, from the profile build (llm_profile): early in a story the weight matmuls are about 75% of the bytecode build's cost; late in a story attention catches up, because it grows with every token. The 2026-11-21 pass rebuilt attention (transposed key cache, flash-style value sum in 64-bit, scales folded at write time) and cut a token at position 200 from 5.26M to 2.84M CU in bytecode. On the rollup, attention now runs natively through sol_infer_attn_i8, so a layer costs a flat ~66K CU at any position and a token about 355K.

On Solana devnet

The plain-bytecode build is deployed on devnet at BfH35VV5cSVsxqU16YJLJQSvSKQjygY5X12Hn8Me5yMt. The model is locked in CutfHeMx…Js7S (SHA-256 9a1efbc0…e439c), and session 4pRrT19F…SUK1 wrote 123 tokens, identical to the host's integer reference. That run was throttled for the public devnet RPC (about one request a second); with a private endpoint a story runs at the account limit, about 4 tokens per block.

On a real Solana validator

The plain-bytecode build deployed to solana-test-validator (stock Agave 3.0.13: no rollup, no custom syscalls) at BfH35VV5cSVsxqU16YJLJQSvSKQjygY5X12Hn8Me5yMt, the address it will have on devnet and mainnet. The model uploaded and locked in 329 transactions (6 s), and a greedy session wrote 123 tokens in 14 s, about 4 tokens per block, identical to the host's integer reference. Four tokens per block is the limit Agave sets for one writable account (12M compute units per block).

sh llm/deploy.sh devnet                                  # keys in ~/.config/neuron
node llm/run-chain.cjs --cluster devnet --greedy        # upload once, then a verified story

The agent terminal

Neuron Terminal runs a cast of characters on the model. Each character is its own session account. A turn restarts the speaker's session with the story so far and a cue (Lily said to Ben, "), then streams Step transactions until the model closes the quote; every third turn the narrator continues the story instead. Every six turns, the words each character uses most join their persona line, which the model reads on their next turn. npm run terminal runs it live on a local rollup node with a feed the page connects to; elsewhere the page replays a recorded live session.

The model

v0 is stories260K from Andrej Karpathy's llama2.c (MIT license), trained on the TinyStories dataset. It's a small model with the same architecture as Llama: RMSNorm, rotary position embeddings, grouped-query attention and a SwiGLU feed-forward.

Weights260K (dim 64, 5 layers, 8 heads sharing 4 key/value heads, feed-forward 172)
Vocabulary512 tokens, which keeps the classifier small
ContextUp to 512 tokens, set per session
On chain313,751 bytes: int8 weights plus scales, RoPE tables and the vocabulary (the feed-forward is padded from 172 to 176 with zero weights, which computes the same function and keeps every matmul in 8-column blocks)

llm/fetch.sh downloads the checkpoint and checks its SHA-256. The host tools reproduce llama2.c's reference output word for word in float, then convert it to the on-chain format.

Integer-only inference

Solana has no floating point, and two validators must never disagree. Every step uses whole numbers, in llm/core, a no_std crate compiled into both the program and the host tools:

The integer path writes the same text as the float model: identical for 71 of the first 255 tokens of the reference story, and for all 118 tokens of another prompt. After that, small rounding differences change a few words, and the story stays coherent.

Two builds

All three kernels (reference, packed, syscall) give bit-identical results; the tests check it.

Program

Accounts

Instructions

TagInstructionWhat it does
0InitModelClaims a fresh model account and records its authority.
1WriteModelWrites a chunk of the blob. Authority only, while loading.
2FinalizeModelValidates the blob, stores its SHA-256, and locks it for good.
3StartSessionStarts a generation from a prompt (BOS first), with greedy or seeded sampling. A restart needs the session's authority.
4StepRuns layers until the story ends, the unit limit is hit, or the compute left can't fit the costliest unit so far. Anyone can call it. Each generated token is written to the log.

A token is six units: five layers, then the classifier. They can span transactions, so a token too big for one transaction still completes.

Run it

npm run llm:fetch               # weights (sha256-checked)
npm run llm -- float            # float reference: llama2.c's output, word for word
npm run llm -- int --kernel packed --temp 0.8 --min-p 0.1 --seed 7
npm run llm:build-programs      # neuron_llm.so and neuron_llm_rollup.so
npm run llm:test                # host tests, then the model on the real SVM

Mainnet costs (estimate)

From the measured transaction counts and Solana's current rent and fee rules:

Model account (rent deposit)~2.19 SOL
Uploading the weights327 transactions, ~0.0016 SOL
A 256-token session (rent deposit, reusable)~1.0 SOL
Deploying the program (101 KB)~0.7 to 1.4 SOL, depending on room left for upgrades
Per token1.9 transactions: ~9,500 lamports in base fees, plus priority fees

The program has no close instruction yet, so model and session deposits stay locked until one is added.

Next