A Working Nintendo in ~1GB of Qwen Weights

nesformer An NES CPU compiled into the weights of a stock Qwen3.5 model. The NES's 6502 CPU (the core of its 2A03 chip), its 2 KB of work RAM, and a cartridge's program ROM run inside a standard Qwen3_5ForCausalLM checkpoint. Nobody trained these weights; a compiler computed every one. When you play a game, the model predicts the CPU's execution log one token at a time. A small host program plays the rest of the console around it: it draws the picture, reads the controller, and fires the vblank interrupt. nesformer builds on Qwenputer by Divyansh Agrawal, which compiled a CHIP-8 and then a RISC-V CPU into Qwen3.5-4B. It reuses Qwenputer's compiler and its central trick: the model's Gated DeltaNet state is the machine's memory. Every CPU token in these runs came from the model. For the bundled test games, every frame also matches a conventional 6502 emulator's output exactly (see How it was checked). Quick start 1. Folder layout Unzip nesformer.zip and nesformer-weights.zip into the same folder. The weights go next to the nesformer/ package, not inside it: nesformer/ <- run commands from here ├── nesformer/ <- the Python package ├── chipformer/ <- Qwenputer's compiler ├── roms/ <- test ROMs (put your own here too) ├── examples/ <- sample GIFs and frame sheets ├── tests/ ├── LICENSE ├── LICENSE.chipformer ├── README.md ├── pyproject.toml └── weights/ <- from nesformer-weights.zip ├── config.json ├── generation_config.json ├── model.safetensors ├── nesformer.json └── SHA256SUMS The CLI looks for weights/ beside the nesformer/ package. To keep the weights elsewhere, pass --weights /path/to/weights to run, check or trace, or give the path to info. To confirm the weights unzipped correctly, run shasum -a 256 -c weights/SHA256SUMS from the top folder. 2. Install You need Python 3.11 or newer. python3 -m venv .venv source .venv/bin/activate # Windows: .venv\Scripts\activate pip install numpy scipy numba pillow 3. Play a game python -m nesformer run roms/thwaite.nes --frames 600 --show \ --buttons "start@90-95+200-205,a@230-233+250-253+270-273+290-293+310-313+330-333+350-353+370-373+390-393+410-413+430-433+450-453+470-473+490-493+510-513+530-533+550-553+570-573+590-593" --show draws each frame in your terminal as the model generates it. When the run ends, the CLI saves an animated GIF named after the ROM (here thwaite.gif) in the current folder. Expect it to be slow. The first run spends about 20 seconds compiling the engine with numba. After that, a frame takes 1–6 seconds, so 600 frames (10 seconds of NES time) take around 25 minutes. Playing your own ROMs Point run at any .nes file that uses the NROM board (iNES mapper 0), such as Super Mario Bros., Donkey Kong or Excitebike. You supply the ROM; nesformer only ships the three freely licensed test ROMs in roms/. python -m nesformer run roms/smb.nes --frames 600 --show \ --buttons "start@60-65,right@200-600,a@250-265+300-315+360-375+420-435+480-495+540-555" That script starts the game, then walks Mario right and jumps about once a second. The CLI loads the ROM into the model's state at startup, so the same weights run any NROM game without a rebuild. Games on other boards stop with a message: see Limitations. Controller scripts --buttons takes a comma-separated list of entries. Each entry names one or more buttons joined by +, then @, then one or more frame ranges joined by +: start@60-65 press Start during frames 60 to 65 right@200-600 hold Right from frame 200 to frame 600 a@250-265+300-315 press A twice right+b@200-600 hold Right and B together (run, in Super Mario Bros.) The buttons are a, b, select, start, up, down, left and right. Frames count from power-on at 60 per second, so frame 600 is 10 seconds in. Games ignore input on screens where you can't move, so holding a button a little early does no harm. In Super Mario Bros., a longer A press makes a higher jump. Commands Options for run: run, check and trace all accept --weights, --engine, --buttons and --no-skip. check --tokens N sets how far to compare (default 50,000). trace --tokens N sets how many tokens to print (default 40). build --out DIR writes the checkpoint somewhere other than weights/. How it works What's in the weights, and what isn't This split follows Qwenputer's CHIP-8 build, whose host injects timer ticks and draws the display from the trace. The host never sees the CPU's registers or RAM except through the trace. The memory trick works like this. Each DeltaNet head keeps a 128 × 128 state matrix that the delta rule updates at every token. With the gate held open and a one-hot key, an update overwrites exactly one row of that matrix, and a one-hot query reads it back. So every row is a memory cell, and the cartridge, RAM and registers all live in rows. The checkpoint weights/ is an ordinary Hugging Face folder. It loads with Qwen3_5ForCausalLM.from_pretrained and needs no custom code. nesformer.json records which layer and head holds each part of the machine. A run writes the cartridge, registers and RAM into those DeltaNet states, then decodes greedily. Layer plan The trace Each instruction becomes a record, and the record's last token commits it. As in Qwenputer's RISC-V core, the PC is implied rather than written: each final token carries its instruction's length, three DeltaNet counters tally the instructions of each length since the last jump, and the PC is the jump target plus that offset. Excerpts from Full Game: SEI P(000100 px0 L1) LDX #$40 X($40 c0 v0 px0 L2) STX $4017 IOW($4017=$40 L3) TXS S($FF L1) BIT $2002 IOR($2002) IN($00) P(000110 px0 L3) <- the host injects IN BPL $8017 J($8017) JSR $8279 PSH($FE) PSH($FD) J($8279) <- the return address is pushed in-circuit RTS PUL($FC) LO($7B) PUL($FD) J($827C) STA ($72),Y LO($08) HI($03) ST($00 a7=0 L2) <- pointer bytes latched, then the store STA $4014 IOW($4014=$02 L0) DM(0:$FF) … DM(255:$00) NT(L3) <- OAM DMA Most instructions take a single token; real game code averages 1.1–1.3 tokens per instruction. nesformer/spec.py documents every token family. How it was checked The reference CPU (nesformer/cpu.py) matches nestest's golden log for 3,348 instructions, and the py65 emulator on 5.7 million instructions of game code. The reference CPU (nesformer/cpu.py) matches nestest's golden log for 3,348 instructions, and the py65 emulator on 5.7 million instructions of game code. The exact circuit, run in Qwenputer's evaluator with its bf16 exactness checks, predicts every reference token: all of nestest plus 283,610 positions of two games' boot sequences. The exact circuit, run in Qwenputer's evaluator with its bf16 exactness checks, predicts every reference token: all of nestest plus 283,610 positions of two games' boot sequences. The checkpoint in stock transformers (bf16, CPU) predicts 2,651 positions of nestest and Full Game with no mismatches. The checkpoint in stock transformers (bf16, CPU) predicts 2,651 positions of nestest and Full Game with no mismatches. The weights, free-running, generate traces identical to the reference CPU's: Run Tokens Frames NES time Result nestest (official opcodes) 4,586 — — identical, ending in the expected HALT Thwaite, check 700,000 271 4.5 s identical trace and frames Thwaite, run 1,916,527 600 10.0 s all 300 saved frames match Full Game, run 2,645,885 420 7.0 s all 210 saved frames match The weights, free-running, generate traces identical to the reference CPU's: The idle-loop skip changes nothing visible: frames and I/O writes, cycle timestamps included, match a run without it. The idle-loop skip changes nothing visible: frames and I/O writes, cycle timestamps included, match a run without it. Speed The default engine computes Qwen3.5's forward pass from model.safetensors, compiled with numba and skipping the all-zero blocks. It ran at about 1,650 tokens/s on one CPU core of the machine it was built on. The plain numpy engine does about 210, and stock transformers about 10. A real NES would need about 700,000 tokens per second. Games spend most of each frame in a loop that waits for vblank. When the host sees a loop body repeat exactly, with no RAM or I/O writes, it advances the clock to just before the next PPU event, and the remaining iterations still run. That cuts the token count by 2.6× to 4.2×. Overall, nesformer runs 150–290× slower than the real console. Limitations NROM cartridges only (iNES mapper 0). The CPU can address only 32 KB of program ROM at a time. Bigger games carry a bank-switching chip (a mapper, such as MMC1 or MMC3) that swaps which part of the ROM the CPU sees. nesformer's ROM reads happen inside the model, from a fixed 32 KB table, so it can't follow a bank switch yet. Supporting a mapper means storing the whole ROM in the state, keeping the bank registers there too, and adding one stage that turns each CPU address into a ROM address. Official opcodes only. An unofficial opcode emits HALT. Code must run from the cartridge ROM. A jump into RAM emits HALT. No IRQs (APU frame IRQ, DMC). BRK and NMI work. Read-modify-write instructions on I/O registers don't produce an I/O write, and $4020–$7FFF reads as 0 (no PRG-RAM). The PPU and APU aren't in the weights. The PPU renders one scanline at a time, so mid-scanline effects aren't reproduced. Sprite-0 hit and sprite overflow are approximate, and there's no sound. Tests pip install pytest py65 python -m pytest -q tests The tests take a few minutes. They check the reference CPU against nestest's log, the circuit against the reference, the weights free-running against the reference, and the idle-loop skip. Running the checkpoint in stock transformers pip install "torch>=2.6" "transformers>=5.22" python -m nesformer run roms/thwaite.nes --frames 3 --engine hf nesformer/hfcheck.py shows the whole procedure: load the checkpoint, decode one padding token, write the machine's state into the cache's DeltaNet states, then feed tokens one at a time. Files Credits Qwenputer and its chipformer compiler: MIT, © 2026 Divyansh Agrawal (LICENSE.chipformer). That license also covers the code adapted from Qwenputer in nesformer/core.py and nesformer/weights.py. Its last paragraph refers to Qwenputer's CHIP-8 ROMs, which this repository does not include. Test ROMs: nestest by Kevin Horton; Full Game by Doug Fraker (MIT); Thwaite by Damian Yerrick (GPL-3.0, source at pinobatch/thwaite-nes). License nesformer is released under the MIT License (see LICENSE). The ROMs in roms/ keep their own licenses, listed in roms/SOURCES.md.