Osegi
AI·1 min read

TINY 2.1: a language model that runs on a 486 under DOS

TINY 2.1 is a small transformer written in Turbo C that runs under DOS on a Pentium or 486DX and fits on one floppy disk. It also talks through the PC speaker.

A beige 486-era PC running a neon-styled text interface next to a 3.5-inch floppy disk.
Image: Хабр

A chatbot that fits on a floppy disk and runs on a 486? TINY 2.1 does exactly that. It is an open source language model for DOS, and its developer, who goes by WollyMonjac, says it was built with help from the AI tools Fable 5.2 and Opus 5.5.

The program is written in Turbo C 2.0 and runs under DOS on an old Pentium or a 486DX. The whole thing, weights included, fits on a single floppy.

How big is TINY 2.1?

Small, even by the standards of small models. It has 367,000 parameters (the numbers a model learns during training), arranged in 4 layers with a dimension of 96. It reads and writes one character at a time rather than in word chunks.

To make it fit, the weights are stored as 8-bit values, with one scaling factor per row. The total comes to 380 KB. At runtime it needs about 540 KB of free conventional memory, which is the first 640 KB that DOS programs have always fought over.

What can it actually answer?

Not much, and that is by design. The model was trained on roughly 3,500 facts, each written in several different phrasings. Ask it something outside that set and it confidently makes up an answer, which is a polite way of saying it hallucinates.

The interface is a full-screen, neon-style display that shows the network working as it runs. It also speaks in a robotic voice through the PC speaker, the single-beeper audio hardware of old PCs.

TINY 2.1 is a curiosity rather than a useful assistant. But it shows how little hardware a working transformer, the architecture behind modern chatbots, can get by with when the task is narrow enough.

Apps worth a look

SUMO: the number puzzle

Tap the right tiles, add them up, hit the target. Easy to learn, hard to master. Free, for all ages.

Tags:tiny 2.1doslanguage modelretro computing486turbo c

More from this day

See all →

More in: AI

See all →