The Fly Language Model (FLM) is a public chatbot that couples the complete retained MaleCNS v1.0 fruit fly connectome to a frozen LiquidAI LFM2.5-1.2B-Instruct backbone. The developer who created the FLM calls it the world’s first Fly Language Model, built on an architecture called GPF (Generative Pre-trained Fly). It does not use the GPF label, explicitly disclaims being the first connectome language model, and reports that a parameter-matched control without the fly graph performs slightly better.
Deployable: Yes, locally. The nftechie/flm repo is MIT-licensed and runs on Python 3.12 (macOS or Linux, MPS, CUDA, or CPU) with no API key.
What was actually built
The system is a reservoir computer bolted onto a language model. All 166,700 retained nodes and 25,582,938 directed edges of the MaleCNS graph participate. The graph, the backbone, and the random input and output projections are all fixed. Only a 278,528-parameter readout is trained, which is about 0.0238% of the 1,170,340,608 backbone parameters.
At each token, a fixed Gaussian projection compresses the 2,048-dimensional token embedding to 128 channels. Each reservoir node receives one channel with a random sign. The whole graph then updates with x = tanh(W(0.6x + 0.4Bc)), where W holds incoming-normalized anatomical contact counts. States are pooled into 128 bins, passed through two trained bias-free matrices (U at 128 by 128, V at 2,048 by 128), and projected through the frozen vocabulary head as a bounded residual added to the backbone logits. The residual is capped at an RMS of 0.25 across vocabulary coordinates.
The results
On a freshly frozen set of 32 SmolTalk everyday-conversation dialogues (1,236 target tokens), three fit seeds gave:
| Condition | NLL (nats/token) |
|---|---|
| Frozen backbone | 1.381995 |
| Fly readout | 1.359816 ± 0.000110 |
| Direct-input readout | 1.359328 ± 0.000108 |
| Relabeled, no refit | 1.381265 ± 0.000802 |
| No edges | 1.381995 |
The fly readout improved on the backbone by 0.0222 nats per token (perplexity 3.98 to 3.90). But a direct-input control, which feeds the same 128-channel token projection straight into an identical readout with no graph, did better in all 3 seeds by 0.000488 nats per token. The paired bootstrap interval (+0.00000502 to +0.00104) does not support a fly-specific gain.
Two other controls matter. Setting W to zero removes the residual exactly, reproducing the backbone’s per-token losses, so the graph verifiably participates. Relabeling node identities without retraining returns NLL near baseline, which shows the readout depends on its learned interface alignment, not that fly topology beats random wiring.
The research report also proves the recurrence contracts initial-state differences by at most 0.6 per token. After 10 tokens that bound is 0.00605; after 20 it is 0.0000366. Piling in 166,700 cells does not buy long memory. Context still comes from the backbone.
Prior work and the ‘first’ claim
The research report cites ngxson/fly-hf, an earlier prototype that used a 49,393-cell central-brain subset of MaleCNS as a reservoir trained on TinyStories without a pretrained backbone, and states plainly that it makes no claim to be the first connectome-based language model. FLM’s distinction is scale (the full retained graph) and the frozen-backbone design that keeps the source of language competence identifiable.
Interactive explainer
Key Takeaways
- Full 166,700-node fly connectome drives a frozen LFM2.5-1.2B; only 278,528 parameters train.
- Fly readout cuts NLL by 0.0222 nats/token, but a no-graph control beats it in every seed.
- Disconnection zeroes the residual exactly; relabeling breaks it. The graph participates, it does not win.
- State forgets at 0.6 per token, so the connectome adds no long-range memory.
- MIT code runs locally on Python 3.12; study artifacts stay private, so results are not independently reproducible yet.
Check out the Paper, GitHub repo, and live demo. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.
Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us
Credit: Source link
