Hugging Face Releases Carbon-X, a 14B Open DNA Language Model that Brings DNA and Natural Language Together for the First Time

Hugging Face has released Carbon-X, an open DNA language model with 14.8B parameters. Carbon-X is a decoder-only model from Hugging Face Bio. It reads raw DNA in a chat and answers in English. It predicts whether mutations are benign or pathogenic, flags regulatory regions, classifies organisms and predicts next bases. The weights ship under Apache 2.0, so labs can keep sensitive sequences on their own GPUs. Hugging Face built it with Zhongguancun Academy and TIGEM / University of Naples Federico II.

TL;DR

Size: 14.8B parameters (dense, no MoE), 40,960-token context.
Runs on: 1 x 80 GB GPU in bfloat16 via vLLM or Transformers. Official GGUF: Q8_0 is 15.8 GB, Q4_K_M is 9.6 GB.
Performance: 66.7% average on 5 DNA benchmarks, best of every open model tested (self-reported).
Best: 74.9% on BioReason Coding, ahead of every model tested, open or closed.
Worst: 29.8% on GenomeQA splice-site questions.

What is Carbon-X and how does it work?

Carbon-X starts from Qwen3-14B. DNA and text share 1 token stream in 1 decoder. There is no separate DNA encoder, unlike earlier work such as BioReason.

The key structure piece is a hybrid tokenizer. It keeps Qwen3’s BPE vocabulary for text and adds 5,515 DNA tokens. These include 4,096 non-overlapping 6-mers, 1,364 tail tokens and 3 markers. One DNA token covers 6 base pairs. DNA runs of 6 or more A/C/G/T bases are tagged automatically.

The backbone is a standard Qwen3 stack. It has 40 layers, hidden size 5120 and GQA with 8 KV heads.

How was Carbon-X trained?

Training ran in four stages, per the model card:

Embedding warmup: about 10B tokens with the transformer frozen, so only the new DNA embeddings learn.
Mid-training: 302B tokens at 8,192 sequence length. The mix is 55% raw DNA, 20% English, 15% biology text and 10% instruction data.
SFT: 3 epochs over 20 dataset subsets, with responses checked by deterministic verifiers built with domain experts.
RL: asynchronous GRPO with verifiable rewards over 2 rounds, using TRL and vLLM.

As per the release post, learning rate was the hardest knob. High rates learned DNA but forgot English. A lower rate, a 14B model and a long run fixed the trade-off. RL applied directly to Qwen3-14B never beat the 50% chance baseline on FrontierVEP.

How does Carbon-X perform on DNA benchmarks?

All scores are self-reported by Hugging Face. Carbon-X ran with thinking disabled. Competitors ran at their best reasoning effort.

Benchmark	Carbon-X	GPT-6.1 Sol (Max)	Opus 5.5 (Max)	Kimi K3 (Low)
BioReason Coding	74.9	71.0	70.3	53.9
BioReason Non-SNV	85.1	91.7	94.8	65.3
GenomeQA	63.6	67.2	62.2	52.0
NextBase	39.4	32.9	1.1	30.6
FrontierVEP	70.4	74.9	76.2	50.1
Average	66.7	67.5	60.9	50.4

Carbon-X sits 27 points above Qwen3-14B and 0.8 points below GPT-6.1 Sol. It leads on coding-variant effects and next-base prediction. Frontier models still win on complex variants and single-base variants.

Refusals shape these numbers. On NextBase, Opus 5.5 falls from 17.7% at low effort to 1.1% at max effort as refusals rise. Gemini 4 Argon drops from 58% to 38% on FrontierVEP as effort increases.

Carbon-X also keeps general skills: 79.3% prompt-level strict accuracy on IFEval.

<div class="pnl on" data-i="0">
<p class="lead">Text and DNA share <b>1 token stream in 1 decoder</b>. Text keeps Qwen3 BPE. DNA becomes <b>6-mers</b>: 1 token per 6 bases.</p>
<div class="raw">Describe this sequence: <span class="dn">ACGTACGATTCATATAAA</span></div>
<div class="lbl">Hybrid tokenizer output</div>
<div class="row" id="cx-toks"></div>
<div class="btns"><button class="btn" id="cx-tok"> Tokenize</button><button class="btn gh" id="cx-rst">Reset</button></div>
<p class="note">5,515 DNA tokens added: 4,096 six-mers, 1,364 tail tokens and 3 markers (<dna>, </dna>, <oov>). Context: 40,960 tokens.</p>
</div>

<div class="pnl" data-i="1">
<p class="lead">Built from <b>Qwen3-14B</b> in 4 stages. Click a stage.</p>
<div class="flow" id="cx-flow">
<div class="st on" data-s="0"><div class="n">01</div><div class="t1">Warmup</div><div class="t2">~10B tokens</div></div>
<div class="st" data-s="1"><div class="n">02</div><div class="t1">Mid-training</div><div class="t2">302B tokens</div></div>
<div class="st" data-s="2"><div class="n">03</div><div class="t1">SFT</div><div class="t2">3 epochs</div></div>
<div class="st" data-s="3"><div class="n">04</div><div class="t1">RL</div><div class="t2">Async GRPO</div></div>
</div>
<div class="box det" id="cx-det"></div>
</div>

<div class="pnl" data-i="2">
<p class="lead">Carbon-X with thinking off vs rivals at their best reasoning effort.</p>
<div class="bsel" id="cx-bsel"></div>
<div id="cx-bars"></div>
<p class="note" id="cx-bnote"></p>
</div>

<div class="pnl" data-i="3">
<p class="lead">Priced at measured throughput on an <b>H200 at $5/hour</b>.</p>
<div class="grid2">
<div class="box"><div class="big" id="cx-c1">$0</div><div class="cap">Estimated cost to score 2.4M ClinVar variants of uncertain significance with Carbon-X.</div></div>
<div class="box"><div class="big" style="color:#a7adbb!important">$32K to $4M</div><div class="cap">Same job on the frontier models Hugging Face evaluated, by model and effort.</div></div>
<div class="box"><div class="big" id="cx-c2">0x</div><div class="cap">Lower inference cost per problem than GPT-6.1 Sol (model card: about 1/1,800).</div></div>
<div class="box"><div class="big" id="cx-c3">+0.0</div><div class="cap">Points gained by Kimi K3 when it consults Carbon-X as a sub-agent (49.4% to 60.5%).</div></div>
</div>
</div>

<div class="pnl" data-i="4">
<p class="lead">Not clinically validated. Tap a card for details.</p>
<div class="lim">
<div class="box"><div class="h">Splice sites <em>29.8%</em></div><div class="b">Weakest GenomeQA task. Histone marks are also low at 34.6%.</div></div>
<div class="box"><div class="h">Exact operations <em>!</em></div><div class="b">It can miscount bases or confuse coordinates in long sequences. Use code for counting and edits.</div></div>
<div class="box"><div class="h">Hallucination <em>!</em></div><div class="b">It may invent a sequence’s identity without context. Give gene, strand and annotations.</div></div>
<div class="box"><div class="h">Long generation <em>!</em></div><div class="b">Quality degrades on long DNA generation. The model card says it was not trained to design new sequences.</div></div>
</div>
</div>

<div class="ft"><span>Sources: HuggingFaceBio/Carbon-X model card and release blog</span><span class="mtp">© Marktechpost</span></div>
</div>
<script>
(function(){
var R=document.getElementById(‘mtp-cx’);
function post(){try{parent.postMessage({mtpcx:1,h:R.offsetHeight+40},’*’)}catch(e){}}
var tabs=R.querySelectorAll(‘.tab’),pn=R.querySelectorAll(‘.pnl’);
tabs.forEach(function(t){t.onclick=function(){tabs.forEach(function(x){x.classList.remove(‘on’)});pn.forEach(function(x){x.classList.remove(‘on’)});t.classList.add(‘on’);var p=R.querySelector(‘.pnl[data-i="’+t.dataset.p+’"]’);p.classList.add(‘on’);if(t.dataset.p===’2′)drawBars(cur);if(t.dataset.p===’3′)countUp();setTimeout(post,60)}});
// tokenizer
var TOK=[[‘Describe’,’t’],[‘this’,’t’],[‘sequence’,’t’],[‘:’,’t’],[‘<dna>’,’m’],[‘ACGTAC’,’d’],[‘GATTCA’,’d’],[‘TATAAA’,’d’],[‘</dna>’,’m’]];
var box=document.getElementById(‘cx-toks’),timer=[];
function build(){box.innerHTML=”;TOK.forEach(function(k){var s=document.createElement(‘span’);s.className=”tk “+k[1];s.textContent=k[0];box.appendChild(s)})}
build();
document.getElementById(‘cx-tok’).onclick=function(){timer.forEach(clearTimeout);build();var el=box.children;for(var i=0;i<el.length;i++)(function(i){timer.push(setTimeout(function(){el[i].classList.add(‘show’)},120*i+40))})(i)};
document.getElementById(‘cx-rst’).onclick=function(){timer.forEach(clearTimeout);build()};
// training
var ST=[
{h:’Embedding warmup’,p:’New DNA tokens get random embeddings. About 10B tokens train only those embeddings, with the transformer body frozen.’,mix:null},
{h:’Mid-training: 302B tokens’,p:’Sequence length 8,192, cosine LR 5e-5 to 5e-6. A lower LR, a 14B model and a long run kept English while learning DNA.’,mix:[[55,’#ffb84d’,’Raw DNA 55%’],[20,’#7cc4ff’,’English 20%’],[15,’#FF9D00′,’Biology text 15%’],[10,’#FFD21E’,’Instruction 10%’]]},
{h:’Supervised fine-tuning’,p:’3 epochs over 20 dataset subsets. Variant-effect reasoning is about 30% of the mix. Answers were checked by deterministic verifiers built with domain experts.’,mix:null},
{h:’Reinforcement learning’,p:’Asynchronous GRPO with code-checked rewards, 2 rounds, 16 samples per prompt. RL alone on Qwen3-14B never beat the 50% chance baseline on FrontierVEP.’,mix:null}];
var det=document.getElementById(‘cx-det’),sts=R.querySelectorAll(‘.st’);
function showSt(i){sts.forEach(function(s){s.classList.toggle(‘on’,s.dataset.s==i)});var s=ST[i],h=”<div style="font-weight:700;font-size:15px;color:#FFD21E">”+s.h+'</div><div class="cap" style="font-size:13px">’+s.p+'</div>’;
if(s.mix){h+='<div class="mix">’+s.mix.map(function(m){return ‘<i style="background:’+m[1]+’" data-w="’+m[0]+’"></i>’}).join(”)+'</div><div class="leg">’+s.mix.map(function(m){return ‘<span style="–c:’+m[1]+’">’+m[2]+'</span>’}).join(”)+'</div>’}
det.innerHTML=h;if(s.mix)setTimeout(function(){det.querySelectorAll(‘.mix i’).forEach(function(x){x.style.width=x.dataset.w+’%’})},40);setTimeout(post,60)}
sts.forEach(function(s){s.onclick=function(){showSt(+s.dataset.s)}});showSt(0);
// benchmarks
var M=[‘Carbon-X’,’GPT-6.1 Sol (Max)’,’Opus 5.5 (Max)’,’Gemini 4 Argon (Low)’,’Kimi K3 (Low)’,’DeepSeek V4.1 Flash (Low)’,’GLM-5.3 (Low)’];
var B={‘Average’:[66.7,67.5,60.9,49.8,50.4,47.1,39.8],’BioReason Coding’:[74.9,71.0,70.3,6.5,53.9,51.4,52.1],’BioReason Non-SNV’:[85.1,91.7,94.8,90.3,65.3,55.9,60.1],’GenomeQA’:[63.6,67.2,62.2,63.1,52.0,49.7,44.6],’NextBase’:[39.4,32.9,1.1,31.2,30.6,27.2,9.7],’FrontierVEP’:[70.4,74.9,76.2,58.0,50.1,51.3,32.7]};
var N={‘Average’:’Carbon-X trails GPT-6.1 Sol by 0.8 points and sits 27 points above Qwen3-14B.’,’BioReason Coding’:’Coding-variant effects, 1,018 variants. Carbon-X leads every model tested.’,’BioReason Non-SNV’:’Insertions, deletions and other non-SNV changes, 873 variants. Opus 5.5 leads.’,’GenomeQA’:’2,600 multiple-choice questions over 6 tasks. GPT-6.1 Sol leads.’,’NextBase’:’Next-base prediction, 600 sequences. Carbon-X leads; Opus 5.5 collapses as refusals rise.’,’FrontierVEP’:’Single-base variants with up to 8,192-base context, 1,000 variants. Opus 5.5 leads.’};
var cur=”Average”,bs=document.getElementById(‘cx-bsel’),bars=document.getElementById(‘cx-bars’);
Object.keys(B).forEach(function(k){var b=document.createElement(‘button’);b.textContent=k;if(k===cur)b.className=”on”;b.onclick=function(){bs.querySelectorAll(‘button’).forEach(function(x){x.className=””});b.className=”on”;cur=k;drawBars(k)};bs.appendChild(b)});
function drawBars(k){var v=B[k];bars.innerHTML=M.map(function(m,i){var me=i===0?’ me’:”;return ‘<div class="bar"><div class="nm’+me+’" title="’+m+’">’+m+'</div><div class="tr"><div class="fl’+me+’" data-w="’+v[i]+’"></div></div><div class="v">’+v[i].toFixed(1)+'</div></div>’}).join(”);
setTimeout(function(){bars.querySelectorAll(‘.fl’).forEach(function(f){f.style.width=f.dataset.w+’%’})},40);document.getElementById(‘cx-bnote’).textContent=N[k];setTimeout(post,60)}
drawBars(cur);
// cost
function anim(el,to,fmt){var t0=null;function f(ts){if(!t0)t0=ts;var p=Math.min(1,(ts-t0)/1000),e=1-Math.pow(1-p,3);el.textContent=fmt(to*e);if(p<1)requestAnimationFrame(f)}requestAnimationFrame(f)}
function countUp(){anim(document.getElementById(‘cx-c1’),300,function(x){return ‘$’+Math.round(x)});anim(document.getElementById(‘cx-c2’),1800,function(x){return ‘~’+Math.round(x).toLocaleString(‘en-US’)+’x’});anim(document.getElementById(‘cx-c3’),11.1,function(x){return ‘+’+x.toFixed(1)})}
// limits
R.querySelectorAll(‘.lim .box’).forEach(function(b){b.onclick=function(){b.classList.toggle(‘open’);setTimeout(post,380)}});
window.addEventListener(‘load’,post);window.addEventListener(‘resize’,post);setTimeout(post,300);
})();
</script>
</body></html>
” style=”width:100%;height:600px;border:0;display:block;background:transparent;” loading=”lazy”>

What does Carbon-X cost to run?

Hugging Face prices Carbon-X at measured throughput on an H200 at $5 per hour. On that basis, it reports about 1/1,800 of GPT-6.1 Sol’s inference cost per problem. Scoring the 2.4 million ClinVar variants of uncertain significance would cost about $300. The same job costs $32,000 to $4 million on frontier models.

Carbon-X also works as a sub-agent. Kimi K3’s DNA index rose from 49.4% to 60.5% when it consulted Carbon-X.

How do you deploy Carbon-X?

vLLM: vllm serve HuggingFaceBio/Carbon-X --trust-remote-code on 1 x 80 GB GPU.
Transformers: v5 or later, with trust_remote_code=True for the DNA tokenizer.
llama.cpp: the official Carbon-X-GGUF repo needs llama.cpp b11742 or newer. Older builds tokenize DNA wrongly.
Agents: the repo ships an OpenAI-style tool, an Inspect agent and a carbon-dna skill for coding agents.

Keep enable_thinking=False. The model was post-trained and evaluated that way.

How does Carbon-X compare with other DNA and general models?

Feature	Carbon-X	Qwen3-14B	GPT-6.1 Sol	Evo 2 40B
Developer	Hugging Face + partners	Alibaba Qwen	OpenAI	Arc Institute
Parameters	14.8B	14.8B	Not disclosed	40B
Context	40,960 tokens	32,768 native, 131,072 with YaRN	Not disclosed	Up to 1M sequence length
Input	DNA + English	Text	Text (general-purpose)	DNA only
Answers in English	Yes	Yes	Yes	No
License	Apache 2.0	Apache 2.0	Proprietary, API only	Apache 2.0
DNA average (HF eval)	66.7%	~40.0%	67.5% (Max)	Not evaluated
BioReason Coding	74.9%	Not disclosed	71.0%	Not evaluated
Price	Open weights	Open weights	$2 / $10 per 1M input / output tokens	Open weights

Key Takeaways

Carbon-X is a 14.8B open model that reads DNA and answers in English.
It averages 66.7% on 5 DNA benchmarks, 0.8 points behind GPT-6.1 Sol.
It leads every tested model on BioReason Coding (74.9%) and NextBase (39.4%).
It runs on 1 x 80 GB GPU, or a 15.8 GB Q8_0 GGUF.
Splice-site accuracy is 29.8%, and it is not clinically validated.

Check out the model card, release blog, GGUF weights and GitHub repo. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.

Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us

What Does The Bell Icon Mean On Your CarPlay Screen?

Tesla Unveils Flying Roadster and Glimpse Into Its Future

The post Hugging Face Releases Carbon-X, a 14B Open DNA Language Model that Brings DNA and Natural Language Together for the First Time appeared first on MarkTechPost.

Credit: Source link