Perplexity has released Photon, an in-house retrieval and ranking engine written in Rust. It replaces an open-source engine Perplexity had forked for its AI-native search stack. Photon now handles retrieval and ranking for all production traffic. It also powers a new Fast Search mode in the Perplexity Search API. Perplexity reports single-call latency of 160 ms at p50 and 230 ms at p95.
Is it deployable? Yes, as a hosted API. Set search_type: "fast" on POST /search and pay $1 per 1,000 requests. Photon itself is not open source, so the engine cannot be self-hosted.
Why Perplexity Replaced its Old Engine
The old engine hit 3 limits as the index grew:
- Tail latency: Production p99 sat near 800 ms. The dataset exceeded RAM, so
mlockwas not an option. Cold reads triggered major page faults that stalled queries. - Merge spikes: During disk index fusion, p99 climbed to about 1.2 s for 10 to 15 minutes.
- Slow recovery: Deploying and syncing an extra cluster could take more than a week. Recovery also raised the share of partial responses.
Perplexity team concluded that building from scratch was simpler and cheaper than maintaining its fork.
How Photon Works
A load balancer routes each request to a Photon broker. The broker fans out to a shard group and watches for timeouts. Each shard runs retrieval, initial ranking, and second-stage ranking. The broker then merges candidates and fetches key document fields.
- Adaptive posting lists: Short lists sit inline within a single page. Longer lists split into blocks of fixed document ID ranges. Sparse blocks store sorted offset arrays and use galloping search. Dense blocks use bitmaps, so membership becomes a single bit lookup.
- Budgeted traversal: A WAND-like algorithm splits lists into driving lists and probe lists. Cheap presence checks bound each candidate’s maximum score first. Exact term frequencies are read only when a candidate can clear the threshold.
- Docblob records: Each document gets a compact record of frequencies, field masks, and positions. Terms use Elias-Fano encoding, so ranking decodes only the matched terms. Ranking a candidate needs just 1 lookup per document.
- Batched async reads: Record offsets are known upfront, so disk reads go out in batches through
io_uring. The cache checks the whole batch first. Readers take no locks, and eviction uses CLOCK instead of a shared LRU list. - Separate build and serve: Indexers build versioned shard indexes from YTsaurus tables on dedicated nodes. A controller rotates serving groups one at a time and warms caches with replayed search-log queries.
A full web index now builds in a single-digit number of hours.
Interactive Explainer: Inside Photon
‘;
resize();return;
}
var nb=Math.min(12,Math.max(3,Math.round(Math.log10(n)*2)));
var g=document.createElement(‘div’);g.className=”plist”;
for(var b=0;b
var el=document.createElement(‘div’);el.className=”blk “+(dense?’dense’:’sparse’);el.style.animationDelay=(b*45)+’ms’;
var h=”
Block “+(b+1)+’: ‘+(dense?’bitmap’:’offset array’)+’
‘;
if(dense){h+=’
‘;for(var k=0;k’;h+=’
‘;}
else{var c=Math.max(2,Math.round(bd*12)),o=0,arr=[];for(var k3=0;k3
‘;}
h+=’
density ‘+Math.round(bd*100)+’%
‘;
el.innerHTML=h;g.appendChild(el);
}
box.appendChild(g);resize();
}
$(‘px-len’).addEventListener(‘input’,drawPL);$(‘px-den’).addEventListener(‘input’,drawPL);drawPL();
/* PANE 2: bounded traversal */
var C=[{id:’D1′,ub:.72,ex:.61},{id:’D2′,ub:.55,ex:.40},{id:’D3′,ub:.90,ex:.82},{id:’D4′,ub:.38,ex:.30},{id:’D5′,ub:.70,ex:.66},{id:’D6′,ub:.58,ex:.52}];
var idx,kept,reads;
function build2(){
idx=0;kept=[];reads=0;var h=””;
C.forEach(function(c){h+=”;});
$(‘px-cands’).innerHTML=h;$(‘px-st2′).innerHTML=’Exact frequency reads: 0 of 6. Press the button to start.’;$(‘px-step2’).disabled=false;resize();
}
function thr(){if(kept.length=C.length)return;var c=C[idx];var row=$(‘px-c’+c.id);var tag=row.querySelector(‘.tag’);
row.querySelector(‘.ub’).style.width=(c.ub*100)+’%’;var t=thr();var msg;
if(c.ub’+c.id+’: upper bound ‘+c.ub.toFixed(2)+’ cannot beat threshold ‘+t.toFixed(2)+’. Skipped without reading probe lists or exact frequencies.’;}
else{reads++;row.querySelector(‘.ex’).style.width=(c.ex*100)+’%’;kept.push(c);kept.sort(function(a,b){return b.ex-a.ex;});var out=null;
if(kept.length>3){out=kept.pop();var orow=$(‘px-c’+out.id).querySelector(‘.tag’);orow.className=”tag drop”;orow.textContent=”evicted”;}
if(!out||out.id!==c.id){tag.className=”tag keep”;tag.textContent=”retained (“+c.ex.toFixed(2)+’)’;}
msg=’‘+c.id+’: bound ‘+c.ub.toFixed(2)+’ clears threshold ‘+t.toFixed(2)+’, so Photon reads exact frequencies: score ‘+c.ex.toFixed(2)+’.’+(out?’ ‘+out.id+’ is evicted from the bounded set.’:”);}
idx++;setThr();
msg+=’
Exact frequency reads: ‘+reads+’ of ‘+idx+’ visited. New threshold: ‘+thr().toFixed(2)+’.’;
if(idx>=C.length){msg+=’ Traversal done: ‘+(C.length-reads)+’ of 6 candidates were pruned cheaply.’;this.disabled=true;}
$(‘px-st2’).innerHTML=msg;resize();
});
$(‘px-reset2’).addEventListener(‘click’,build2);build2();
/* PANE 3: batched reads */
var batch=[];var U=26;
function newBatch(){batch=[];for(var i=0;ithread
Credit: Source link
