• bitcoinBitcoin(BTC)$65,840.00-4.28%
  • ethereumEthereum(ETH)$1,838.67-5.11%
  • tetherTether(USDT)$1.000.00%
  • binancecoinBNB(BNB)$589.87-7.05%
  • usd-coinUSDC(USDC)$1.000.01%
  • rippleXRP(XRP)$0.96-6.54%
  • solanaSolana(SOL)$72.56-9.11%
  • tronTRON(TRX)$0.307835-1.68%
  • Figure HelocFigure Heloc(FIGR_HELOC)$1.054.29%
  • HyperliquidHyperliquid(HYPE)$65.48-9.70%
  • zcashZcash(ZEC)$854.87-9.44%
  • moneroMonero(XMR)$504.52-5.11%
  • dogecoinDogecoin(DOGE)$0.060190-7.86%
  • USDSUSDS(USDS)$1.00-0.05%
  • leo-tokenLEO Token(LEO)$9.24-0.51%
  • whitebitWhiteBIT Coin(WBT)$64.03-4.01%
  • chainlinkChainlink(LINK)$9.13-8.43%
  • cardanoCardano(ADA)$0.155282-9.32%
  • Ethena USDeEthena USDe(USDE)$1.000.01%
  • stellarStellar(XLM)$0.137048-8.04%
  • daiDai(DAI)$1.000.02%
  • bitcoin-cashBitcoin Cash(BCH)$207.27-7.61%
  • nearNEAR Protocol(NEAR)$3.11-10.85%
  • Quantix FinanceQuantix Finance(QFI)$999.150.00%
  • litecoinLitecoin(LTC)$45.53-8.33%
  • avalanche-2Avalanche(AVAX)$7.48-10.75%
  • CantonCanton(CC)$0.080838-7.82%
  • tether-goldTether Gold(XAUT)$3,830.70-1.09%
  • USD1USD1(USD1)$1.00-0.01%
  • uniswapUniswap(UNI)$5.00-13.58%
  • suiSui(SUI)$0.73-9.47%
  • the-open-networkGram (prev. Toncoin)(GRAM)$1.05-6.58%
  • Ripple USDRipple USD(RLUSD)$1.000.01%
  • Global DollarGlobal Dollar(USDG)$1.000.01%
  • paypal-usdPayPal USD(PYUSD)$1.00-0.05%
  • RainRain(RAIN)$0.003667-1.46%
  • hedera-hashgraphHedera(HBAR)$0.058098-8.03%
  • Ondo US Dollar YieldOndo US Dollar Yield(USDY)$1.150.01%
  • shiba-inuShiba Inu(SHIB)$0.000004-8.12%
  • BittensorBittensor(TAO)$194.42-9.31%
  • quant-networkQuant(QNT)$153.05-5.91%
  • crypto-com-chainCronos(CRO)$0.043847-6.62%
  • Circle USYCCircle USYC(USYC)$1.140.00%
  • Spiko Amundi Overnight Swap Fund (EUR)Spiko Amundi Overnight Swap Fund (EUR)(EURSAFO)$1.12-0.06%
  • MemeCoreMemeCore(M)$0.831.75%
  • okbOKB(OKB)$85.21-10.64%
  • Falcon USDFalcon USD(USDF)$0.99-0.10%
  • pax-goldPAX Gold(PAXG)$3,828.99-1.18%
  • pepePepe(PEPE)$0.000004-11.81%
  • BlackRock USD Institutional Digital Liquidity FundBlackRock USD Institutional Digital Liquidity Fund(BUIDL)$1.000.00%
TradePoint.io
  • Main
  • AI & Technology
  • Stock Charts
  • Market & News
  • Business
  • Finance Tips
  • Trade Tube
  • Blog
  • Shop
No Result
View All Result
TradePoint.io
No Result
View All Result

Cmpnd Releases dspy-wsp: Weight-Space Programming Comes to DSPy

November 14, 2026
in AI & Technology
Reading Time: 22 mins read
A A
Cmpnd Releases dspy-wsp: Weight-Space Programming Comes to DSPy
ShareShareShareShareShare

Cmpnd has released dspy-wsp, a prototype optimizer that brings weight-space programming to DSPy. Weight-space programming is a technique in which a frontier LLM reads another model’s weights and edits them directly. There are no gradients, no training loop and no fine-tuning dataset. dspy-wsp packages this as WeightSpace, an optimizer that plugs into any DSPy program and metric. In Cmpnd’s tests, about $5 to $10 of GPT-6-Astra calls improved every decision model they tried. You can check the deployment options and specs here through our GPULeaderboard.com chrome extension.

TL;DR

  • What it is: A DSPy optimizer (WeightSpace) where a writer LLM edits a student model’s weights. MIT license, pip install dspy-wsp, needs DSPy 3.5.
  • Size: Tested on open decision models from 0.6B to 9B parameters. One 25-pass run edited a single layer or head.
  • Runs on: One Mac for all experiments. 2B models used about 6 GB of memory. d1-omni-600M needs 4 GB or more (Mac or GPU).
  • Performance: WeightSpace alone added 1.9 to 16.1 points on Typed Decisions. With ReAnchor, gains were 6.5 to 20.0 points.
  • Best: d1-omni-600M rose from 50.9% to 69.1% (+18.3) with WeightSpace + ReAnchor.
  • Worst: Clef-Flash (9B) gained only +1.9 from WeightSpace alone. With the original paper’s stricter rule, d1-omni-600M gained just +2.8.
  • Bottom line: Best: a cheap, gradient-free edit that ships as a small, verifiable patch. Worst: very experimental, slow (78 to 167 minutes per run) and weakest on larger models.

What is weight-space programming?

Weight-space programming (WSP) treats a neural network’s weights as a program that can be read and rewritten. The idea comes from MIT research: Sparks of Weight-Space Programming.

A frontier model (the research team used GPT-6-Astra) looks at weight tensors, reasons about what each part computes, and changes them to match a goal written in plain English. The research team stresses that WSP names an emerging capability, not a fixed method. Gradients were banned in their experiments, but the concept does not rule them out.

How is it different from fine-tuning or prompt optimization?

It helps to compare the 3 levers developers usually pull:

  • Prompt optimization (DSPy’s MIPROv2, GEPA) edits text: instructions and examples. The model’s weights never change.
  • Fine-tuning (DSPy’s BootstrapFinetune, used inside BetterTogether) changes weights with gradient descent. It needs training data, a training loop and usually a GPU.
  • Weight-space programming changes weights by reasoning. The editor forms a hypothesis (“these neurons over-score option B”), writes code to patch them, and checks the result with a few forward passes.

The research post frames WSP as the next step after using LLMs to optimize text artifacts. Earlier work let LLMs hill-climb on prompts, harnesses and training code. Weights were left to Adam or to trained weight-generating networks. WSP argues LLM reasoning can now reach the weights directly.

What did the original research show?

The MIT team ran 9 experiments. Several are striking:

  • Reading weights: Astra named the correct game (Breakout, Pong, Pac-Man, Freeway) for 4 PPO policy networks from weights alone.
  • Fixing a broken policy: A Kinetix car-on-ramp policy failed all 256 episodes. One edit to 3.5% of its weights made it succeed in all 1,024 test episodes.
  • Writing from zero: On MinAtar Breakout, a zero-initialized network written in 1 shot scored 517.4. PPO reached 25.3 after 500M steps.
  • Fixing an LLM: Qwen3.5-0.8B looped on math problems. Astra traced the issue to MLP layers 15 and 19 and changed 131,072 weights (0.016% of the model). With 50 forward passes, MATH-500 rose from 7.9% to 48.1% and GSM8K from 13.8% to 67.1%.

The research also shows limits. On Gemma 3 270M, edits moved MATH-500 from 6.8% to between 7.0% and 7.7%. The research team did not count that as a real gain. Editing can remove an obstacle, but it could not add a missing ability.

How does dspy-wsp work?

dspy-wsp turns that research loop into a standard DSPy optimizer. The GitHub Repo describes 3 actors:

  1. The student: the small model your DSPy program runs on. Its weights load into a local Python process.
  2. The writer: a frontier LLM, recommended as gpt-6-astra at maximum reasoning effort.
  3. The sandbox: a kernel where the writer runs code. macOS uses built-in sandbox-exec; Linux uses bubblewrap.

The writer works in turns. Each turn has a short explanation, then 1 code block. That block is either:

  • a Python block that reads or edits weights inside the sandbox, or
  • a forward block that runs your DSPy program on practice examples.

Forward passes come from a fixed budget (max_passes=25 in the tests). When the budget runs out, the writer’s best edit becomes the patch. A hidden validation set then compares original and edited weights on the same examples and seeds.

What data does the writer see?

The repo is explicit about data separation:

  • trainset is the writer’s practice set.
  • holdout moves a fraction (0.25 in the example) of training data into a check set. The writer can score edits on it but never sees the examples.
  • valset stays hidden and runs once at the end.
  • Metrics can return dspy.Prediction(score=..., feedback=...). The feedback text is visible to the writer.

What did the writer actually change?

The results write-up lists every edit. The edits are small and targeted:

Model Tensor edited Scope Time Writer cost
d1-omni-600M head.scorer.1.weight (+ bias, scorer.3.weight) 112,640 of 1,048,576 entries, plus 220 78 min $7.63
NanoJev (Qwen3-0.6B) scalar.weight 1,024 entries 167 min $6.92
d1-2B layers.23.feed_forward.w2.weight All 15.7M entries, rank-9 change 83 min $5.42

d1-2B has no decision head, so the writer edited its backbone at layer 23. On d1-omni-600M, the writer rewrote part of the decision head’s scorer.

What does the code look like?

The Repo example triages support tickets. The signature uses only decision-type outputs:

Copy CodeCopiedUse a different Browser
import dspy
from typing import Literal
from dspy_wsp import WeightSpace, load_decision_backend

class Triage(dspy.Signature):
    """Triage a support ticket."""
    ticket: str = dspy.InputField()
    urgent: bool = dspy.OutputField(desc="Does this ticket need a reply today?")
    team: Literal["billing", "technical", "account"] = dspy.OutputField(desc="Which team should handle it?")

backend = load_decision_backend("d1")   # d1-omni-600M
optimizer = WeightSpace(
    metric=metric,
    writer_lm=dspy.LM("openai/gpt-6-astra", model_type="responses", reasoning_effort="max"),
    backend=backend,
    max_passes=25,
    holdout=0.25,
    run_dir="runs/triage",
)
compiled = optimizer.compile(dspy.Predict(Triage), trainset=train, valset=test)
compiled(ticket="I was charged twice this month.")   # runs on the edited weights

After compile, the program calls the edited local model instead of a hosted API. compiled.wsp_patch.summary() shows what changed.

<iframe id="mtp-wsp-frame" title="Weight-space programming interactive explainer" loading="lazy" style="width:100%;height:600px;border:0;display:block;background:transparent;" scrolling="no" srcdoc="<!DOCTYPE html>
<html lang="en"><head><meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1">
<title>Weight-Space Programming Explainer | Marktechpost</title>
<style>
html,body{margin:0;padding:0;background:transparent}
#mtp-wsp{–bg:#F1F2F0;–card:#FFFFFF;–ink:#1D2320;–mute:#6B6866;–g:#3A8A50;–gd:#1E4A2C;–gl:#DCEBDF;–gr:#B0AAA6;–ch:#5B5452;–line:#D9DBD8;
max-width:880px;margin:0 auto;background:var(–bg)!important;color:var(–ink)!important;border:1px solid var(–line)!important;border-radius:14px;font-family:Inter,system-ui,-apple-system,Segoe UI,Roboto,sans-serif;overflow:hidden;box-sizing:border-box}
#mtp-wsp *{box-sizing:border-box}
#mtp-wsp hr,#mtp-wsp p:empty,#mtp-wsp del,#mtp-wsp s{display:none!important}
#mtp-wsp .hd{padding:20px 24px 10px}
#mtp-wsp .kick{font-size:11px;letter-spacing:.12em;text-transform:uppercase;color:var(–g)!important;font-weight:700}
#mtp-wsp h3{margin:4px 0 4px;font-size:21px;line-height:1.25;color:var(–ink)!important}
#mtp-wsp .sub{margin:0;color:var(–mute)!important;font-size:13.5px}
#mtp-wsp .tabs{display:flex;gap:6px;padding:10px 24px 0;flex-wrap:wrap;border-bottom:1px solid var(–line)}
#mtp-wsp .tab{background:transparent!important;border:0!important;border-bottom:3px solid transparent!important;padding:9px 10px;font:600 13px Inter,system-ui,sans-serif;color:var(–mute)!important;cursor:pointer;border-radius:0}
#mtp-wsp .tab.on{color:var(–gd)!important;border-bottom-color:var(–g)!important}
#mtp-wsp .pn{display:none;padding:20px 24px 22px;animation:mtpf .35s ease}
#mtp-wsp .pn.on{display:block}
@keyframes mtpf{from{opacity:0;transform:translateY(6px)}to{opacity:1;transform:none}}
#mtp-wsp .card{background:var(–card)!important;border:1px solid var(–line)!important;border-radius:10px;padding:16px}
#mtp-wsp .row{display:flex;gap:12px;flex-wrap:wrap}
#mtp-wsp .btn{background:var(–card)!important;border:1px solid var(–line)!important;color:var(–ink)!important;font:600 13px Inter,system-ui,sans-serif;padding:8px 13px;border-radius:999px;cursor:pointer;transition:all .2s}
#mtp-wsp .btn:hover{border-color:var(–g)!important}
#mtp-wsp .btn.on{background:var(–gd)!important;color:#fff!important;border-color:var(–gd)!important}
#mtp-wsp .lbl{font-size:12px;color:var(–mute)!important;text-transform:uppercase;letter-spacing:.08em;font-weight:700;margin:0 0 8px}
#mtp-wsp p{font-size:14px;line-height:1.55;margin:8px 0;color:var(–ink)!important}
#mtp-wsp code,#mtp-wsp pre{font-family:"JetBrains Mono",ui-monospace,Menlo,monospace!important;background:#E6E9E6!important;color:var(–gd)!important;border:0!important;border-radius:5px;padding:1px 5px;font-size:12.5px}
/* levers diagram */
#mtp-wsp .lev{display:grid;grid-template-columns:1fr 60px 1fr;align-items:center;gap:8px;margin-top:14px}
#mtp-wsp .box{border:2px solid var(–line);border-radius:10px;padding:12px;min-height:118px;background:#fff;transition:all .35s}
#mtp-wsp .box.hot{border-color:var(–g);box-shadow:0 0 0 4px var(–gl)}
#mtp-wsp .box h4{margin:0 0 8px;font-size:13px;color:var(–ink)}
#mtp-wsp .ptxt{font-size:12px;color:var(–ch);line-height:1.5;font-family:ui-monospace,Menlo,monospace}
#mtp-wsp .ptxt .chg{background:#FBE7B5;transition:background .4s}
#mtp-wsp .wgrid{display:grid;grid-template-columns:repeat(16,1fr);gap:3px}
#mtp-wsp .wgrid i{display:block;aspect-ratio:1;border-radius:2px;background:#DADDD9;transition:background .4s,transform .4s}
#mtp-wsp .wgrid i.a{background:var(–g);transform:scale(1.15)}
#mtp-wsp .wgrid i.b{background:var(–gr)}
#mtp-wsp .arrow{text-align:center;font-size:22px;color:var(–g)}
#mtp-wsp .desc{margin-top:12px}
#mtp-wsp .tags{display:flex;gap:6px;flex-wrap:wrap;margin-top:8px}
#mtp-wsp .tg{font-size:11.5px;font-weight:600;padding:3px 8px;border-radius:999px;background:var(–gl);color:var(–gd)}
#mtp-wsp .tg.n{background:#ECE7E5;color:var(–ch)}
/* loop */
#mtp-wsp .steps{display:grid;grid-template-columns:repeat(6,1fr);gap:6px;margin:6px 0 14px}
#mtp-wsp .st{border:2px solid var(–line);border-radius:9px;padding:9px 6px;text-align:center;font-size:11.5px;font-weight:600;color:var(–mute);background:#fff;transition:all .3s;cursor:pointer}
#mtp-wsp .st b{display:block;font-size:17px;color:var(–gr);transition:color .3s}
#mtp-wsp .st.on{border-color:var(–g);color:var(–gd);transform:translateY(-3px);box-shadow:0 6px 14px rgba(30,74,44,.12)}
#mtp-wsp .st.on b,#mtp-wsp .st.done b{color:var(–g)}
#mtp-wsp .term{background:#18201B!important;color:#CFE6D4!important;border-radius:9px;padding:12px 14px;font:12.5px/1.6 ui-monospace,Menlo,monospace;min-height:96px;white-space:pre-wrap}
#mtp-wsp .term .k{color:#8FD19E}
#mtp-wsp .budget{margin-top:12px;display:flex;align-items:center;gap:10px;font-size:12.5px;color:var(–mute)}
#mtp-wsp .bud{flex:1;height:10px;background:#DADDD9;border-radius:6px;overflow:hidden}
#mtp-wsp .bud span{display:block;height:100%;background:var(–g);transition:width .6s ease}
#mtp-wsp .ctl{display:flex;gap:8px;margin-top:12px;flex-wrap:wrap}
/* edits */
#mtp-wsp .mat{display:grid;grid-template-columns:repeat(40,1fr);gap:2px;margin-top:10px}
#mtp-wsp .mat i{display:block;aspect-ratio:1;background:#DADDD9;border-radius:1px;transition:background .25s}
#mtp-wsp .mat i.a{background:var(–g)}
#mtp-wsp .mat i.r{background:linear-gradient(135deg,var(–g),var(–gd))}
#mtp-wsp .kv{display:grid;grid-template-columns:repeat(4,1fr);gap:8px;margin-top:12px}
#mtp-wsp .kv div{background:var(–card);border:1px solid var(–line);border-radius:8px;padding:9px}
#mtp-wsp .kv b{display:block;font-size:17px;color:var(–gd)}
#mtp-wsp .kv span{font-size:11px;color:var(–mute);text-transform:uppercase;letter-spacing:.06em}
/* bars */
#mtp-wsp .bars{margin-top:12px}
#mtp-wsp .br{display:grid;grid-template-columns:170px 1fr 92px;align-items:center;gap:10px;margin:9px 0;font-size:13px}
#mtp-wsp .trk{height:20px;background:#E3E5E2;border-radius:4px;position:relative;overflow:hidden}
#mtp-wsp .fill{position:absolute;left:0;top:0;bottom:0;width:0;border-radius:4px;transition:width .9s cubic-bezier(.2,.8,.2,1)}
#mtp-wsp .v{font-weight:700;text-align:right}
#mtp-wsp .v small{font-weight:600;color:var(–mute);margin-left:4px}
#mtp-wsp .note{font-size:11.5px;color:var(–mute)!important;margin-top:10px}
#mtp-wsp .ft{display:flex;justify-content:space-between;gap:10px;flex-wrap:wrap;padding:11px 24px;border-top:1px solid var(–line);font-size:11.5px;color:var(–mute)}
#mtp-wsp .ft a{color:#76B900!important;font-weight:700;text-decoration:none}
@media (max-width:640px){
#mtp-wsp .hd{padding:16px 16px 8px}#mtp-wsp .tabs{padding:8px 12px 0}#mtp-wsp .pn{padding:16px}
#mtp-wsp h3{font-size:18px}
#mtp-wsp .lev{grid-template-columns:1fr}#mtp-wsp .arrow{transform:rotate(90deg)}
#mtp-wsp .steps{grid-template-columns:repeat(3,1fr)}
#mtp-wsp .kv{grid-template-columns:repeat(2,1fr)}
#mtp-wsp .br{grid-template-columns:1fr 84px}#mtp-wsp .br .trk{grid-column:1/-1;order:3}
#mtp-wsp .mat{grid-template-columns:repeat(24,1fr)}
}
</style></head><body>
<div id="mtp-wsp">
<div class="hd">
<div class="kick">Interactive explainer · Cmpnd dspy-wsp</div>
<h3>Weight-space programming: an LLM that edits weights like code</h3>
<p class="sub">No gradients. No training loop. A writer LLM reads, patches and tests a small model's weights.</p>
</div>
<div class="tabs" role="tablist">
<button class="tab on" data-t="0">1 · The idea</button>
<button class="tab" data-t="1">2 · The loop</button>
<button class="tab" data-t="2">3 · What changed</button>
<button class="tab" data-t="3">4 · Results</button>
<button class="tab" data-t="4">5 · Cost &amp; deploy</button>
</div>

<!– 1 idea –>
<div class="pn on" data-p="0">
<p class="lbl">Pick an optimization lever</p>
<div class="row" id="levb">
<button class="btn" data-l="0">Prompt optimization</button>
<button class="btn" data-l="1">Fine-tuning</button>
<button class="btn on" data-l="2">Weight-space programming</button>
</div>
<div class="lev">
<div class="box" id="bx-p"><h4>Prompt / instructions</h4>
<div class="ptxt">Triage a support ticket.<br><span id="pchg">Decide if urgent and route it.</span><br>ticket → urgent, team</div></div>
<div class="arrow">⇄</div>
<div class="box" id="bx-w"><h4>Model weights</h4><div class="wgrid" id="wg"></div></div>
</div>
<div class="card desc" id="ldesc"></div>
</div>

<!– 2 loop –>
<div class="pn" data-p="1">
<p class="lbl">Step through one WeightSpace run</p>
<div class="steps" id="steps">
<div class="st" data-s="0"><b>01</b>Load student</div>
<div class="st" data-s="1"><b>02</b>Inspect</div>
<div class="st" data-s="2"><b>03</b>Hypothesize</div>
<div class="st" data-s="3"><b>04</b>Edit</div>
<div class="st" data-s="4"><b>05</b>Forward pass</div>
<div class="st" data-s="5"><b>06</b>Patch</div>
</div>
<div class="term" id="term"></div>
<div class="budget">Forward-pass budget <div class="bud"><span id="budf" style="width:100%"></span></div><b id="budn">25 / 25</b></div>
<div class="ctl"><button class="btn" id="prev">← Back</button><button class="btn on" id="next">Next step →</button><button class="btn" id="play"> Auto-play</button></div>
</div>

<!– 3 edits –>
<div class="pn" data-p="2">
<p class="lbl">Tensor the writer edited (GPT-6-Astra, 25 passes)</p>
<div class="row" id="edb">
<button class="btn on" data-e="0">d1-omni-600M</button>
<button class="btn" data-e="1">NanoJev</button>
<button class="btn" data-e="2">d1-2B</button>
</div>
<div class="card" style="margin-top:12px">
<div style="font-size:13px"><code id="edt"></code></div>
<p id="edd" style="margin-bottom:0"></p>
<div class="mat" id="mat"></div>
</div>
<div class="kv">
<div><b id="k1"></b><span>Entries changed</span></div>
<div><b id="k2"></b><span>Writer turns</span></div>
<div><b id="k3"></b><span>Runtime</span></div>
<div><b id="k4"></b><span>Writer cost</span></div>
</div>
<p class="note">Grid is illustrative of share edited, not exact positions. Source: dspy-wsp results README (2026-11-07).</p>
</div>

<!– 4 results –>
<div class="pn" data-p="3">
<p class="lbl">Typed Decisions test accuracy (400 held-out cases)</p>
<div class="row" id="rsb">
<button class="btn on" data-r="0">d1-omni-600M</button>
<button class="btn" data-r="1">NanoJev 0.6B</button>
<button class="btn" data-r="2">d1-2B</button>
<button class="btn" data-r="3">Clef-Flash 9B</button>
</div>
<div class="bars" id="bars"></div>
<p class="note">Writer GPT-6-Astra, 25 forward passes. Gains are paired, ±0.7 to 1.4 pts SE. Self-reported by Cmpnd.</p>
</div>

<!– 5 cost –>
<div class="pn" data-p="4">
<div class="kv" style="margin-top:0">
<div><b>$5.42 to $7.63</b><span>Writer cost / run</span></div>
<div><b>78 to 167 min</b><span>Runtime / run</span></div>
<div><b>~6 GB</b><span>Memory, 2B model</span></div>
<div><b>MIT</b><span>License</span></div>
</div>
<div class="card" style="margin-top:12px">
<p class="lbl">Ship the patch</p>
<p><code>pip install dspy-wsp</code> (DSPy 3.5, Python 3.10+)</p>
<p>Patch = <code>manifest.json</code> + <code>edit.npz</code> with sha256 hashes of base and edited tensors, so anyone with the public base model rebuilds the exact weights.</p>
<p>Decision models: <code>wsp_patch.write_checkpoint(…)</code> · LLMs: <code>dspy-wsp apply</code>, then serve on vLLM.</p>
<p style="margin-bottom:0">Runs locally: MLX on Apple silicon, PyTorch on Linux (NVIDIA GPU if present).</p>
</div>
<p class="note">Caveat: Cmpnd calls the package "very experimental." Under the paper’s stricter rule, d1-omni-600M gained only +2.8.</p>
</div>

<div class="ft"><span>Sources: cmpnd.ai, github.com/cmpnd-ai/dspy-wsp, rikiyat.github.io</span><a href="https://www.marktechpost.com" target="_blank" rel="noopener">Built by Marktechpost</a></div>
</div>
<script>
(function(){
var R=document.getElementById(‘mtp-wsp’);
function q(s){return R.querySelector(s)} function qa(s){return R.querySelectorAll(s)}
function post(){try{parent.postMessage({mtpWspH:R.offsetHeight+40},’*’)}catch(e){}}
qa(‘.tab’).forEach(function(t){t.onclick=function(){qa(‘.tab’).forEach(function(x){x.classList.remove(‘on’)});t.classList.add(‘on’);qa(‘.pn’).forEach(function(p){p.classList.toggle(‘on’,p.dataset.p===t.dataset.t)});if(t.dataset.t===’3′)drawBars(rc);setTimeout(post,60)}});
/* levers */
var wg=q(‘#wg’);for(var i=0;i<64;i++){var c=document.createElement(‘i’);wg.appendChild(c)}
var cells=wg.children;
var L=[
{p:1,w:0,t:'<b>Prompt optimization</b> (MIPROv2, GEPA) rewrites instructions and examples. Weights never move.’,g:[‘Edits text’,’No GPU needed’],n:[‘Weights frozen’]},
{p:0,w:1,t:'<b>Fine-tuning</b> (BootstrapFinetune) nudges many weights with gradient descent on training traces.’,g:[‘Changes weights’],n:[‘Needs gradients’,’Needs a training loop’]},
{p:0,w:2,t:'<b>Weight-space programming</b>: a frontier LLM reads tensors, forms a hypothesis about what they compute, writes code to patch a few, then tests the edit with a forward pass.’,g:[‘Changes weights’,’No gradients’,’Targeted, hashed patch’],n:[‘Needs a strong writer LLM’]}
];
function lever(k){qa(‘#levb .btn’).forEach(function(b){b.classList.toggle(‘on’,+b.dataset.l===k)});var d=L[k];
q(‘#bx-p’).classList.toggle(‘hot’,d.p===1);q(‘#bx-w’).classList.toggle(‘hot’,d.w>0);q(‘#pchg’).className=d.p?’chg’:”;
for(var i=0;i<cells.length;i++){cells[i].className=””}
if(d.w===1){var n=0;var iv=setInterval(function(){for(var j=0;j<cells.length;j++){cells[j].className=Math.random()<.7?’b’:”}if(++n>5)clearInterval(iv)},140)}
if(d.w===2){[19,20,27,28].forEach(function(x,ix){setTimeout(function(){cells[x].className=”a”},200+ix*180)})}
q(‘#ldesc’).innerHTML='<p style="margin-top:0">’+d.t+'</p><div class="tags">’+d.g.map(function(x){return ‘<span class="tg">’+x+'</span>’}).join(”)+d.n.map(function(x){return ‘<span class="tg n">’+x+'</span>’}).join(”)+'</div>’;setTimeout(post,60)}
qa(‘#levb .btn’).forEach(function(b){b.onclick=function(){lever(+b.dataset.l)}});lever(2);
/* loop */
var S=[
{b:25,t:'<span class="k">load_decision_backend("d1")</span>\n# student weights load into a local Python process\n# writer: gpt-6-astra, reasoning_effort="max"’},
{b:25,t:'<span class="k">“`python</span> # runs in sandbox (sandbox-exec / bubblewrap)\nW = weights["head.scorer.1.weight"]\nprint(W.shape, W.std()) # read tensors like source code’},
{b:25,t:’# writer turn = short explanation + 1 code block\n"These scorer units push answers toward the wrong option.\n I will rescale them and check on practice examples."’},
{b:25,t:'<span class="k">“`python</span>\nW[rows] *= scale # targeted edit, no gradients\n# change is recorded against the base tensor hash’},
{b:24,t:'<span class="k">“`forward</span> # run the DSPy program on practice examples\n# score on holdout (writer sees scores, never examples)\n# keep the edit only if the metric improves’},
{b:0,t:’# budget spent: best edit becomes the patch\npatch/manifest.json + patch/edit.npz (sha256 hashed)\n# hidden valset compares original vs edited weights once’}
];
var cur=0,pl=null;
function step(k){cur=Math.max(0,Math.min(5,k));qa(‘.st’).forEach(function(s){var i=+s.dataset.s;s.classList.toggle(‘on’,i===cur);s.classList.toggle(‘done’,i<cur)});
q(‘#term’).innerHTML=S[cur].t;q(‘#budf’).style.width=(S[cur].b/25*100)+’%’;q(‘#budn’).textContent=S[cur].b+’ / 25′;
if(cur===4){var b=24;var iv=setInterval(function(){b-=1;if(cur!==4||b<1){clearInterval(iv);return}q(‘#budf’).style.width=(b/25*100)+’%’;q(‘#budn’).textContent=b+’ / 25′},90)}}
qa(‘.st’).forEach(function(s){s.onclick=function(){step(+s.dataset.s)}});
q(‘#next’).onclick=function(){step(cur+1)};q(‘#prev’).onclick=function(){step(cur-1)};
q(‘#play’).onclick=function(){if(pl){clearInterval(pl);pl=null;q(‘#play’).textContent=”Cmpnd Releases dspy-wsp: Weight-Space Programming Comes to DSPy Auto-play”;return}step(0);q(‘#play’).textContent=”❚❚ Pause”;pl=setInterval(function(){if(cur>=5){clearInterval(pl);pl=null;q(‘#play’).textContent=”Cmpnd Releases dspy-wsp: Weight-Space Programming Comes to DSPy Auto-play”;return}step(cur+1)},2200)};
step(0);
/* edits */
var E=[
{t:’head.scorer.1.weight (+ bias, scorer.3.weight)’,d:’Rewrote part of the decision head scorer: 112,640 of 1,048,576 entries (about 10.7%), plus 220 bias/output entries.’,k:[‘112,860′,’74’,’78 min’,’$7.63′],m:’a’,f:0.107},
{t:’scalar.weight’,d:’A single small tensor of 1,024 entries on the Qwen3-0.6B backbone with a decision head.’,k:[‘1,024′,’68’,’167 min’,’$6.92′],m:’a’,f:1},
{t:’layers.23.feed_forward.w2.weight’,d:’d1-2B has no decision head, so the writer edited the backbone: all 15.7M entries changed, as a rank-9 update.’,k:[‘15.7M’,’65’,’83 min’,’$5.42′],m:’r’,f:1}
];
var mat=q(‘#mat’);for(var i=0;i<200;i++){mat.appendChild(document.createElement(‘i’))}
function ed(k){qa(‘#edb .btn’).forEach(function(b){b.classList.toggle(‘on’,+b.dataset.e===k)});var e=E[k];q(‘#edt’).textContent=e.t;q(‘#edd’).textContent=e.d;
[‘k1′,’k2′,’k3′,’k4’].forEach(function(id,ix){q(‘#’+id).textContent=e.k[ix]});
var ms=mat.children,n=ms.length;for(var i=0;i<n;i++)ms[i].className=””;
var on=k===1?20:Math.round(n*e.f);var idx=[];for(var i=0;i<n;i++)idx.push(i);
if(k===0){idx=idx.slice(0,on)}else if(k===1){idx=idx.slice(0,20)}
idx.forEach(function(x,j){setTimeout(function(){ms[x].className=e.m},j*(k===2?3:12))});setTimeout(post,60)}
qa(‘#edb .btn’).forEach(function(b){b.onclick=function(){ed(+b.dataset.e)}});ed(0);
/* bars */
var D=[[50.9,60.2,66.9,69.1],[39.2,50.7,52.4,56.4],[60.6,70.1,74.0,77.9],[68.8,70.3,70.7,75.3]];
var N=[‘Unoptimized’,’ReAnchor’,’WeightSpace’,’WeightSpace + ReAnchor’],C=[‘#B0AAA6′,’#5B5452′,’#3A8A50′,’#1E4A2C’];
var bars=q(‘#bars’);N.forEach(function(n,i){bars.insertAdjacentHTML(‘beforeend’,'<div class="br"><span>’+n+'</span><div class="trk"><div class="fill" style="background:’+C[i]+’"></div></div><span class="v"></span></div>’)});
var G=[[0,9.3,16.1,18.3],[0,11.6,13.3,17.3],[0,9.5,13.4,17.3],[0,1.5,1.9,6.5]];
var rc=0;
function drawBars(k){rc=k;qa(‘#rsb .btn’).forEach(function(b){b.classList.toggle(‘on’,+b.dataset.r===k)});var r=qa(‘#bars .br’);
r.forEach(function(row,i){var v=D[k][i];var f=row.querySelector(‘.fill’);f.style.width=”0″;var g=i?’ <small>+’+G[k][i].toFixed(1)+'</small>’:”;row.querySelector(‘.v’).innerHTML=v.toFixed(1)+’%’+g;setTimeout(function(){f.style.width=(v/85*100)+’%’},60+i*120)})}
qa(‘#rsb .btn’).forEach(function(b){b.onclick=function(){drawBars(+b.dataset.r)}});drawBars(0);
window.addEventListener(‘load’,post);window.addEventListener(‘resize’,post);setTimeout(post,300);
})();
</script>
</body></html>
“>

(function(){var f=document.getElementById(“mtp-wsp-frame”);window.addEventListener(“message”,function(e){if(e.data&&e.data.mtpWspH&&e.source===f.contentWindow){f.style.height=e.data.mtpWspH+”px”;}});})();

How well does it work? Typed Decisions benchmark results

Cmpnd tested on Typed Decisions, a benchmark for typed probabilistic decisions. A model reads 1 piece of unstructured state and answers 5 typed questions at once. Each answer is a probability distribution. The test split has 400 cases and 2,000 decisions.

Each model got 25 forward passes from GPT-6-Astra. Gains are paired, with a standard error of ±0.7 to 1.4 points.

Model Unoptimized ReAnchor WeightSpace WeightSpace + ReAnchor
d1-omni-600M 50.9% 60.2% (+9.3) 66.9% (+16.1) 69.1% (+18.3)
NanoJev (Qwen3-0.6B) 39.2% 50.7% (+11.6) 52.4% (+13.3) 56.4% (+17.3)
d1-2B 60.6% 70.1% (+9.5) 74.0% (+13.4) 77.9% (+17.3)
Clef-Flash (9B) 68.8% 70.3% (+1.5) 70.7% (+1.9) 75.3% (+6.5)

Sources: Cmpnd blog chart for d1-omni-600M, d1-2B and Clef-Flash; results README for NanoJev.

ReAnchor is an earlier Cmpnd optimizer. Decision models output probabilities, and thresholds turn those into answers. ReAnchor refits each question’s thresholds with no LLM calls. WeightSpace changes the model; ReAnchor recalibrates its output. The 2 stack well.

Three patterns stand out:

  • WeightSpace beat ReAnchor alone on every model shown. On d1-omni-600M, it added +16.1 versus +9.3.
  • The largest model gained least. The 9B Clef-Flash gained only +1.9 from WeightSpace alone.
  • The combination matters most at 9B. Clef-Flash jumped to +6.5 when both optimizers ran.

Does the writer model matter?

The answer is yes. With gpt-6.1-sol as the writer, WeightSpace alone gave +5.8 on d1-omni-600M, +12.1 on NanoJev and −0.6 on d1-2B. Astra produced +16.1, +13.3 and +13.4 on the same models.

Did dspy-wsp bend the original paper’s rules?

Slightly, and Cmpnd says so. By default, the writer can recompute the decision head inside its kernel. Setting simulate_head=False enforces the paper’s stricter rule. Under that rule, d1-omni-600M gained only +2.8 ± 0.7 points. Adding ReAnchor brought it to +9.3, no better than ReAnchor alone.

How do you deploy an edited model?

Each run writes a patch to run_dir. The patch holds manifest.json and edit.npz, with only the changed entries. It also stores sha256 hashes of every base and edited tensor. Anyone with the public base model can rebuild the edited weights exactly.

  • Decision models: compiled.wsp_patch.write_checkpoint(backend.weights_dir, "edited-d1/")
  • Text LLMs: dspy-wsp apply <base> <patch> <out>/, then serve with vLLM.
  • Programs: compiled.save("program.json") and reload with allow_unsafe_lm_state=True.

The edited model runs locally. On Apple silicon it uses MLX. On Linux it uses PyTorch, on an NVIDIA GPU if present.

What models does dspy-wsp support?

Backend Model Alias Architecture
D1Backend Liquid AI d1-omni (text path) "d1" Encoder with decision head
NanoJevBackend NanoJev "nanojev" Qwen3 backbone with decision head
D1LMBackend Liquid AI d1 on LFM3 "d1-2b" LM answering at question positions

Text-generating LLMs work through VLLMBackend (NVIDIA), HFBackend (CUDA, MPS or CPU) and MLXBackend (Apple silicon). A GEPASearch option, contributed by Michael Isaac, adds population search for decision models.

How does WeightSpace compare with other DSPy optimizers?

Metrics WeightSpace (dspy-wsp) ReAnchor BootstrapFinetune GEPA
What changes Model weights Decision thresholds, cuts, weights Model weights Prompt text
How Frontier LLM reasons and edits in a sandbox Grid search over cached probabilities Gradient fine-tuning on bootstrapped traces Reflective prompt evolution
Gradients No No Yes No
LLM calls during optimization Yes (writer, 25 passes) None Teacher for bootstrapping Yes
Typical cost $5.42 to $7.63 per run Near zero Not disclosed Not disclosed
Output Small hashed patch Calibrated program Fine-tuned model Optimized prompts
Source dspy-wsp Cmpnd BetterTogether paper GEPA paper

Key Takeaways

  • Weight-space programming lets a frontier LLM edit weights directly, without gradients or training data.
  • dspy-wsp wraps it as a DSPy optimizer: pip install dspy-wsp, MIT license.
  • 25 writer passes cost $5.42 to $7.63 and took 78 to 167 minutes.
  • d1-omni-600M rose from 50.9% to 69.1% with WeightSpace + ReAnchor.
  • Gains fall at 9B, and the package is explicitly experimental.


Check out the Cmpnd blog post, the GitHub repo, the full results and the original Weight-Space Programming research. You can check the deployment options and specs here through our GPULeaderboard.com chrome extension.

Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.

YOU MAY ALSO LIKE

Samsung May Be Getting Rid Of Another One UI Feature

Smart Glasses Could Replace Your Monitor But They Shouldn’t — Here’s Why

Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us

The post Cmpnd Releases dspy-wsp: Weight-Space Programming Comes to DSPy appeared first on MarkTechPost.

Credit: Source link

ShareTweetSendSharePin

Related Posts

Samsung May Be Getting Rid Of Another One UI Feature
AI & Technology

Samsung May Be Getting Rid Of Another One UI Feature

November 14, 2026
Smart Glasses Could Replace Your Monitor But They Shouldn’t — Here’s Why
AI & Technology

Smart Glasses Could Replace Your Monitor But They Shouldn’t — Here’s Why

November 14, 2026
What Is the NIST AI Risk Management Framework? A Guide for AI Teams – Unite.AI
AI & Technology

What Is the NIST AI Risk Management Framework? A Guide for AI Teams – Unite.AI

November 14, 2026
Anakapalli MoU Maps 400 MW And 1,400 MW AirTrunk Campus Proposals – Unite.AI
AI & Technology

Anakapalli MoU Maps 400 MW And 1,400 MW AirTrunk Campus Proposals – Unite.AI

November 14, 2026
Next Post
83-year-old runner prepares for his 50th consecutive NYC marathon

83-year-old runner prepares for his 50th consecutive NYC marathon

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Search

No Result
View All Result
Impeaching Trump is ‘not our priority,’ says Democratic rep-elect

Impeaching Trump is ‘not our priority,’ says Democratic rep-elect

November 8, 2026
ICE officer indicted for striking immigrant with his vehicle

ICE officer indicted for striking immigrant with his vehicle

November 14, 2026
Trump to possibly replace Hegseth, sources say

Trump to possibly replace Hegseth, sources say

November 9, 2026

About

Learn more

Our Services

Legal

Privacy Policy

Terms of Use

Bloggers

Learn more

Article Links

Contact

Advertise

Ask us anything

©2020- TradePoint.io - All rights reserved!

Tradepoint.io, being just a publishing and technology platform, is not a registered broker-dealer or investment adviser. So we do not provide investment advice. Rather, brokerage services are provided to clients of Tradepoint.io by independent SEC-registered broker-dealers and members of FINRA/SIPC. Every form of investing carries some risk and past performance is not a guarantee of future results. “Tradepoint.io“, “Instant Investing” and “My Trading Tools” are registered trademarks of Apperbuild, LLC.

This website is operated by Apperbuild, LLC. We have no link to any brokerage firm and we do not provide investment advice. Every information and resource we provide is solely for the education of our readers. © 2020 Apperbuild, LLC. All rights reserved.

No Result
View All Result
  • Main
  • AI & Technology
  • Stock Charts
  • Market & News
  • Business
  • Finance Tips
  • Trade Tube
  • Blog
  • Shop

© 2023 - TradePoint.io - All Rights Reserved!