• bitcoinBitcoin(BTC)$59,800.003.53%
  • ethereumEthereum(ETH)$1,685.303.27%
  • tetherTether(USDT)$1.000.01%
  • binancecoinBNB(BNB)$546.154.51%
  • usd-coinUSDC(USDC)$1.000.00%
  • rippleXRP(XRP)$0.933.98%
  • solanaSolana(SOL)$69.773.04%
  • Figure HelocFigure Heloc(FIGR_HELOC)$1.073.34%
  • tronTRON(TRX)$0.3042650.49%
  • HyperliquidHyperliquid(HYPE)$61.235.21%
  • zcashZcash(ZEC)$794.177.43%
  • moneroMonero(XMR)$478.335.32%
  • dogecoinDogecoin(DOGE)$0.0548363.67%
  • leo-tokenLEO Token(LEO)$9.160.46%
  • whitebitWhiteBIT Coin(WBT)$57.532.98%
  • chainlinkChainlink(LINK)$8.643.46%
  • USDSUSDS(USDS)$1.000.03%
  • cardanoCardano(ADA)$0.1441164.00%
  • Ethena USDeEthena USDe(USDE)$1.000.04%
  • stellarStellar(XLM)$0.1284084.08%
  • daiDai(DAI)$1.00-0.02%
  • bitcoin-cashBitcoin Cash(BCH)$196.974.18%
  • nearNEAR Protocol(NEAR)$2.927.73%
  • litecoinLitecoin(LTC)$39.411.26%
  • avalanche-2Avalanche(AVAX)$6.604.28%
  • CantonCanton(CC)$0.072813-0.31%
  • Ripple USDRipple USD(RLUSD)$1.000.01%
  • tether-goldTether Gold(XAUT)$3,540.860.73%
  • USD1USD1(USD1)$1.000.06%
  • uniswapUniswap(UNI)$4.687.82%
  • the-open-networkGram (prev. Toncoin)(GRAM)$0.963.33%
  • suiSui(SUI)$0.655.65%
  • Global DollarGlobal Dollar(USDG)$1.000.02%
  • Ondo US Dollar YieldOndo US Dollar Yield(USDY)$1.150.10%
  • RainRain(RAIN)$0.003319-7.94%
  • paypal-usdPayPal USD(PYUSD)$1.000.01%
  • hedera-hashgraphHedera(HBAR)$0.0510613.05%
  • BittensorBittensor(TAO)$189.837.43%
  • shiba-inuShiba Inu(SHIB)$0.0000043.13%
  • crypto-com-chainCronos(CRO)$0.0406043.34%
  • Spiko Amundi Overnight Swap Fund (EUR)Spiko Amundi Overnight Swap Fund (EUR)(EURSAFO)$1.12-0.02%
  • quant-networkQuant(QNT)$139.336.08%
  • Circle USYCCircle USYC(USYC)$1.140.00%
  • MemeCoreMemeCore(M)$0.77-1.69%
  • Falcon USDFalcon USD(USDF)$1.000.01%
  • okbOKB(OKB)$79.083.74%
  • United StablesUnited Stables(U)$1.000.06%
  • pax-goldPAX Gold(PAXG)$3,544.010.73%
  • BlackRock USD Institutional Digital Liquidity FundBlackRock USD Institutional Digital Liquidity Fund(BUIDL)$1.000.00%
  • aaveAave(AAVE)$96.460.41%
TradePoint.io
  • Main
  • AI & Technology
  • Stock Charts
  • Market & News
  • Business
  • Finance Tips
  • Trade Tube
  • Blog
  • Shop
No Result
View All Result
TradePoint.io
No Result
View All Result

RA3: Mid-Training with Temporal Action Abstractions for Faster Reinforcement Learning (RL) Post-Training in Code LLMs

October 9, 2025
in AI & Technology
Reading Time: 4 mins read
A A
RA3: Mid-Training with Temporal Action Abstractions for Faster Reinforcement Learning (RL) Post-Training in Code LLMs
ShareShareShareShareShare

TL;DR: A new research from Apple, formalizes what “mid-training” should do before reinforcement learning RL post-training and introduces RA3 (Reasoning as Action Abstractions)—an EM-style procedure that learns temporally consistent latent actions from expert traces, then fine-tunes on those bootstrapped traces. It shows mid-training should (1) prune to a compact near-optimal action subspace and (2) shorten the effective planning horizon, improving RL convergence. Empirically, RA3 improves HumanEval/MBPP by ~8/4 points over base/NTP and accelerates RLVR on HumanEval+, MBPP+, LiveCodeBench, and Codeforces.

What does the research present?

The research team present the first formal treatment of how mid-training shapes post-training reinforcement learning RL: they breakdown outcomes into (i) pruning efficiency—how well mid-training selects a compact near-optimal action subset that shapes the initial policy prior—and (ii) RL convergence—how quickly post-training improves within that restricted set. The analysis argues mid-training is most effective when the decision space is compact and the effective horizon is short, favoring temporal abstractions over primitive next-token actions.

YOU MAY ALSO LIKE

Tesla Rolls Out FSD V15 with Reality View and Vision Upgrades – Unite.AI

OpenRouter Cuts Three Providers After AI Prompt Exposure Findings – Unite.AI

https://arxiv.org/pdf/2509.25810

Algorithm: RA3 in one pass

RA3 derives a sequential variational lower bound (a temporal ELBO) and optimizes it with an EM-like loop:

  • E-step (latent discovery): use RL to infer temporally consistent latent structures (abstractions) aligned to expert sequences.
  • M-step (model update): perform next-token prediction on the bootstrapped, latent-annotated traces to make those abstractions part of the model’s policy.

Results: code generation and RLVR

On Python code tasks, the research team reports that across multiple base models, RA3 improves average pass@k on HumanEval and MBPP by ~8 and ~4 points over the base model and an NTP mid-training baseline. In post-training, RLVR converges faster and to higher final performance on HumanEval+, MBPP+, LiveCodeBench, and Codeforces when initialized from RA3. These are mid- and post-training effects respectively; the evaluation scope is code generation.

Key Takeaways

  1. The research team formalizes mid-training via two determinants—pruning efficiency and impact on RL convergence—arguing effectiveness rises when the decision space is compact and the effective horizon is short.
  2. RA3 optimizes a sequential variational lower bound by iteratively discovering temporally consistent latent structures with RL and then fine-tuning on bootstrapped traces (EM-style).
  3. On code generation, RA3 reports ~+8 (HumanEval) and ~+4 (MBPP) average pass@k gains over base/NTP mid-training baselines across several model scales.
  4. Initializing post-training with RA3 accelerates RLVR convergence and improves asymptotic performance on HumanEval+, MBPP+, LiveCodeBench, and Codeforces.

Editorial Comments

RA3’s contribution is concrete and narrow: it formalizes mid-training around two determinants—pruning efficiency and RL convergence—and operationalizes them via a temporal ELBO optimized in an EM loop to learn persistent action abstractions before RLVR. The researchers report ~+8 (HumanEval) and ~+4 (MBPP) average pass@k gains over base/NTP and faster RLVR convergence on HumanEval+, MBPP+, LiveCodeBench, and Codeforces.


Check out the Technical Paper. Feel free to check out our GitHub Page for Tutorials, Codes and Notebooks. Also, feel free to follow us on Twitter and don’t forget to join our 100k+ ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.

The post RA3: Mid-Training with Temporal Action Abstractions for Faster Reinforcement Learning (RL) Post-Training in Code LLMs appeared first on MarkTechPost.

Credit: Source link

ShareTweetSendSharePin

Related Posts

Tesla Rolls Out FSD V15 with Reality View and Vision Upgrades – Unite.AI
AI & Technology

Tesla Rolls Out FSD V15 with Reality View and Vision Upgrades – Unite.AI

November 22, 2026
OpenRouter Cuts Three Providers After AI Prompt Exposure Findings – Unite.AI
AI & Technology

OpenRouter Cuts Three Providers After AI Prompt Exposure Findings – Unite.AI

November 22, 2026
Is It Actually Worth Upgrading To The Pixel 11?
AI & Technology

Is It Actually Worth Upgrading To The Pixel 11?

November 22, 2026
What’s The Difference Between Them?
AI & Technology

What’s The Difference Between Them?

November 22, 2026
Next Post
Yankees left in disbelief: 'Didn't finish the goal' – ESPN

Yankees left in disbelief: 'Didn't finish the goal' - ESPN

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Search

No Result
View All Result
Can your ‘digital twin’ help to predict and prevent sickness?

Can your ‘digital twin’ help to predict and prevent sickness?

November 17, 2026
CoreWeave Boosts Forecast, Easing Slowdown Concerns

CoreWeave Boosts Forecast, Easing Slowdown Concerns

November 15, 2026
Odd Lots: The AI Trade Shrugs Off a Blue Wave and a Tech Scandal

Odd Lots: The AI Trade Shrugs Off a Blue Wave and a Tech Scandal

November 15, 2026

About

Learn more

Our Services

Legal

Privacy Policy

Terms of Use

Bloggers

Learn more

Article Links

Contact

Advertise

Ask us anything

©2020- TradePoint.io - All rights reserved!

Tradepoint.io, being just a publishing and technology platform, is not a registered broker-dealer or investment adviser. So we do not provide investment advice. Rather, brokerage services are provided to clients of Tradepoint.io by independent SEC-registered broker-dealers and members of FINRA/SIPC. Every form of investing carries some risk and past performance is not a guarantee of future results. “Tradepoint.io“, “Instant Investing” and “My Trading Tools” are registered trademarks of Apperbuild, LLC.

This website is operated by Apperbuild, LLC. We have no link to any brokerage firm and we do not provide investment advice. Every information and resource we provide is solely for the education of our readers. © 2020 Apperbuild, LLC. All rights reserved.

No Result
View All Result
  • Main
  • AI & Technology
  • Stock Charts
  • Market & News
  • Business
  • Finance Tips
  • Trade Tube
  • Blog
  • Shop

© 2023 - TradePoint.io - All Rights Reserved!