• bitcoinBitcoin(BTC)$64,954.001.11%
  • ethereumEthereum(ETH)$1,821.591.24%
  • tetherTether(USDT)$1.000.00%
  • binancecoinBNB(BNB)$593.411.10%
  • usd-coinUSDC(USDC)$1.000.00%
  • rippleXRP(XRP)$0.972.25%
  • solanaSolana(SOL)$73.684.04%
  • tronTRON(TRX)$0.3055210.65%
  • Figure HelocFigure Heloc(FIGR_HELOC)$1.040.00%
  • HyperliquidHyperliquid(HYPE)$69.526.34%
  • zcashZcash(ZEC)$949.999.17%
  • moneroMonero(XMR)$516.232.47%
  • dogecoinDogecoin(DOGE)$0.0594451.77%
  • leo-tokenLEO Token(LEO)$9.281.18%
  • USDSUSDS(USDS)$1.000.00%
  • whitebitWhiteBIT Coin(WBT)$62.730.88%
  • chainlinkChainlink(LINK)$9.271.94%
  • cardanoCardano(ADA)$0.1548592.25%
  • stellarStellar(XLM)$0.1409582.71%
  • Ethena USDeEthena USDe(USDE)$1.000.02%
  • daiDai(DAI)$1.000.00%
  • nearNEAR Protocol(NEAR)$3.283.86%
  • bitcoin-cashBitcoin Cash(BCH)$211.613.58%
  • RainRain(RAIN)$0.005580-14.48%
  • litecoinLitecoin(LTC)$45.172.06%
  • CantonCanton(CC)$0.082456-1.88%
  • uniswapUniswap(UNI)$5.395.97%
  • avalanche-2Avalanche(AVAX)$7.231.93%
  • tether-goldTether Gold(XAUT)$3,844.43-0.15%
  • suiSui(SUI)$0.743.11%
  • USD1USD1(USD1)$1.000.01%
  • the-open-networkGram (prev. Toncoin)(GRAM)$1.03-0.47%
  • Ripple USDRipple USD(RLUSD)$1.000.01%
  • Global DollarGlobal Dollar(USDG)$1.000.00%
  • hedera-hashgraphHedera(HBAR)$0.0579620.23%
  • Ondo US Dollar YieldOndo US Dollar Yield(USDY)$1.15-0.02%
  • paypal-usdPayPal USD(PYUSD)$1.000.00%
  • crypto-com-chainCronos(CRO)$0.0447374.88%
  • BittensorBittensor(TAO)$194.854.09%
  • shiba-inuShiba Inu(SHIB)$0.0000041.09%
  • quant-networkQuant(QNT)$156.261.56%
  • MemeCoreMemeCore(M)$0.9111.06%
  • Circle USYCCircle USYC(USYC)$1.140.00%
  • Spiko Amundi Overnight Swap Fund (EUR)Spiko Amundi Overnight Swap Fund (EUR)(EURSAFO)$1.12-0.20%
  • okbOKB(OKB)$87.033.77%
  • aaveAave(AAVE)$113.975.12%
  • pax-goldPAX Gold(PAXG)$3,847.59-0.11%
  • Falcon USDFalcon USD(USDF)$1.00-0.01%
  • United StablesUnited Stables(U)$1.000.01%
  • pepePepe(PEPE)$0.0000041.94%
TradePoint.io
  • Main
  • AI & Technology
  • Stock Charts
  • Market & News
  • Business
  • Finance Tips
  • Trade Tube
  • Blog
  • Shop
No Result
View All Result
TradePoint.io
No Result
View All Result

Can Language Models Reason Beyond Words? Exploring Implicit Reasoning in Multi-Layer Hidden States for Complex Tasks

November 15, 2023
in AI & Technology
Reading Time: 5 mins read
A A
Can Language Models Reason Beyond Words? Exploring Implicit Reasoning in Multi-Layer Hidden States for Complex Tasks
ShareShareShareShareShare

Large Language Models (LLMs) have shown remarkable capabilities in tasks like language understanding and reasoning, marking a paradigm shift in how we interact with AI systems. To augment the proficiency of LLMs, researchers generally employ the chain of thought prompting technique, which involves intermediate reasoning steps to guide the model’s response. Although this technique is similar to how humans solve a problem, it does not fully utilize the computational prowess of LLMs, and the authors of this paper have tried to explore an alternate reasoning approach.

Chain of thought (CoT) methods have shown great results, but the downside to their use is that they delay the generation of the desired final answer. The researchers have introduced a new approach called implicit chain-of-though that, as the name suggests, makes the steps involved in CoT reasoning implicit so that the model produces the final answer directly.

Unlike explicit CoT reasoning, where the LLM is trained to produce the intermediate steps before the final output, in implicit CoT reasoning, the model sees the intermediate steps only during the training phase and not during testing. It processes these steps in its internal states and learns to internalize the concept thoroughly, bypassing explicit reasoning.

The researchers used a ‘teacher training’ method instead of the traditional ‘teacher forcing’ method to achieve implicit CoT reasoning. Their strategy first involves training a student model to read the teacher’s hidden states and utilize some of them to produce the final answer. They then employ knowledge distillation, a process of transferring knowledge from a larger model to a smaller one. They train an emulator to predict the teacher’s hidden states based on input. Importantly, this emulation happens vertically across the model’s layers, eliminating the need for explicit reasoning steps.

The final step involves combining the emulator with the student, which produces the final output based on the emulated teacher’s thought process. The integrated system is then optimized end-to-end, enabling the student model to develop its own reasoning methods, which may differ from the teacher’s.

The researchers conducted experiments on two tasks – multi-digit multiplication and grade school math problems. The results showed that their method equipped the models to solve previously unsolvable tasks without explicit CoT. They observed that the GPT-2 Small model, which achieved 97% accuracy on 4-digit multiplication under implicit CoT, performed poorly when tested on 5-digit multiplications, which suggests that the effectiveness of the technique is dependent on having sufficient intermediate layers for the required calculations. They also observed that the implicit CoT technique has a higher inference speed, especially for tasks that require multiple intermediate steps.

A few major issues associated with this technique are the lack of transparency, heavy dependence on the teacher’s thought processes, and lagging in performance compared to explicit CoT. However, this work marks just an initial step toward building implicit CoT, and the researchers believe that many adjustments could be built on top of this work to optimize this process further and augment LLMs’ ability to reason.


Check out the Paper. All credit for this research goes to the researchers of this project. Also, don’t forget to join our 33k+ ML SubReddit, 41k+ Facebook Community, Discord Channel, and Email Newsletter, where we share the latest AI research news, cool AI projects, and more.

If you like our work, you will love our newsletter..

We are also on Telegram and WhatsApp.


YOU MAY ALSO LIKE

Sarvam Launches Sarvam Token Factory: Bringing India-Hosted Inference to 8 Open Models

Adam Mosseri Thinks It Will Be ‘Pretty Challenging’ To Design Instagram For The iPhone Duo

I am a Civil Engineering Graduate (2022) from Jamia Millia Islamia, New Delhi, and I have a keen interest in Data Science, especially Neural Networks and their application in various areas.


🔥 Join The AI Startup Newsletter To Learn About Latest AI Startups

Credit: Source link

ShareTweetSendSharePin

Related Posts

Sarvam Launches Sarvam Token Factory: Bringing India-Hosted Inference to 8 Open Models
AI & Technology

Sarvam Launches Sarvam Token Factory: Bringing India-Hosted Inference to 8 Open Models

November 17, 2026
Adam Mosseri Thinks It Will Be ‘Pretty Challenging’ To Design Instagram For The iPhone Duo
AI & Technology

Adam Mosseri Thinks It Will Be ‘Pretty Challenging’ To Design Instagram For The iPhone Duo

November 17, 2026
TRMNL Frame Is A 00 Full-Color E-Paper Display That Looks Like Real Art
AI & Technology

TRMNL Frame Is A $1300 Full-Color E-Paper Display That Looks Like Real Art

November 17, 2026
ElevenLabs to Acquire Kotoba for Japanese Voice AI Expansion – Unite.AI
AI & Technology

ElevenLabs to Acquire Kotoba for Japanese Voice AI Expansion – Unite.AI

November 17, 2026
Next Post
Morning News NOW Full Broadcast – Nov. 8

Morning News NOW Full Broadcast – Nov. 8

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Search

No Result
View All Result
Restaurant chain may rethink controversial 20% service charge in some venues after customer pushback

Restaurant chain may rethink controversial 20% service charge in some venues after customer pushback

November 14, 2026
Iranian plots to kill Americans in Middle East have reached ‘advanced stages,’ intel memo warns

Iranian plots to kill Americans in Middle East have reached ‘advanced stages,’ intel memo warns

November 17, 2026
Cute babies and otters dress up and play for Halloween

Cute babies and otters dress up and play for Halloween

November 13, 2026

About

Learn more

Our Services

Legal

Privacy Policy

Terms of Use

Bloggers

Learn more

Article Links

Contact

Advertise

Ask us anything

©2020- TradePoint.io - All rights reserved!

Tradepoint.io, being just a publishing and technology platform, is not a registered broker-dealer or investment adviser. So we do not provide investment advice. Rather, brokerage services are provided to clients of Tradepoint.io by independent SEC-registered broker-dealers and members of FINRA/SIPC. Every form of investing carries some risk and past performance is not a guarantee of future results. “Tradepoint.io“, “Instant Investing” and “My Trading Tools” are registered trademarks of Apperbuild, LLC.

This website is operated by Apperbuild, LLC. We have no link to any brokerage firm and we do not provide investment advice. Every information and resource we provide is solely for the education of our readers. © 2020 Apperbuild, LLC. All rights reserved.

No Result
View All Result
  • Main
  • AI & Technology
  • Stock Charts
  • Market & News
  • Business
  • Finance Tips
  • Trade Tube
  • Blog
  • Shop

© 2023 - TradePoint.io - All Rights Reserved!