• bitcoinBitcoin(BTC)$63,931.00-1.61%
  • ethereumEthereum(ETH)$1,790.53-1.79%
  • tetherTether(USDT)$1.00-0.02%
  • binancecoinBNB(BNB)$589.35-0.48%
  • usd-coinUSDC(USDC)$1.00-0.01%
  • rippleXRP(XRP)$0.991.05%
  • solanaSolana(SOL)$74.250.48%
  • tronTRON(TRX)$0.3096301.06%
  • Figure HelocFigure Heloc(FIGR_HELOC)$1.01-3.05%
  • HyperliquidHyperliquid(HYPE)$67.67-3.22%
  • zcashZcash(ZEC)$930.59-2.67%
  • moneroMonero(XMR)$506.08-0.86%
  • dogecoinDogecoin(DOGE)$0.059337-0.29%
  • leo-tokenLEO Token(LEO)$9.29-0.03%
  • USDSUSDS(USDS)$1.000.00%
  • whitebitWhiteBIT Coin(WBT)$61.76-1.59%
  • chainlinkChainlink(LINK)$9.12-2.15%
  • cardanoCardano(ADA)$0.153479-1.22%
  • stellarStellar(XLM)$0.1408230.31%
  • Ethena USDeEthena USDe(USDE)$1.00-0.03%
  • daiDai(DAI)$1.00-0.01%
  • nearNEAR Protocol(NEAR)$3.332.20%
  • bitcoin-cashBitcoin Cash(BCH)$208.53-1.90%
  • RainRain(RAIN)$0.005091-8.15%
  • litecoinLitecoin(LTC)$44.17-1.88%
  • CantonCanton(CC)$0.0831391.28%
  • avalanche-2Avalanche(AVAX)$7.09-2.48%
  • uniswapUniswap(UNI)$5.19-4.52%
  • tether-goldTether Gold(XAUT)$3,768.62-2.04%
  • suiSui(SUI)$0.72-2.25%
  • USD1USD1(USD1)$1.000.02%
  • the-open-networkGram (prev. Toncoin)(GRAM)$1.02-1.80%
  • Ripple USDRipple USD(RLUSD)$1.000.00%
  • Global DollarGlobal Dollar(USDG)$1.000.00%
  • hedera-hashgraphHedera(HBAR)$0.057017-1.67%
  • paypal-usdPayPal USD(PYUSD)$1.000.01%
  • Ondo US Dollar YieldOndo US Dollar Yield(USDY)$1.15-0.13%
  • BittensorBittensor(TAO)$194.800.08%
  • shiba-inuShiba Inu(SHIB)$0.000004-0.57%
  • crypto-com-chainCronos(CRO)$0.043281-2.85%
  • quant-networkQuant(QNT)$153.64-2.11%
  • MemeCoreMemeCore(M)$0.88-3.93%
  • Circle USYCCircle USYC(USYC)$1.140.01%
  • Spiko Amundi Overnight Swap Fund (EUR)Spiko Amundi Overnight Swap Fund (EUR)(EURSAFO)$1.12-0.17%
  • okbOKB(OKB)$87.140.46%
  • aaveAave(AAVE)$109.63-3.19%
  • pax-goldPAX Gold(PAXG)$3,769.51-2.09%
  • Falcon USDFalcon USD(USDF)$1.000.04%
  • United StablesUnited Stables(U)$1.00-0.09%
  • BlackRock USD Institutional Digital Liquidity FundBlackRock USD Institutional Digital Liquidity Fund(BUIDL)$1.000.00%
TradePoint.io
  • Main
  • AI & Technology
  • Stock Charts
  • Market & News
  • Business
  • Finance Tips
  • Trade Tube
  • Blog
  • Shop
No Result
View All Result
TradePoint.io
No Result
View All Result

Stanford study challenges assumptions about language models: Larger context doesn’t mean better understanding 

July 21, 2023
in AI & Technology
Reading Time: 4 mins read
A A
Stanford study challenges assumptions about language models: Larger context doesn’t mean better understanding 
ShareShareShareShareShare

Head over to our on-demand library to view sessions from VB Transform 2023. Register Here


A study released this month by researchers from Stanford University, UC Berkeley and Samaya AI has found that large language models (LLMs) often fail to access and use relevant information given to them in longer context windows.

YOU MAY ALSO LIKE

Plaid Connects Financial Accounts to Instinct’s AI Assistant – Unite.AI

NASA’s Voyager 1 Space Probe Is Now One Light-Day Away From Earth

In language models, a context window refers to the length of text a model can process and respond to in a given instance. It can be thought of as a working memory for a particular text analysis or chatbot conversation.

The study caught widespread attention last week after its release because many developers and other users experimenting with LLMs had assumed that the trend toward larger context windows would continue to improve LLM performance and their usefulness across various applications.

>>Don’t miss our special issue: The Future of the data center: Handling greater and greater demands.<<

Event

VB Transform 2023 On-Demand

Did you miss a session from VB Transform 2023? Register to access the on-demand library for all of our featured sessions.

 

Register Now

If an LLM could take an entire document or article as input for its context window, the conventional thinking went, the LLM could provide perfect comprehension of the full scope of that document when asked questions about it. 

Assumptions around context window flawed

LLM companies like Anthropic have fueled excitement around the idea of longer content windows, where users can provide ever more input to be analyzed or summarized. Anthropic just released a new model called Claude 2, which provides a huge 100k token context window, and said it can enable new use cases such as summarizing long conversations or drafting memos and op-eds.

But the study shows that some assumptions around the context window are flawed when it comes to the LLM’s ability to search and analyze it accurately. 

The study found that LLMs performed best “when relevant information occurs at the beginning or end of the input context, and significantly degrades when models must access relevant information in the middle of long contexts. Furthermore, performance substantially decreases as the input context grows longer, even for explicitly long-context models.”

Last week, industry insiders like Bob Wiederhold, COO of vector database company Pinecone, cited the study as evidence that stuffing entire documents into a document window for doing things like search and analysis won’t be the panacea many had hoped for. 

Semantic search preferable to document stuffing

Vector databases like Pinecone help developers increase LLM memory by searching for relevant information to pull into the context window. Wiederhold pointed to the study as evidence that vector databases will remain viable for the foreseeable future, since the study suggests semantic search provided by vector databases is better than document stuffing. 

Stanford University’s Nelson Liu, study lead author, agreed that if you try to inject an entire PDF into a language model context window and then ask questions about the document, a vector database search will generally be more efficient to use.

“If you’re searching over large amounts of documents, you want to be using something that’s built for search, at least for now,” said Liu. 

Liu cautioned, however, that the study isn’t necessarily claiming that sticking entire documents into a context window won’t work. Results will depend specifically on the sort of content contained in the documents the LLMs are analyzing. Language models are bad at differentiating between many things that are closely related or which seem relevant, Liu explained. But they are good at finding the one thing that is clearly relevant when most other things are not relevant.

“So I think it’s a bit more nuanced than ‘You should always use a vector database, or you should never use a vector database’,” he said.

Language models’ best use case: Generating content

Liu said his study assumed that most commercial applications are operating in a setting where they use some sort of vector database to help return multiple possible results into a context window. The study found that having more results in the context window didn’t always improve performance. 

As a specialist in language processing, Liu said he was surprised that people were thinking of using a context window to search for content, or to aggregate or synthesize it, although he said he could understand why people would want to. He said people should continue to think of language models as best used to generate content, and search engines as best to search content. 

“The hope that you can just throw everything into a language model and just sort of pray it works, I don’t think we’re there yet,” he said. “But maybe we’ll be there in a few years or even a few months. It’s not super clear to me how fast this space will move, but I think right now, language models aren’t going to replace vector databases and search engines.”

VentureBeat’s mission is to be a digital town square for technical decision-makers to gain knowledge about transformative enterprise technology and transact. Discover our Briefings.

Credit: Source link

ShareTweetSendSharePin

Related Posts

Plaid Connects Financial Accounts to Instinct’s AI Assistant – Unite.AI
AI & Technology

Plaid Connects Financial Accounts to Instinct’s AI Assistant – Unite.AI

November 18, 2026
NASA’s Voyager 1 Space Probe Is Now One Light-Day Away From Earth
AI & Technology

NASA’s Voyager 1 Space Probe Is Now One Light-Day Away From Earth

November 18, 2026
Marble Draws .5M in Seed Funding After Signing 150+ Restaurants – Unite.AI
AI & Technology

Marble Draws $5.5M in Seed Funding After Signing 150+ Restaurants – Unite.AI

November 18, 2026
Palmer Luckey Says His SLAB Seastead Is ‘Way Cooler’ Than A Floating Data Center
AI & Technology

Palmer Luckey Says His SLAB Seastead Is ‘Way Cooler’ Than A Floating Data Center

November 18, 2026
Next Post
Corona Light, Coors Light gain traction over Bud Light

Corona Light, Coors Light gain traction over Bud Light

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Search

No Result
View All Result
Secretive hedge fund RenTech dodges Nvidia bloodbath

Secretive hedge fund RenTech dodges Nvidia bloodbath

November 17, 2026
U.S. attorney warns Sen. Elizabeth Warren against investigating Trump associates – NBC News

U.S. attorney warns Sen. Elizabeth Warren against investigating Trump associates – NBC News

November 16, 2026
Democrats push into deep red states to flip Senate

Democrats push into deep red states to flip Senate

November 11, 2026

About

Learn more

Our Services

Legal

Privacy Policy

Terms of Use

Bloggers

Learn more

Article Links

Contact

Advertise

Ask us anything

©2020- TradePoint.io - All rights reserved!

Tradepoint.io, being just a publishing and technology platform, is not a registered broker-dealer or investment adviser. So we do not provide investment advice. Rather, brokerage services are provided to clients of Tradepoint.io by independent SEC-registered broker-dealers and members of FINRA/SIPC. Every form of investing carries some risk and past performance is not a guarantee of future results. “Tradepoint.io“, “Instant Investing” and “My Trading Tools” are registered trademarks of Apperbuild, LLC.

This website is operated by Apperbuild, LLC. We have no link to any brokerage firm and we do not provide investment advice. Every information and resource we provide is solely for the education of our readers. © 2020 Apperbuild, LLC. All rights reserved.

No Result
View All Result
  • Main
  • AI & Technology
  • Stock Charts
  • Market & News
  • Business
  • Finance Tips
  • Trade Tube
  • Blog
  • Shop

© 2023 - TradePoint.io - All Rights Reserved!