• bitcoinBitcoin(BTC)$63,953.00-1.62%
  • ethereumEthereum(ETH)$1,792.86-1.53%
  • tetherTether(USDT)$1.00-0.01%
  • binancecoinBNB(BNB)$588.49-0.89%
  • usd-coinUSDC(USDC)$1.000.00%
  • rippleXRP(XRP)$0.980.98%
  • solanaSolana(SOL)$74.160.40%
  • tronTRON(TRX)$0.3092841.11%
  • Figure HelocFigure Heloc(FIGR_HELOC)$1.010.00%
  • HyperliquidHyperliquid(HYPE)$68.52-1.45%
  • zcashZcash(ZEC)$932.82-1.82%
  • moneroMonero(XMR)$503.33-2.77%
  • dogecoinDogecoin(DOGE)$0.059221-0.51%
  • leo-tokenLEO Token(LEO)$9.280.05%
  • USDSUSDS(USDS)$1.000.01%
  • whitebitWhiteBIT Coin(WBT)$61.76-1.60%
  • chainlinkChainlink(LINK)$9.13-1.94%
  • cardanoCardano(ADA)$0.152959-1.49%
  • stellarStellar(XLM)$0.140385-0.47%
  • Ethena USDeEthena USDe(USDE)$1.00-0.03%
  • daiDai(DAI)$1.000.00%
  • nearNEAR Protocol(NEAR)$3.332.02%
  • bitcoin-cashBitcoin Cash(BCH)$207.88-2.28%
  • litecoinLitecoin(LTC)$44.04-2.64%
  • CantonCanton(CC)$0.083304-0.53%
  • RainRain(RAIN)$0.004659-15.72%
  • avalanche-2Avalanche(AVAX)$7.09-2.34%
  • uniswapUniswap(UNI)$5.21-4.02%
  • tether-goldTether Gold(XAUT)$3,751.92-2.36%
  • suiSui(SUI)$0.72-2.37%
  • the-open-networkGram (prev. Toncoin)(GRAM)$1.02-0.55%
  • USD1USD1(USD1)$1.000.03%
  • Ripple USDRipple USD(RLUSD)$1.00-0.02%
  • Global DollarGlobal Dollar(USDG)$1.000.00%
  • hedera-hashgraphHedera(HBAR)$0.057151-2.19%
  • paypal-usdPayPal USD(PYUSD)$1.000.00%
  • Ondo US Dollar YieldOndo US Dollar Yield(USDY)$1.15-0.02%
  • BittensorBittensor(TAO)$194.56-0.33%
  • shiba-inuShiba Inu(SHIB)$0.000004-1.07%
  • crypto-com-chainCronos(CRO)$0.043250-3.41%
  • quant-networkQuant(QNT)$154.14-1.60%
  • MemeCoreMemeCore(M)$0.88-2.38%
  • Circle USYCCircle USYC(USYC)$1.140.01%
  • Spiko Amundi Overnight Swap Fund (EUR)Spiko Amundi Overnight Swap Fund (EUR)(EURSAFO)$1.12-0.27%
  • okbOKB(OKB)$87.140.13%
  • aaveAave(AAVE)$110.51-3.17%
  • Falcon USDFalcon USD(USDF)$1.000.04%
  • pax-goldPAX Gold(PAXG)$3,751.70-2.46%
  • United StablesUnited Stables(U)$1.00-0.05%
  • BlackRock USD Institutional Digital Liquidity FundBlackRock USD Institutional Digital Liquidity Fund(BUIDL)$1.000.00%
TradePoint.io
  • Main
  • AI & Technology
  • Stock Charts
  • Market & News
  • Business
  • Finance Tips
  • Trade Tube
  • Blog
  • Shop
No Result
View All Result
TradePoint.io
No Result
View All Result

Microsoft and Columbia Researchers Propose LLM-AUGMENTER: An AI System that Augments a Black-Box LLM with a Set of Plug-and-Play Modules

July 21, 2023
in AI & Technology
Reading Time: 4 mins read
A A
Microsoft and Columbia Researchers Propose LLM-AUGMENTER: An AI System that Augments a Black-Box LLM with a Set of Plug-and-Play Modules
ShareShareShareShareShare

Large language models (LLMs) like GPT-3 are widely recognized for their ability to generate coherent and informative natural language texts due to their vast amount of world knowledge. However, encoding this knowledge in LLMs is lossy and can lead to memory distortion, resulting in hallucinations that can be detrimental to mission-critical tasks. Additionally, LLMs cannot encode all necessary information for some applications, making them unsuitable for time-sensitive tasks like news question answering. Although various methods have been proposed to enhance LLMs using external knowledge, these typically require fine-tuning LLM parameters, which can be prohibitively expensive. Consequently, there is a need for plug-and-play modules that can be added to a fixed LLM to improve its performance in mission-critical tasks.

The paper proposes a system called LLM-AUGMENTER that addresses the challenges of applying Large Language Models (LLMs) to mission-critical applications. The system is designed to augment a black-box LLM with plug-and-play modules to ground its responses in external knowledge stored in task-specific databases. It also includes iterative prompt revision using feedback generated by utility functions to improve the factuality score of LLM-generated responses. The system’s effectiveness is validated empirically in task-oriented dialog and open-domain question-answering scenarios, where it significantly reduces hallucinations without sacrificing the fluency and informativeness of reactions. The source code and models of the system are publicly available.

The LLM-Augmenter process involves three main steps. Firstly, when given a user query, it retrieves evidence from external knowledge sources such as web searches or task-specific databases. It can also connect the retrieved raw evidence with relevant context and reason on the concatenation to create “evidence chains.” Secondly, the LLM-Augmenter prompts a fixed LLM like ChatGPT by using the consolidated evidence to generate a response rooted in evidence. Lastly, LLM-Augmenter checks the generated response and creates a corresponding feedback message. This feedback message modifies and iterates the ChatGPT query until the candidate’s response meets verification requirements.

🚀 Build high-quality training datasets with Kili Technology and solve NLP machine learning challenges to develop powerful ML applications

The work presented in this study shows that the LLM-Augmenter approach can effectively augment black-box LLMs with external knowledge pertinent to their interactions with users. This augmentation greatly reduces the problem of hallucinations without compromising the fluency and informative quality of the responses generated by the LLMs.

LLM-AUGMENTER’s performance was evaluated on information-seeking dialog tasks using both automatic metrics and human evaluations. Commonly used metrics, such as Knowledge F1 (KF1) and BLEU-4, were used to assess the overlap between the model’s output and the ground-truth human response and the overlap with the knowledge that the human used as a reference during dataset collection. Additionally, the researchers included these metrics that best correlate with human judgment on the DSTC9 and DSTC11 customer support tasks. Other metrics, such as BLEURT, BERTScore, chrF, and BARTScore, were also considered, as they are among the best-performing text generation metrics on the dialog.


Check out the Paper and Project. All Credit For This Research Goes To the Researchers on This Project. Also, don’t forget to join our 26k+ ML SubReddit, Discord Channel, and Email Newsletter, where we share the latest AI research news, cool AI projects, and more.


YOU MAY ALSO LIKE

Plaid Connects Financial Accounts to Instinct’s AI Assistant – Unite.AI

NASA’s Voyager 1 Space Probe Is Now One Light-Day Away From Earth

Niharika is a Technical consulting intern at Marktechpost. She is a third year undergraduate, currently pursuing her B.Tech from Indian Institute of Technology(IIT), Kharagpur. She is a highly enthusiastic individual with a keen interest in Machine learning, Data science and AI and an avid reader of the latest developments in these fields.


🔥 Gain a competitive
edge with data: Actionable market intelligence for global brands, retailers, analysts, and investors. (Sponsored)

Credit: Source link

ShareTweetSendSharePin

Related Posts

Plaid Connects Financial Accounts to Instinct’s AI Assistant – Unite.AI
AI & Technology

Plaid Connects Financial Accounts to Instinct’s AI Assistant – Unite.AI

November 18, 2026
NASA’s Voyager 1 Space Probe Is Now One Light-Day Away From Earth
AI & Technology

NASA’s Voyager 1 Space Probe Is Now One Light-Day Away From Earth

November 18, 2026
Marble Draws .5M in Seed Funding After Signing 150+ Restaurants – Unite.AI
AI & Technology

Marble Draws $5.5M in Seed Funding After Signing 150+ Restaurants – Unite.AI

November 18, 2026
Palmer Luckey Says His SLAB Seastead Is ‘Way Cooler’ Than A Floating Data Center
AI & Technology

Palmer Luckey Says His SLAB Seastead Is ‘Way Cooler’ Than A Floating Data Center

November 18, 2026
Next Post
How to Balance Work, Studying and Having Fun

How to Balance Work, Studying and Having Fun

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Search

No Result
View All Result
Crew uses ‘R2D2’ robot to rescue kitten from drain pipe

Crew uses ‘R2D2’ robot to rescue kitten from drain pipe

November 18, 2026
Elizabeth Holmes advocate threatens suing A24 over documentary portrayal

Elizabeth Holmes advocate threatens suing A24 over documentary portrayal

November 15, 2026
Will the DOJ’s Latest Charges Against James Comey Stick?; Anthropic Heads to Wall Street – Oct. 27

Will the DOJ’s Latest Charges Against James Comey Stick?; Anthropic Heads to Wall Street – Oct. 27

November 15, 2026

About

Learn more

Our Services

Legal

Privacy Policy

Terms of Use

Bloggers

Learn more

Article Links

Contact

Advertise

Ask us anything

©2020- TradePoint.io - All rights reserved!

Tradepoint.io, being just a publishing and technology platform, is not a registered broker-dealer or investment adviser. So we do not provide investment advice. Rather, brokerage services are provided to clients of Tradepoint.io by independent SEC-registered broker-dealers and members of FINRA/SIPC. Every form of investing carries some risk and past performance is not a guarantee of future results. “Tradepoint.io“, “Instant Investing” and “My Trading Tools” are registered trademarks of Apperbuild, LLC.

This website is operated by Apperbuild, LLC. We have no link to any brokerage firm and we do not provide investment advice. Every information and resource we provide is solely for the education of our readers. © 2020 Apperbuild, LLC. All rights reserved.

No Result
View All Result
  • Main
  • AI & Technology
  • Stock Charts
  • Market & News
  • Business
  • Finance Tips
  • Trade Tube
  • Blog
  • Shop

© 2023 - TradePoint.io - All Rights Reserved!