• bitcoinBitcoin(BTC)$64,642.000.15%
  • ethereumEthereum(ETH)$1,823.870.79%
  • tetherTether(USDT)$1.00-0.02%
  • binancecoinBNB(BNB)$589.55-0.12%
  • usd-coinUSDC(USDC)$1.00-0.01%
  • rippleXRP(XRP)$0.960.77%
  • solanaSolana(SOL)$71.08-0.25%
  • tronTRON(TRX)$0.3061120.53%
  • Figure HelocFigure Heloc(FIGR_HELOC)$1.060.00%
  • HyperliquidHyperliquid(HYPE)$66.84-0.27%
  • zcashZcash(ZEC)$886.80-1.19%
  • moneroMonero(XMR)$498.170.85%
  • dogecoinDogecoin(DOGE)$0.0592341.66%
  • leo-tokenLEO Token(LEO)$9.210.64%
  • USDSUSDS(USDS)$1.000.02%
  • whitebitWhiteBIT Coin(WBT)$62.550.13%
  • chainlinkChainlink(LINK)$9.16-0.07%
  • cardanoCardano(ADA)$0.152407-0.13%
  • RainRain(RAIN)$0.00720794.20%
  • Ethena USDeEthena USDe(USDE)$1.00-0.02%
  • stellarStellar(XLM)$0.136790-0.95%
  • daiDai(DAI)$1.000.00%
  • nearNEAR Protocol(NEAR)$3.171.09%
  • bitcoin-cashBitcoin Cash(BCH)$206.37-0.41%
  • litecoinLitecoin(LTC)$44.60-0.65%
  • CantonCanton(CC)$0.082173-0.75%
  • avalanche-2Avalanche(AVAX)$7.13-0.43%
  • tether-goldTether Gold(XAUT)$3,842.320.53%
  • uniswapUniswap(UNI)$5.11-0.48%
  • USD1USD1(USD1)$1.000.06%
  • the-open-networkGram (prev. Toncoin)(GRAM)$1.050.74%
  • suiSui(SUI)$0.720.38%
  • Ripple USDRipple USD(RLUSD)$1.000.00%
  • Global DollarGlobal Dollar(USDG)$1.000.00%
  • hedera-hashgraphHedera(HBAR)$0.057587-0.25%
  • paypal-usdPayPal USD(PYUSD)$1.000.00%
  • Ondo US Dollar YieldOndo US Dollar Yield(USDY)$1.150.04%
  • shiba-inuShiba Inu(SHIB)$0.000004-0.59%
  • quant-networkQuant(QNT)$155.971.18%
  • crypto-com-chainCronos(CRO)$0.043171-0.22%
  • BittensorBittensor(TAO)$189.30-0.40%
  • Circle USYCCircle USYC(USYC)$1.140.00%
  • Spiko Amundi Overnight Swap Fund (EUR)Spiko Amundi Overnight Swap Fund (EUR)(EURSAFO)$1.12-0.14%
  • MemeCoreMemeCore(M)$0.852.77%
  • okbOKB(OKB)$85.05-2.69%
  • pax-goldPAX Gold(PAXG)$3,844.760.50%
  • aaveAave(AAVE)$108.271.33%
  • Falcon USDFalcon USD(USDF)$1.000.01%
  • BlackRock USD Institutional Digital Liquidity FundBlackRock USD Institutional Digital Liquidity Fund(BUIDL)$1.000.00%
  • United StablesUnited Stables(U)$1.00-0.01%
TradePoint.io
  • Main
  • AI & Technology
  • Stock Charts
  • Market & News
  • Business
  • Finance Tips
  • Trade Tube
  • Blog
  • Shop
No Result
View All Result
TradePoint.io
No Result
View All Result

CMU Researchers Explore Expert Guidance and Strategic Deviations in Multi-Agent Imitation Learning

July 31, 2024
in AI & Technology
Reading Time: 5 mins read
A A
CMU Researchers Explore Expert Guidance and Strategic Deviations in Multi-Agent Imitation Learning
ShareShareShareShareShare

The problem of a mediator learning to coordinate a group of strategic agents is considered through action recommendations without knowing their underlying utility functions, such as routing drivers through a road network. The challenge lies in the difficulty of manually specifying the quality of these recommendations, making it necessary to provide the mediator with data on desired coordination behavior. This transforms the problem into one of multi-agent imitation learning (MAIL). A fundamental question in MAIL is identifying the right objective for the learner, explored through the development of personalized route recommendations for users.

Current research to solve the challenges in multi-agent imitation learning includes several methodologies. Single-agent imitation Learning techniques like behavioral cloning reduce imitation to supervised learning but suffer from covariate shifts, leading to compounding errors. Interactive approaches like inverse reinforcement learning (RL) allow learners to observe the consequences of their actions, preventing compounding errors but are sample-inefficient. The next approach is multi-agent imitation learning in which the concept of the regret gap has been explored but not utilized fully in Markov Games. The third approach, Inverse game theory focuses on recovering utility functions rather than learning coordination from demonstrations.

YOU MAY ALSO LIKE

Alecto Raises $20 Million to Build an AI-Native Consumer Law Firm – Unite.AI

Why Microsoft Is Ditching A Few Of Its 365 Companion Apps In 2026

Researchers from Carnegie Mellon University have proposed an alternative objective for multi-agent imitation learning (MAIL) in Markov Games called the regret gap, which explicitly accounts for potential deviations by agents in the group. They investigated the relationship between the value and regret gaps, showing that while the value gap can be minimized using single-agent imitation learning (IL) algorithms, it does not prevent the regret gap from becoming arbitrarily large. This finding indicates that achieving regret equivalence is more challenging than achieving value equivalence in MAIL. To address this, two efficient reductions are developed to no-regret online convex optimization, (a) MALICE, under a coverage assumption on the expert, and (b) BLADES, with access to a queryable expert.

Although the value gap is considered a ‘weaker’ objective, it can be a reasonable learning objective in real-world applications where agents are non-strategic. The natural multi-agent generalization of single-agent imitation learning algorithms can efficiently minimize the value gap, making it relatively easy to achieve in MAIL. Two such single-agent IL algorithms, Behavior Cloning (BC) and Inverse Reinforcement Learning (IRL), are used to minimize the value gap. These algorithms run over joint policies where BC and IRL are applied to the multi-agent setting, becoming Joint Behavior Cloning (J-BC) and Joint Inverse Reinforcement Learning (J-IRL). These adaptations result in the same value gap bounds as in the single-agent setting.

Multi-agent Aggregation of Losses to Imitate Cached Experts (MALICE), is an efficient algorithm extending the ALICE algorithm to the multi-agent setting. ALICE is an interactive algorithm that uses importance sampling to re-weight the BC loss based on the density ratio between the current learner policy and that of the expert. It requires full demonstration coverage to ensure finite importance weights. ALICE utilizes a no-regret algorithm to learn a policy that minimizes reweighed on-policy error, ensuring a linear-in-H bound on the value gap under a recoverability assumption. MALICE adapts these principles to multi-agent environments, providing a robust solution for minimizing the regret gap.

In conclusion, researchers from Carnegie Mellon University have introduced an alternative objective for MAIL in Markov Games called the regret gap. For strategic agents that are not mere puppets, another source of distribution shift arises from deviations by agents within the population. This shift cannot be efficiently controlled through environmental interaction, such as inverse RL. So, it requires estimating the expert’s actions in counterfactual states. Utilizing this insight, the researchers derived two reductions that can minimize the regret gap under a coverage or queryable expert assumption. Future work includes developing and implementing practical approximations of these idealized algorithms.


Check out the Paper. All credit for this research goes to the researchers of this project. Also, don’t forget to follow us on Twitter and join our Telegram Channel and LinkedIn Group. If you like our work, you will love our newsletter..

Don’t Forget to join our 47k+ ML SubReddit

Find Upcoming AI Webinars here


Sajjad Ansari is a final year undergraduate from IIT Kharagpur. As a Tech enthusiast, he delves into the practical applications of AI with a focus on understanding the impact of AI technologies and their real-world implications. He aims to articulate complex AI concepts in a clear and accessible manner.


Credit: Source link

ShareTweetSendSharePin

Related Posts

Alecto Raises  Million to Build an AI-Native Consumer Law Firm – Unite.AI
AI & Technology

Alecto Raises $20 Million to Build an AI-Native Consumer Law Firm – Unite.AI

November 16, 2026
Why Microsoft Is Ditching A Few Of Its 365 Companion Apps In 2026
AI & Technology

Why Microsoft Is Ditching A Few Of Its 365 Companion Apps In 2026

November 16, 2026
AMD Closes World Labs Acquisition and Names Fei-Fei Li Chief Scientist – Unite.AI
AI & Technology

AMD Closes World Labs Acquisition and Names Fei-Fei Li Chief Scientist – Unite.AI

November 16, 2026
Hands-On (And Off) With Mercedes-Benz’s Upcoming MB.Drive Assist Pro System
AI & Technology

Hands-On (And Off) With Mercedes-Benz’s Upcoming MB.Drive Assist Pro System

November 16, 2026
Next Post
Democratic Rep. Henry Cuellar indicted on bribery charges

Democratic Rep. Henry Cuellar indicted on bribery charges

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Search

No Result
View All Result
LIVE: Fed Chair Warsh holds press conference on interest rate decision | NBC News

LIVE: Fed Chair Warsh holds press conference on interest rate decision | NBC News

November 15, 2026
Martha Stewart apologizes after Halloween costume backlash

Martha Stewart apologizes after Halloween costume backlash

November 12, 2026
ICE officer indicted for allegedly hitting man with car

ICE officer indicted for allegedly hitting man with car

November 13, 2026

About

Learn more

Our Services

Legal

Privacy Policy

Terms of Use

Bloggers

Learn more

Article Links

Contact

Advertise

Ask us anything

©2020- TradePoint.io - All rights reserved!

Tradepoint.io, being just a publishing and technology platform, is not a registered broker-dealer or investment adviser. So we do not provide investment advice. Rather, brokerage services are provided to clients of Tradepoint.io by independent SEC-registered broker-dealers and members of FINRA/SIPC. Every form of investing carries some risk and past performance is not a guarantee of future results. “Tradepoint.io“, “Instant Investing” and “My Trading Tools” are registered trademarks of Apperbuild, LLC.

This website is operated by Apperbuild, LLC. We have no link to any brokerage firm and we do not provide investment advice. Every information and resource we provide is solely for the education of our readers. © 2020 Apperbuild, LLC. All rights reserved.

No Result
View All Result
  • Main
  • AI & Technology
  • Stock Charts
  • Market & News
  • Business
  • Finance Tips
  • Trade Tube
  • Blog
  • Shop

© 2023 - TradePoint.io - All Rights Reserved!