• bitcoinBitcoin(BTC)$64,596.000.46%
  • ethereumEthereum(ETH)$1,808.79-0.58%
  • tetherTether(USDT)$1.00-0.02%
  • binancecoinBNB(BNB)$590.71-0.10%
  • usd-coinUSDC(USDC)$1.00-0.01%
  • rippleXRP(XRP)$1.004.20%
  • solanaSolana(SOL)$74.663.07%
  • tronTRON(TRX)$0.3084971.02%
  • Figure HelocFigure Heloc(FIGR_HELOC)$1.01-3.07%
  • HyperliquidHyperliquid(HYPE)$68.41-0.67%
  • zcashZcash(ZEC)$945.353.30%
  • moneroMonero(XMR)$505.17-0.61%
  • dogecoinDogecoin(DOGE)$0.0595901.39%
  • leo-tokenLEO Token(LEO)$9.300.14%
  • USDSUSDS(USDS)$1.000.00%
  • whitebitWhiteBIT Coin(WBT)$62.370.18%
  • chainlinkChainlink(LINK)$9.220.88%
  • cardanoCardano(ADA)$0.1543581.04%
  • stellarStellar(XLM)$0.1413312.95%
  • Ethena USDeEthena USDe(USDE)$1.00-0.03%
  • daiDai(DAI)$1.00-0.01%
  • nearNEAR Protocol(NEAR)$3.343.18%
  • bitcoin-cashBitcoin Cash(BCH)$208.87-0.18%
  • litecoinLitecoin(LTC)$44.54-0.10%
  • CantonCanton(CC)$0.0849185.04%
  • avalanche-2Avalanche(AVAX)$7.14-0.31%
  • uniswapUniswap(UNI)$5.23-1.28%
  • RainRain(RAIN)$0.004338-20.43%
  • the-open-networkGram (prev. Toncoin)(GRAM)$1.073.45%
  • tether-goldTether Gold(XAUT)$3,759.79-2.47%
  • suiSui(SUI)$0.731.21%
  • USD1USD1(USD1)$1.000.00%
  • Ripple USDRipple USD(RLUSD)$1.000.00%
  • Global DollarGlobal Dollar(USDG)$1.000.00%
  • hedera-hashgraphHedera(HBAR)$0.0574751.69%
  • paypal-usdPayPal USD(PYUSD)$1.00-0.01%
  • Ondo US Dollar YieldOndo US Dollar Yield(USDY)$1.15-0.04%
  • BittensorBittensor(TAO)$196.202.71%
  • crypto-com-chainCronos(CRO)$0.043810-1.61%
  • shiba-inuShiba Inu(SHIB)$0.0000040.45%
  • quant-networkQuant(QNT)$154.930.90%
  • MemeCoreMemeCore(M)$0.88-2.65%
  • Circle USYCCircle USYC(USYC)$1.140.01%
  • Spiko Amundi Overnight Swap Fund (EUR)Spiko Amundi Overnight Swap Fund (EUR)(EURSAFO)$1.12-0.36%
  • okbOKB(OKB)$86.55-0.38%
  • aaveAave(AAVE)$111.59-1.16%
  • pax-goldPAX Gold(PAXG)$3,760.76-2.54%
  • Falcon USDFalcon USD(USDF)$1.000.07%
  • United StablesUnited Stables(U)$1.00-0.07%
  • BlackRock USD Institutional Digital Liquidity FundBlackRock USD Institutional Digital Liquidity Fund(BUIDL)$1.000.00%
TradePoint.io
  • Main
  • AI & Technology
  • Stock Charts
  • Market & News
  • Business
  • Finance Tips
  • Trade Tube
  • Blog
  • Shop
No Result
View All Result
TradePoint.io
No Result
View All Result

Do Models like GPT-4 Behave Safely When Given the Ability to Act?: This AI Paper Introduces MACHIAVELLI Benchmark to Improve Machine Ethics and Build Safer Adaptive Agents

July 18, 2023
in AI & Technology
Reading Time: 4 mins read
A A
Do Models like GPT-4 Behave Safely When Given the Ability to Act?: This AI Paper Introduces MACHIAVELLI Benchmark to Improve Machine Ethics and Build Safer Adaptive Agents
ShareShareShareShareShare

Natural language processing is one area where AI systems are making rapid strides, and it is important that the models need to be rigorously tested and guided toward safer behavior to reduce deployment risks. Prior evaluation metrics for such sophisticated systems focused on measuring language comprehension or reasoning in vacuums. But now, models are being taught for actual, interactive work. This means that benchmarks need to evaluate how models perform in social settings.

Interactive agents can be put through their paces in text-based games. Agents need planning abilities and the ability to grasp the natural language to progress in these games. Agents’ immoral tendencies should be considered alongside their technical talents while setting benchmarks.

A new work by the University of California, Center For AI Safety, Carnegie Mellon University, and Yale University proposes the Measuring Agents’ Competence & Harmfulness In A Vast Environment of Long-horizon Language Interactions (MACHIAVELLI) benchmark. MACHIAVELLI is an advancement in evaluating an agent’s capacity for planning in naturalistic social settings. The setting is inspired by text-based Choose Your Own Adventure games available at choiceofgames.com, which actual humans developed. These games feature high-level decisions while giving agents realistic objectives while abstracting away low-level environment interactions.

🚀 Automate labeling to save time with smart tools & model predictions

The environment reports the degree to which agent acts are dishonest, lower utility, and seek power, among other behavioral qualities, to keep tabs on unethical behavior. The team achieves this by following the below-mentioned steps:

  1. Operationalizing these behaviors as mathematical formulas
  2. Densely annotating social notions in the games, such as characters’ wellbeing
  3. Using the annotations and formulas to produce a numerical score for each behavior. 

They demonstrate empirically that GPT-4 (OpenAI, 2023) is more effective at collecting annotations than human annotators.

Artificial intelligence agents face the same internal conflict as humans do. Like language models trained for next-token prediction often produce toxic text, artificial agents trained for goal optimization often exhibit immoral and power-seeking behaviors. Amorally trained agents may develop Machiavellian strategies for maximizing their rewards at the expense of others and the environment. By encouraging agents to act morally, this trade-off can be improved.

The team discovers that moral training (nudging the agent to be more ethical) decreases the incidence of harmful activity for language-model agents. Furthermore, behavioral regularization restricts undesirable behavior in both agents without substantially decreasing reward. This work contributes to the development of trustworthy sequential decision-makers.

The researchers try techniques like an artificial conscience and ethics prompts to control agents. Agents can be guided to display less Machiavellian behavior, although much progress remains possible. They advocate for more research into these trade-offs and emphasize expanding the Pareto frontier rather than chasing after limited rewards.


Check out the Paper. All Credit For This Research Goes To the Researchers on This Project. Also, don’t forget to join our 18k+ ML SubReddit, Discord Channel, and Email Newsletter, where we share the latest AI research news, cool AI projects, and more.

🚀 Check Out 100’s AI Tools in AI Tools Club


YOU MAY ALSO LIKE

Anthropic Adds Code Mode to Claude Code: Claude Now Chains Tool Calls in JavaScript

YouTube Will Start Boosting Public View Counts When You Watch On A TV

Tanushree Shenwai is a consulting intern at MarktechPost. She is currently pursuing her B.Tech from the Indian Institute of Technology(IIT), Bhubaneswar. She is a Data Science enthusiast and has a keen interest in the scope of application of artificial intelligence in various fields. She is passionate about exploring the new advancements in technologies and their real-life application.


🔥 StoryBird.ai just dropped some amazing features. Generate an illustrated story from a prompt. Check it out here. (Sponsored)

Credit: Source link

ShareTweetSendSharePin

Related Posts

Anthropic Adds Code Mode to Claude Code: Claude Now Chains Tool Calls in JavaScript
AI & Technology

Anthropic Adds Code Mode to Claude Code: Claude Now Chains Tool Calls in JavaScript

November 18, 2026
YouTube Will Start Boosting Public View Counts When You Watch On A TV
AI & Technology

YouTube Will Start Boosting Public View Counts When You Watch On A TV

November 18, 2026
Recast Brings AI Troubleshooting to Right Click Tools With MCP Server – Unite.AI
AI & Technology

Recast Brings AI Troubleshooting to Right Click Tools With MCP Server – Unite.AI

November 18, 2026
Nexalus and RENDA Launch Nova8 Liquid-Cooled AI Server for Data Center Retrofits – Unite.AI
AI & Technology

Nexalus and RENDA Launch Nova8 Liquid-Cooled AI Server for Data Center Retrofits – Unite.AI

November 18, 2026
Next Post
Cable TV Users Can Choose Channels With New Verizon Service

Cable TV Users Can Choose Channels With New Verizon Service

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Search

No Result
View All Result
You.com Launches Extract API for Document Retrieval and PDF Parsing – Unite.AI

You.com Launches Extract API for Document Retrieval and PDF Parsing – Unite.AI

November 17, 2026
First Lady Melania Trump secures ceasefire for Ukrainian city

First Lady Melania Trump secures ceasefire for Ukrainian city

November 17, 2026
Restaurant chain may rethink controversial 20% service charge in some venues after customer pushback

Restaurant chain may rethink controversial 20% service charge in some venues after customer pushback

November 14, 2026

About

Learn more

Our Services

Legal

Privacy Policy

Terms of Use

Bloggers

Learn more

Article Links

Contact

Advertise

Ask us anything

©2020- TradePoint.io - All rights reserved!

Tradepoint.io, being just a publishing and technology platform, is not a registered broker-dealer or investment adviser. So we do not provide investment advice. Rather, brokerage services are provided to clients of Tradepoint.io by independent SEC-registered broker-dealers and members of FINRA/SIPC. Every form of investing carries some risk and past performance is not a guarantee of future results. “Tradepoint.io“, “Instant Investing” and “My Trading Tools” are registered trademarks of Apperbuild, LLC.

This website is operated by Apperbuild, LLC. We have no link to any brokerage firm and we do not provide investment advice. Every information and resource we provide is solely for the education of our readers. © 2020 Apperbuild, LLC. All rights reserved.

No Result
View All Result
  • Main
  • AI & Technology
  • Stock Charts
  • Market & News
  • Business
  • Finance Tips
  • Trade Tube
  • Blog
  • Shop

© 2023 - TradePoint.io - All Rights Reserved!