Most of the time, AI companies are locked in a race to the top, treating each other as rivals and competitors. Today, OpenAI and Anthropic revealed that they agreed to evaluate the alignment of each other’s publicly available systems and shared the results of their analyses. The full reports get pretty technical, but are worth a read for anyone who’s following the nuts and bolts of AI development. A broad summary showed some flaws with each company’s offerings, as well as revealing pointers for how to improve future safety tests.
Anthropic said it for “sycophancy, whistleblowing, self-preservation, and supporting human misuse, as well as capabilities related to undermining AI safety evaluations and oversight.” Its review found that o3 and o4-mini models from OpenAI fell in line with results for its own models, but raised concerns about possible misuse with the GPT-4o and GPT-4.1 general-purpose models. The company also said sycophancy was an issue to some degree with all tested models except for o3.
Anthropic’s tests did not include OpenAI’s most recent release. has a feature called Safe Completions, which is meant to protect users and the public against potentially dangerous queries. OpenAI recently faced its after a tragic case where a teenager discussed attempts and plans for suicide with ChatGPT for months before taking his own life.
On the flip side, OpenAI for instruction hierarchy, jailbreaking, hallucinations and scheming. The Claude models generally performed well in instruction hierarchy tests, and had a high refusal rate in hallucination tests, meaning they were less likely to offer answers in cases where uncertainty meant their responses could be wrong.
The move for these companies to conduct a joint assessment is intriguing, particularly since OpenAI allegedly violated Anthropic’s terms of service by having programmers use Claude in the process of building new GPT models, which led to Anthropic OpenAI’s access to its tools earlier this month. But safety with AI tools has become a bigger issue as more critics and legal experts seek guidelines to protect users, particularly minors.
Credit: Source link
