Patronus AI Introduces the Industry’s First Multimodal LLM-as-a-Judge (MLLM-as-a-Judge): Designed to Evaluate and Optimize AI Systems that Convert Image Inputs into Text Outputs

​In recent years, the integration of image generation technologies into various platforms has opened new avenues for enhancing user experiences. However, as these multimodal AI systems—capable of processing and generating multiple data forms like text and images—expand, challenges such as “caption hallucination” have emerged. This phenomenon occurs when AI-generated descriptions of images contain inaccuracies or irrelevant details, potentially diminishing user trust and engagement. Traditional methods of evaluating these systems often rely on manual inspection, which is neither scalable nor efficient, highlighting the need for automated and reliable evaluation tools tailored to multimodal AI applications.​

Addressing these challenges, Patronus AI has introduced the industry’s first Multimodal LLM-as-a-Judge (MLLM-as-a-Judge), designed to evaluate and optimize AI systems that convert image inputs into text outputs. This tool utilizes Google’s Gemini model, selected for its balanced judgment approach and consistent scoring distribution, distinguishing it from alternatives like OpenAI’s GPT-4V, which has shown higher levels of egocentricity. The MLLM-as-a-Judge aligns with Patronus AI’s commitment to advancing scalable oversight of AI systems, providing developers with the means to assess and enhance the performance of their multimodal applications.

Technically, the MLLM-as-a-Judge is equipped to process and evaluate image-to-text generation tasks. It offers built-in evaluators that create a ground truth snapshot of images by analyzing attributes such as text presence and location, grid structures, spatial orientation, and object identification. The suite of evaluators includes criteria like:​

• caption-describes-primary-object​
• caption-describes-non-primary-objects​
• caption-hallucination​
• caption-hallucination-strict​
• caption-mentions-primary-object-location​

These evaluators enable a thorough assessment of image captions, ensuring that generated descriptions accurately reflect the visual content. Beyond verifying caption accuracy, the MLLM-as-a-Judge can be used to test the relevance of product screenshots in response to user queries, validate the accuracy of Optical Character Recognition (OCR) extractions for tabular data, and assess the fidelity of AI-generated brand images and logos. ​

A practical application of the MLLM-as-a-Judge is its implementation by Etsy, a prominent e-commerce platform specializing in handmade and vintage products. Etsy’s AI team employs generative AI to automatically generate captions for product images uploaded by sellers, streamlining the listing process. However, they encountered quality issues with their multimodal AI systems, as the autogenerated captions often contained errors and unexpected outputs. To address this, Etsy integrated Judge-Image, a component of the MLLM-as-a-Judge, to evaluate and optimize their image captioning system. This integration allowed Etsy to reduce caption hallucinations, thereby improving the accuracy of product descriptions and enhancing the overall user experience. ​

In conclusion, as organizations continue to adopt and scale multimodal AI systems, addressing the unpredictability of these systems becomes essential. Patronus AI’s MLLM-as-a-Judge offers an automated solution to evaluate and optimize image-to-text AI applications, mitigating issues such as caption hallucination. By providing built-in evaluators and leveraging advanced models like Google Gemini, the MLLM-as-a-Judge enables developers and organizations to enhance the reliability and accuracy of their multimodal AI systems, ultimately fostering greater user trust and engagement.

Check out the Technical Details. All credit for this research goes to the researchers of this project. Also, feel free to follow us on Twitter and don’t forget to join our 80k+ ML SubReddit.