Check Out KNOWNO (Know When You Don’t Know): A Framework that Provides Robots with the Ability To Know When They Don’t Know, Enabling Them to Ask For Help in Ambiguous Situations

Large Language Models (LLMs) are known for their human-like capabilities to generate content, answer questions, and that too with linguistic accuracy and consistency. These models use deep learning techniques and have been trained on large amounts of textual data to perform a number of Natural Language Processing, Natural Language Understanding, and Natural Language Generation tasks. LLMs are able to produce coherent text quickly while understanding and responding to prompts and even learn from a small number of instances. 

For the development of an effective robot, good reasoning skills and the ability to look out for uncertainty and unique environments is most necessary. Though LLMs recently have shown some great improvements in these fields, a limitation of hallucinations still exists. It happens when an AI model produces results that are different from what was anticipated and basically gives results that were not even in the training data the model was trained on. To address the challenge, recently, a team of researchers from Princeton University and Google DeepMind have introduced a framework called Know When You Don’t Know (KNOWNO). KNOWNO solves the issue of hallucinations by quantifying and coordinating the uncertainty of LLM-based planners. It makes it possible for robots to recognize when they are in the wrong and request assistance if needed.

KNOWNO has been made to use the theory of Conformal Prediction (CP) in complicated multi-step planning scenarios to provide statistical guarantees on job completion while minimizing the requirement for human input. KNOWNO is capable of calculating the degree of uncertainty in the predictions made by the LLM-based planner by applying conformal prediction. The robot can select when to seek clarification or more information to increase the dependability of its operations using this uncertainty measurement.

The experiments conducted by the team include real and simulated robot setups with tasks that display various degrees of ambiguity, like linguistic riddles known as Winograd schemas, numerical uncertainties, human preferences, and spatial uncertainties. Upon evaluation, the results have shown that KNOWNO outperforms modern baselines that may rely on ensembles or extensive prompt tuning in terms of improving efficiency and autonomy while providing formal assurances. 

Being a lightweight approach for modeling uncertainties that can scale with the expanding capabilities of foundation models, KNOWNO can be utilized with LLMs ‘out of the box’ without the need for model finetuning. The major contribution is summarized as follows.

1. The authors have used a pre-trained LLM with uncalibrated confidence and a language command to construct a list of potential actions for the robot’s next move. This strategy makes use of LLMs’ capacity to comprehend language and produce plans based on directives.

2. The team has provided theoretical assurances on calibrated confidence for single-step and multi-step planning problems. The robot asks for assistance when necessary and completes tasks accurately in 1−ϵ% of instances with a user-specified level of confidence 1−ϵ. This guarantees that the robot asks for help when there is doubt, increasing the dependability of its activities.

3. Experiments have confirmed KNOWNO’s capacity to deliver statistically guaranteed levels of task accomplishment while requiring 10 to 24% less assistance than baseline methods.

In conclusion, the KNOWNO framework seems promising as it can endow robots with the ability to know when they don’t know, enabling them to ask for help in ambiguous situations.

Check out the Paper and Project Page. Don’t forget to join our 26k+ ML SubReddit, Discord Channel, and Email Newsletter, where we share the latest AI research news, cool AI projects, and more. If you have any questions regarding the above article or if we missed anything, feel free to email us at [email protected]

🚀 Check Out 100’s AI Tools in AI Tools Club