Meet BOSS: A Reinforcement Learning (RL) Framework that Trains Agents to Solve New Tasks in New Environments with LLM Guidance

Introducing BOSS (Bootstrapping your own SkillS): a groundbreaking approach that leverages large language models to autonomously build a versatile skill library for tackling intricate tasks with minimal guidance. Compared to conventional unsupervised skill acquisition techniques and simplistic bootstrapping methods, BOSS performs better in executing unfamiliar tasks within novel environments. This innovation marks a significant leap in autonomous skill acquisition and application.

Reinforcement learning seeks to optimize policies in Markov Decision Processes for maximizing expected returns—past RL research pre-trained reusable skills for complex tasks. Unsupervised RL, focusing on curiosity, controllability, and diversity, learned skills without human input. The language was used for skill parameterization and open-loop planning. BOSS extends skill repertoires with large language models, guiding exploration and rewarding skill chain completion, yielding higher success rates in long-horizon task execution.

Traditional robot learning relies heavily on supervision, while humans excel at learning complex tasks independently. Researchers introduced BOSS as a framework to acquire diverse long-horizon skills with minimal human intervention autonomously. Through skill bootstrapping and guided by large language models (LLMs), BOSS progressively builds and combines skills to handle complex tasks. Unsupervised environment interactions enhance its policy robustness for solving challenging tasks in new environments.

BOSS introduces a two-phase framework. In the first phase, it acquires a foundational skill set using unsupervised RL objectives. The second phase, skill bootstrapping, employs LLMs to guide skill chaining and rewards based on skill completion. This approach allows agents to construct complex behaviors from basic skills. Experiments in household environments show that LLM-guided bootstrapping outperforms naïve bootstrapping and prior unsupervised methods in executing unfamiliar long-horizon tasks in new settings.

Experimental findings confirm BOSS, guided by LLMs, excels in solving extended household tasks in novel settings, surpassing prior LLM-based planning and unsupervised exploration methods. Results present inter-quartile means and standard deviations of oracle-normalized returns and oracle-normalized success rates for tasks of varying lengths in ALFRED evaluations. LLM-guided bootstrapping-trained agents outperform those from naïve bootstrapping and prior unsupervised methods. BOSS can autonomously acquire diverse, complex behaviors from basic skills, showcasing its potential for expert-free robotic skill acquisition.

The BOSS framework, guided by LLMs, excels in autonomously solving intricate tasks without expert guidance. LLM-guided bootstrapping-trained agents outperform naive bootstrapping and prior unsupervised methods when executing unfamiliar functions in new environments. Realistic household experiments confirm BOSS’s effectiveness in acquiring diverse, complex behaviors from basic skills, emphasizing its potential for autonomous robotics skill acquisition. BOSS also demonstrates promise in connecting reinforcement learning with natural language understanding, utilising pre-trained language models for guided learning.

Future research directions may include:

• Investigating reset-free RL for autonomous skill learning.
• Proposing long-horizon task breakdown with BOSS’s skill-chaining approach.
• Expanding unsupervised RL for low-level skill acquisition.

Enhancing the integration of reinforcement learning with natural language understanding in the BOSS framework is also a promising avenue. Applying BOSS to diverse domains and evaluating its performance in various environments and task contexts offers potential for further exploration.

Check out the Paper and Project. All Credit For This Research Goes To the Researchers on This Project. Also, don’t forget to join our 31k+ ML SubReddit, 40k+ Facebook Community, Discord Channel, and Email Newsletter, where we share the latest AI research news, cool AI projects, and more.

If you like our work, you will love our newsletter..

We are also on WhatsApp. Join our AI Channel on Whatsapp..