UniLLMRec: An End-to-End LLM-Centered Recommendation Framework to Execute Multi-Stage Recommendation Tasks Through Chain-of-Recommendations

The goal of recommender systems is to predict user preferences based on historical data. Mainly, they are designed in sequential pipelines and require lots of data to train different sub-systems, making it hard to scale to new domains. Recently, Large Language Models (LLMs)  such as ChatGPT and Claude have demonstrated remarkable generalized capabilities, enabling a singular model to tackle diverse recommendation tasks across various scenarios. However,  these systems face challenges in presenting large-scale item sets to LLMs in natural language format due to the constraint of input length.

In prior research, recommendation tasks have been approached within the natural language generation framework. These methods involve fine-tuning LLMs to address various recommendation scenarios through Parameter Efficient Fine Tuning (PEFT), including approaches such as LoRA and P-tuning. However, in these approaches, three key challenges exist: challenge 1: though claiming to be efficient, these fine-tuning techniques heavily rely on substantial amounts of training data, which can be costly and time-consuming to obtain.  challenge 2: They tend to under-utilize the strong general or multi-task capabilities of LLMs. Challenge 3: They lack the ability to effectively present a large-scale item corpus to LLMs in a natural language format.

Researchers from the City University of Hong Kong and Huawei Noah’s Ark Lab propose UniLLMRec, an innovative framework that capitalizes on a singular LLM to seamlessly perform items recall, ranking, and re-ranking within a unified end-to-end recommendation framework. A key advantage of UniLLMRec lies in its utilization of the inherent zero-shot capabilities of LLMs, which eliminates the need for training or fine-tuning. Hence, UniLLMRec offers a more streamlined and resource-efficient solution compared to traditional systems, facilitating more effective and scalable implementations across a variety of recommendation contexts. 

To ensure that UniLLMRec can effectively handle a large-scale item corpus, researchers have developed a unique tree-based recall strategy. Specifically, this involves constructing a tree that organizes items based on semantic attributes such as categories, subcategories, and keywords, creating a manageable hierarchy from an extensive list of items. Each leaf node in this tree encompasses a manageable subset of the complete item inventory, enabling efficient traversal from the root to the appropriate leaf nodes. Hence, one can only search items from the selected leaf nodes. This approach sharply contrasts with traditional methods that require searching through the entire item list, resulting in a significant optimization of the recall process. Existing LLM-based systems mainly focus on the ranking stage in the recommender system, and they rank only a small number of candidate items. In comparison, UniLLMRec is a comprehensive framework that unitizes LLM to integrate multi-stage tasks (e.g., recall, ranking, re-ranking) by chain of recommendation.

The results obtained by UniLLMRec can be concluded as:

• Both UniLLMRec (GPT-3.5) and UniLLMRec (GPT-4), which do not require training, achieve competitive performance compared with conventional recommendation models that require training. 
• UniLLMRec (GPT-4) significantly outperforms UniLLMRec (GPT3.5). The enhanced semantic understanding and language processing capabilities of UniLLMRec (GPT-4) make it proficient in utilizing project trees to complete the entire recommendation process.
• UniLLMRec (GPT-3.5) exhibits a performance decrease in the Amazon dataset due to the challenge of addressing the imbalance in the item tree and the limited information available in the item title index. However, UniLLMRec (GPT-4) continues to perform superiorly on Amazon. 
• UniLLMRec with both backbones can effectively enhance the diversity of recommendations. UniLLMRec (GPT-3.5) tends to provide more homogeneous items than UniLLMRec (GPT-4).

In conclusion, this research introduces UniLLMRec, the first end-to-end LLM-centered recommendation framework to execute multi-stage recommendation tasks (e.g., recall, ranking, re-ranking) through a chain of recommendations. To deal with large-scale item sets, researchers design an innovative strategy to structure all items into a hierarchical tree structure, i.e., item tree. The item tree can be dynamically updated to incorporate new items and effectively retrieved according to user interests. Based on the item tree, LLM effectively reduces the candidate item set by utilizing this hierarchical structure for search. UniLLMRec achieves competitive performance compared to conventional recommendation models. 

Check out the Paper. All credit for this research goes to the researchers of this project. Also, don’t forget to follow us on Twitter. Join our Telegram Channel, Discord Channel, and LinkedIn Group.

If you like our work, you will love our newsletter..

Don’t Forget to join our 39k+ ML SubReddit