Snowflake unveils Cortex, a managed service to build LLM apps in the data cloud

Today, the Montana-based data-as-a-service and cloud storage company Snowflake announced Cortex, a fully managed service that brings the power of large language models (LLMs) into its data cloud.

Unveiled at the company’s annual Snowday event, Cortex provides enterprises using Snowflake data cloud with a suite of AI building blocks, including open-source LLMs, to analyze data and build applications targeting different business-specific use cases. 

“With Snowflake Cortex, businesses can now tap into…large language models in seconds, build custom LLM-powered apps within minutes, and maintain flexibility and control over their data — while reimagining how all users tap into generative AI to deliver business value,” Sridhar Ramaswamy, SVP of AI at Snowflake, said in a statement.

The offering goes into private preview today and comes bundled with a set of task-specific models, designed to streamline certain functions within the data cloud. Snowflake is also using it for three of its gen AI tools: Snowflake copilot, Universal search and Document AI.

Building LLM apps with Cortex

Today, enterprises want to embrace generative AI, but given the constraints associated with the technology — including the need for AI talent and complex GPU infrastructure management — many find it difficult to bring applications to production. Snowflake Cortex aims to streamline this entire process.

The service provides users with a set of serverless specialized and general-purpose AI functions. Users can access these functions with a call in SQL or Python code and start their journey to functional AI use cases – all running on Cortex’s cost-optimized infrastructure. 

The specialized functions leverage language and machine learning models to let users accelerate specific analytical tasks through natural language inputs. For instance, the models can extract answers, summarize that information or translate it into another another language. In other cases, they can help build a forecast based on data or detect anomalies.

Meanwhile, the general-purpose functions make the broader option that developers can tap into. They cover a variety of models, right from open-source LLMs such as Llama 2 to Snowflake’s own proprietary models, including the one for converting text inputs into SQL for querying data. 

Most importantly, these general-purpose functions also come with vector embedding and search capabilities that allow users to easily contextualize the responses of the model based on their data and create custom applications targeting different use cases. This aspect is handled with Streamlit in Snowflake. 

“This is great for our users because they don’t have to do any provisioning,” Ramaswamy, who founded Neeva, the AI company Snowflake acquired a few months ago, said in a press briefing. “We do the provisioning and deployment. It is just like an API, similar to what OpenAI offers but built right within Snowflake. The data does not leave anywhere and it comes with the kind of guarantees that our customers want and demand, which is that their data is always kept isolated. It’s never intermingled for any kind of cross-customer training. It’s a safe, secure and highly competitive environment,”

Ramaswamy further went on to emphasize that the offering does not require extensive programming. Users just have to operate in the environment of SQL to get things done.

On the application front, he said users can easily build conversational chatbots catered to their business knowledge, like a copilot trained specifically on help content.

Native LLM experiences underpinned by Cortex

While Cortex has just been announced for enterprise use, Snowflake is already using the service to enhance the functionality of its platform with native LLM experiences. The company has launched three Cortex-powered capabilities in private preview: Snowflake copilot, Universal Search and Document AI.

The copilot works as a conversational assistant for the users of the platform, allowing them to ask questions about their data in plain text, write SQL queries against relevant data sets, refine queries and filter down insights and more.

Universal search ropes in LLM-powered search functionality to help users find and start getting value from the most relevant data and apps for their use cases. 

Finally, Document AI helps in extracting information (like invoice amounts or contractual terms) from unstructured documents hosted in the Snowflake data cloud.

Notably, similar capabilities have also been built by other players in the data industry, including Databricks, which recently debuted LakehouseIQ and is one of the biggest competitors of Snowflake.

Informatica and Dremio have also made their respective LLM plays, allowing enterprises to manage their data or query it through natural language inputs. 

More announcements at Snowday 2023

Beyond Cortex, Snowflake announced it is advancing support for Iceberg Tables, enabling users to eliminate silos and unite all their data in the data cloud, and adding new capabilities to its Horizon governance solution.

This includes data quality monitoring, a new interface to understand data lineage, enhanced classification of data and a trust center to streamline cross-cloud security and compliance monitoring.

Finally, the company also announced the launch of a funding program that intends to invest up to $100 million dollars toward early-stage startups building Snowflake native apps.

The program has been backed by its own VC arm as well as multiple venture capital firms including Altimeter Capital, Amplify Partners, Anthos Capital, Coatue, ICONIQ Growth, IVP, Madrona, Menlo Ventures and Redpoint Ventures.