Zillow, the real estate technology company, doesn’t get one conversation with its customers. They move from a phone screen to a loan officer to a real estate agent, sometimes over months or years, and expect the context to follow them. A single chatbot could never carry that thread.
At VB Transform 2026, Zillow SVP of Engineering Toby Roberts and Glean co-founder and CEO Arvind Jain described how they built AI architecture meant to carry context across that entire journey — and why context, not raw data, turned out to be the harder problem to solve. Zillow’s products touch roughly 80% of U.S. real estate transactions each year, and the company has been using AI long before ChatGPT existed.
“We pretty quickly identified that we were going to need a persistent context layer that was going to meet our customers and the professionals wherever they were,” Roberts said.
Data was never the hard part
Roberts said Zillow’s AI effort started where most enterprise AI efforts start, with the data itself.
“We started with a large push around making sure our data did have the right foundation,” Roberts said. That meant a data mesh approach, clear data lineage and a governance structure with permissions and identity attached to the data itself.
None of that turned out to be the hard problem. The hard problem was building something that remembered where a customer was in their journey and carried that forward, no matter which surface they showed up on next.
“This context layer has to live to be able to support you where you are at any given point in your journey,” Roberts said. Zillow chose to own that layer itself rather than depend on a single external chat interface, a decision Roberts said the team reached quickly once it looked at the shape of a real transaction rather than a single conversation.
Why Zillow built its own architecture, and where Glean fits into it
Zillow built its own harness rather than route customers through a single model API. The team drew on 20 years of machine learning history behind products like Zestimate, leaning into smaller, task-specific fine-tuned models instead of one general-purpose model.
Internally, that harness runs alongside Glean. Roberts said Zillow now has thousands of Glean agents in production, handling repetitive tasks with tens of thousands of executions across the company. Glean’s pitch, per Jain, is centralizing that integration work once, through the Glean MCP gateway, rather than letting finance, legal and marketing each rebuild their own connections to the same systems.
That centralization is also a cost lever. Jain pointed to two mechanisms: model routing, which sends most tasks to smaller, cheaper models instead of defaulting to frontier models, and precomputed context, which avoids an agent burning tokens assembling its own context from scratch.
“Claude is also very slow because the first part of assembling that context actually takes forever,” Jain said. Routing that request through Glean instead, he said, can cut token consumption by as much as half.
What Zillow and Glean’s approach means for enterprises
Across data, cost and permissions, the session offered a few practical takeaways for enterprises building agentic AI on their own systems.
Build the measurement baseline before the AI push, not after. Roberts said Zillow’s ability to credibly attribute a 40% increase in shipped code to AI adoption rests on a DORA metrics baseline the team put in place years earlier, not on the AI rollout itself.
Centralize context once instead of letting every team rebuild it. Jain’s core argument for Glean’s platform is that duplicated integration work across finance, legal and marketing teams is a hidden cost most enterprises haven’t accounted for.
Don’t assume permission inheritance is enough for regulated data. Even with a permissions-aware context platform in place, Zillow layered hard rules and a standing compliance check on top for its most sensitive categories, rather than trusting the architecture to handle it automatically.
Treat context as a cost lever, not just a capability. Model routing and precomputed context were the two mechanisms Jain pointed to for cutting AI spend, both aimed at reducing wasted token consumption rather than adding new capability.
“Models by themselves are not enough to bring automation with AI inside your enterprise,” Jain said. “You do have to connect it with your enterprise context.”
Credit: Source link
