Google reveals BigQuery innovations to transform working with data

Google is pushing the bar on how teams work with their data.

Today, at its annual Cloud Next conference, the internet giant announced major improvements for BigQuery — its fully managed, serverless data warehouse, including a unified experience aimed at interconnecting data and workloads. The company also shared how it plans to bring AI to the data stored in the platform, and how it plans to leverage its generative AI collaborator to boost the productivity of teams looking to consume insights from data.

“These innovations will help organizations harness the potential of data and AI to realize business value — from personalizing customer experiences, improving supply chain efficiency, and helping reduce operating costs, to helping drive incremental revenue,” Gerrit Kazmaier, VP/GM, data and analytics at Google, wrote in a blog post.

However, it must be noted that most of these capabilities are still being previewed and not generally available to customers.

Unified experience with BigQuery Studio

Within BigQuery, which allows users to perform scalable analysis over petabytes of data, Google is adding a unified interface called BigQuery Studio. This offering will provide users with a single environment for data engineering, analytics and predictive analysis.

Until now, data teams had to work with different tools for different tasks, from managing data warehouses and data lakes to governance and machine learning. Handling these tools took a lot of time and slowed down productivity. With BigQuery Studio, Google is enabling these teams to work with all of these tools in one place, to quickly discover, prepare, analyze their datasets and run machine learning workloads on them.

“BigQuery Studio provides data teams with a single interface for your data analytics in Google Cloud, including editing of SQL, Python, Spark and other languages, to easily run analytics at petabyte scale without any additional infrastructure management overhead. This means a data worker doesn’t have to switch from one tool to another; it’s all in one place making their lives easier and getting to results faster,” a company spokesperson told VentureBeat.

The offering is available in preview right now and is already being tested by multiple enterprises including Shopify. Kazmaier also said Google is adding enhanced support for open-source formats like Hudi and Delta Lake within BigLake; performance acceleration for Apache Iceberg; and cross-cloud materialized views and cross-cloud joins in BigQuery Omni to analyze and train on data without moving it.

Even more for data teams

Along with BigQuery Studio, Google is providing access to Vertex AI foundation models, including PaLM 2, directly from BigQuery. This will allow data teams using BigQueryML (to create and run ML models on their datasets) to scale SQL statements against large language models (LLMs) and gain more insights, quickly and easily. The company also said it is adding new model inference capabilities and vector embeddings in BigQuery to help teams run LLMs at scale on unstructured datasets.

“Using new model inference in BigQuery, customers can run model inferences across formats like TensorFlow, ONNX and XGBoost. In addition, new capabilities for real-time inference can identify patterns and automatically generate alerts,” Kazmaier noted.

Finally, the company said it is integrating its always-on generative AI-powered collaborator, Duet AI, into BigQuery, Looker and Dataplex. This will bring natural language interaction and automatic recommendations to these tools, boosting the productivity of teams and opening access to more users.

This integration is also in preview with no word on general availability yet.

Google Cloud Next runs from August 29 to 31, 2023.