Early adopters of data streaming see up to 10x returns, survey finds

As data volumes continue to surge, organizations no longer have the liberty to move at their own pace. They need to capture streams of data as soon as they are generated to become more responsive and act in real-time. But, the question is, how effective can data streaming really be from a business standpoint? 

According to a new survey of 2,250 global IT and engineering leaders, companies in the early stages of data streaming adoption are already seeing or to see expecting 2X to 5X returns on their investments. This reaps benefits of increased IT and business efficiency, enhanced customer experience, faster operational decision-making and increased product or service profitability.

The poll, commissioned by data streaming company Confluent and conducted by Freeform Dynamics and Radma Research between February and March 2023, not only highlights what companies can achieve in today’s digital-first era with data streaming, but also brings to light certain roadblocks that can affect the adoption and execution of such projects. This holds importance as data streaming goes mainstream across sectors.

Correlation between data streaming maturity and positive outcomes

Confluent classified respondents into five groups based on the maturity of their data streaming stack. The so-called level 1 respondents (7%) had data streaming experiments in pre-production, level 2 folks (20%) had projects deployed for non-critical applications, level 3 (63%) had deployment for a few critical systems, while levels 4 and 5 (9%) had several deployments in production for critical systems with streams managed as a product.

While most of the respondents were found to be in the first three stages, the survey results showed that even as beginners they are driving high ROI from data streaming. Specifically, 64% of organizations in level 2 and 78% in level 3 said they are achieving or anticipating a 2X to 5X return. 

What’s even more interesting is the fact that a small percentage of respondents from both these groups (3% from level 2 and 8% from level 3) also claimed to be getting 10X or more ROI. This is important as the general trend suggests that ROI grows as adoption and breadth of activity become more widespread and integrated. Even in the survey, 63% of those that qualified as level 4 orgs said they are achieving or anticipating 5X to 10X ROI.

Driving efficiencies, support systems

Along with high returns, the survey also found that data streaming is driving efficiencies across key business functions. As many as 76% of IT leaders said their organizations are already seeing significant or emerging benefits when it comes to increased business efficiency and IT efficiency; 74% said they are witnessing enhanced customer experience; and 73% cited faster decision-making and increased product or service profitability. 

Notably, a large number of the respondents (72%) also said they are using data streaming to power business-critical applications. The specific use cases detailed in the report include improving cybersecurity, monitoring IT infrastructure, improving customer experience, automating internal processes and helping with business decisions.

Still, roadblocks remain

Just as data streaming becomes more pervasive, teams are also running into certain adoption-linked challenges — much like any other growing technology.

In the survey, 74% of IT leaders said fragmented projects, uncoordinated teams and budgets can be a challenge or a major hurdle, while 72% cited a lack of relevant skills, inconsistent use of integration methods and standards and legacy systems as the problem area.

To address these gaps, Andrew Sellers, staff technologist at Confluent’s CTO’s office, suggested establishing a common operating model with data governance top of mind. 

“This will empower individuals to produce data and support them with infrastructure that ensures that data is easy to discover and highly trustworthy for others to consume,” he said. “As a result, developers can focus exclusively on building differentiated applications rather than wrestling with data infrastructure or relying on a central data engineering team for bespoke data transformations. Such an organization creates incredible business value in an efficient way.”

Other known players in the real-time streaming data and analytics market are AWS Kinesis, Cloudera DataFlow, Hitachi Vantara Lumada Edge Intelligence, IBM Streaming Analytics, Informatica Ultra Messaging, Kinetica Active Analytics, Microsoft Azure Stream Analytics and Oracle Event Processing.