Splunk unveils Splunk AI to ease security and observability through generative AI 

During Splunk’s .conf23 event, the company announced Splunk AI, a suite of AI-driven solutions designed to enhance its unified security and observability platform. According to the company, the latest development combines automation with human-in-the-loop experiences to empower organizations to improve their detection, investigation and response capabilities while maintaining control over AI implementation. 

The new Splunk AI Assistant employs generative AI to give users an interactive chat experience using natural language. Users can create Splunk Processing Language (SPL) queries through this interface, thereby expanding their understanding of the platform.

Through the AI Assistant, Splunk aims to optimize time-to-value and increase accessibility to SPL, democratizing an organization’s access to valuable data insights.

Splunk said that the AI will empower SecOps, ITOps and engineering teams to automate data mining, anomaly detection and risk assessment. so they can focus on more strategic tasks and reduce errors. 

“As a company, we have been deliberate in ensuring our Splunk AI innovations combine automation with human-in-the-loop experiences, so organizations can strengthen human decision-making with threat response by increasing speed and effectiveness, but not replace human decision-making,” Min Wang, CTO at Splunk, told VentureBeat. “Both our embedded and foundational AI offerings within Splunk AI provide recommendations on large, rich sets of information to enhance and accelerate human decision-making regarding detection, investigation and response.”

The model is integrated with domain-specific large language models (LLMs) and ML algorithms, leveraging security and observability data to boost productivity and cost efficiency. The company emphasized its commitment to openness and extensibility, as it enables organizations to integrate their AI models or third-party tools.

“What differentiates Splunk’s AI-powered offerings is they optimize domain-specific large language models and ML algorithms built on security and observability data,” Wang told VentureBeat. “These domain-specific insights will provide SecOps, ITOps and engineering teams with relevant data to automatically detect anomalies and then prioritize their attention to where it’s most needed based on intelligent risk assessment, minimizing repetitive processes and human error.”

Easing security and IT workloads through AI 

Splunk asserts that as tech infrastructure becomes more complex and distributed, and with ongoing talent shortages, organizations need tools that enable them to act swiftly and efficiently without exhausting their teams.

“With Splunk AI, we want to help make the jobs of SecOps, ITOps and engineering easier, so they can focus on more strategic work … [and] act faster and more accurately to ensure their systems remain resilient,” said Splunk’s Wang. 

Splunk’s new AI-powered capabilities aim to enhance alerting speed and accuracy, bolstering digital resilience. According to the company, its app for anomaly detection streamlines and automates the entire operational workflow for anomaly detection.

Meanwhile, IT Service Intelligence 4.17 service introduces outlier exclusion for adaptive thresholding, which identifies and excludes abnormal data points. In addition, “ML-assisted thresholding” generates dynamic thresholds based on historical data and patterns, resulting in more precise alerting.

“ML-assisted thresholding uses historical data and patterns to create dynamic thresholds with just one click. Thresholds that better mirror the expected workload on an hour-by-hour basis help ITOps and engineering teams reduce false positives and drive more accurate alerting on the health of an organization’s technology environment,” Wang explained. 

In another development, the company unveiled ML-powered foundational offerings that grant organizations access to comprehensive information. The Splunk Machine Learning Toolkit (MLTK) 5.4 now provides guided access to ML technology, enabling users of all skill levels to leverage forecasting and predictive analytics.

“MLTK can be deployed on top of [the] Splunk Enterprise or Cloud platform to extend the platform with techniques like an outlier and anomaly detection, predictive analytics, and clustering, to filter out noise and address common ML use cases,” said Wang. 

Wang said the latest MLTK release enables users to easily upload their pre-trained models to MLTK through a user-friendly interface.

Once the model is within Splunk, users can seamlessly apply it to their Splunk data without altering their existing workflows. This functionality expands the applicability of MLTK and ML-SPL to encompass models trained using methods other than MLTK.

Emphasizing data science for better detection and analysis

According to Wang, domain specificity is crucial for models. She emphasized the importance of tuning models specifically for their respective use cases and having experts in the field build them. While generic large language models (LLMs) can serve as a starting point, she said that the most effective models are those tailored to specific domains.

Wang highlighted that although generative AI is valuable for learning curves and generating new insights, deep learning tools may be better suited for embedding purpose-built complex anomaly detection algorithms into security offerings.

“As experts in security and observability, I believe we have the best domain-specific insights derived from real-world experience by our development team, go-to-market team, and customers,” she said. 

To facilitate this transition, Splunk has introduced the Splunk App for Data Science and Deep Learning (DSDL) 5.1. This extension of MLTK enhances the integration of advanced custom machine learning and deep learning systems with the Splunk ecosystem, thereby bolstering its capabilities.

“The DSDL extends MLTK with prebuilt Docker containers for additional machine learning libraries. Data scientists and machine learning or deep learning engineers can use DSDL to leverage GPU computing for compute-intense training tasks and flexibly deploy models on CPU or GPU-enabled containers,” explained Wang. “This offering is specific to our customers who store their data in Splunk environments and need tools to incorporate powerful ML algorithms trained on their data for their unique purposes.”

DSDL 5.1 also introduces two new AI assistants that will enable customers to use LLMs to build and train models specific to their domain. These assistants will focus specifically on text summarization and text classification applications.

Wang said AI/ML and analytics are crucial in enhancing anomaly detection and alerting accuracy. These technologies reduce false positives and customize thresholds based on unique customer data patterns, resulting in more effective alerting.

Along the same lines, the company’s new Splunk app for Anomaly Detection employs machine learning to automate the detection of anomalies in one’s environment. It also offers consistent health diagnostics.

“The app provides an end-to-end operationalization workflow so organizations can create and run consistent anomaly detection jobs, view SPL queries and create alerts. This leads to more accurate overall alerting,” said Wang.