Empowering Time Series AI: How Salesforce is Leveraging Synthetic Data to Enhance Foundation Models

Time series analysis faces significant hurdles in data availability, quality, and diversity, critical factors in developing effective foundation models. Real-world datasets often fall short due to regulatory limitations, inherent biases, poor quality, and limited paired textual annotations, making it difficult to create robust, generalizable Time Series Foundation Models (TSFMs) and Large Language Model-based Time Series Models (TSLLMs). This scarcity impacts tasks such as forecasting, classification, anomaly detection, reasoning, and captioning, limiting the full potential of current advancements in artificial intelligence.

Salesforce AI Research has addressed these challenges by proposing a comprehensive approach to leveraging synthetic data for enhancing TSFMs and TSLLMs. Their recent study, “Empowering Time Series Analysis with Synthetic Data,” presents a novel strategy of using synthetic data to improve model training, evaluation, and fine-tuning, focusing on mitigating biases, increasing dataset diversity, and enriching contextual information. By developing innovative data-generation frameworks and incorporating synthetic datasets, Salesforce AI aims to advance the practical application of TSFMs and TSLLMs, especially in sensitive domains like healthcare and finance, where data sharing is heavily regulated.

The technical cornerstone of Salesforce AI Research’s methodology involves various synthetic data generation approaches, each addressing specific aspects of time series dynamics, such as trends, seasonal patterns, and noise characteristics. For instance, the ForecastPFN method combines linear-exponential trends and periodic seasonalities with Weibull-distributed noise, effectively simulating realistic yet diverse scenarios. Similarly, TimesFM integrates piecewise linear trends and autoregressive moving average (ARMA) models with periodic patterns. Another innovative technique, KernelSynth by Chronos, employs Gaussian Processes (GPs) combined with linear, periodic, and radial basis function (RBF) kernels to generate rich synthetic datasets. These methods enable a controlled yet varied synthetic data creation that helps in capturing a comprehensive range of realistic time series behaviors.

The Salesforce team’s findings highlight substantial benefits derived from synthetic data in multiple stages of model development. In pretraining, synthetic datasets provided clear performance enhancements, notably demonstrated in models like ForecastPFN, Mamba4Cast, and TimesFM. For example, ForecastPFN pretrained entirely on synthetic data showed significant improvements in zero-shot forecasting scenarios, while Chronos found optimal performance gains by mixing around 10% synthetic data with real-world datasets, beyond which additional synthetic data could potentially degrade performance due to less diverse representations. Additionally, synthetic data also played a crucial role in evaluation, allowing researchers to precisely assess the model’s capabilities, understanding internal representations, and identifying gaps in the learned patterns. Moment utilized synthetically generated sinusoidal waves to evaluate internal embeddings and model sensitivity to variations in time series characteristics, demonstrating its effectiveness in capturing subtle trends and frequencies.

The paper also addresses current limitations in synthetic data usage, identifying areas for future improvement. One critical gap is the absence of systematic integration methods for synthetic datasets, suggesting the need for structured frameworks to identify and fill missing real-world data patterns strategically. Another limitation noted is the dominance of statistical methods, prompting a call for exploring data-driven generative techniques, like diffusion models, to enhance realism. Salesforce researchers further emphasize untapped potential in leveraging synthetic data during fine-tuning phases to address specific domain gaps or model weaknesses more efficiently and adaptively.

In conclusion, Salesforce AI Research demonstrates that synthetic data offers a powerful toolset for overcoming data-related challenges in time series analysis. By systematically integrating high-quality synthetic datasets into various stages of model development, TSFMs and TSLLMs can achieve enhanced generalization, reduced biases, and improved performance across diverse analytical tasks. Despite existing limitations, such as ensuring realism and alignment, the proactive advancement and exploration of synthetic data generation methodologies indicate significant potential. Future research, as suggested by Salesforce, should focus on improving data realism, systematically addressing data gaps, and exploiting iterative, human-in-the-loop synthetic data generation processes. These advancements could dramatically expand the applicability and reliability of time series models, laying a solid foundation for future innovations in artificial intelligence.

Check out the Paper. All credit for this research goes to the researchers of this project. Also, feel free to follow us on Twitter and don’t forget to join our 85k+ ML SubReddit.