Baseten announced on November 6, 2026 that Serverless Loops is generally available, opening its managed SDK for supervised fine-tuning and reinforcement learning on open models to everyone. The service uses per-token training billing and provides a direct path from checkpoints to Baseten Dedicated Inference.
Loops is built around a division of labor: users supply the training signal, while Baseten operates the GPUs, kernels, model architectures and distributed-training backend. The Python SDK trains LoRA adapters with base-model weights frozen. Users choose the base model, LoRA rank, learning rate, loss function, data or environment and algorithm; Baseten runs the trainer, sampler, weight synchronization and supporting infrastructure.
A compatibility package preserves Tinker imports and core training and sampling calls for teams migrating existing scripts. The migration guide still changes authentication to a Baseten API key, returns checkpoints as paginated file URLs, requires inline image inputs and fixes LoRA module coverage by model family. Compatibility covers the training API while leaving several infrastructure-specific details for teams to adjust.
Serverless Models and Billing
The serverless catalog contains three models: Qwen/Qwen3.6-35B-A3B and zai-org/GLM-5.3-Flash for text and vision, and zai-org/GLM-5.3 for text only. Each supports a maximum sequence length of 262,144 tokens. GLM-5.3’s long-context configuration does not support cross_entropy or dpo losses. Dedicated mode covers the wider model catalog, bills for GPU time and adds controls for replicas, warm reuse and spot capacity.
Serverless runs use shared capacity without requiring GPU reservations and bill for tokens used in training and sampling. Time spent reading results, changing a reward function or sitting idle incurs no token charges. Default mode selection can fall back to dedicated capacity for an unsupported model, LoRA rank or sequence length, or when a run reuses another run’s deployments. Setting the scope to serverless makes an unsupported request fail instead.
Model enablement does not guarantee provisioning. Workspace-specific access, unavailable GPUs and organization trainer limits can still block a run, giving the pay-per-token model a set of practical capacity boundaries beyond the published catalog.
Checkpoints and Production Inference
The quickstart exposes four core operations: forward_backward for loss and gradient work, optim_step for adapter updates, sample for generation and save_state for resumable training. Publishing sampler weights creates a deployable checkpoint and a sampling client pinned to it. A separate save_state operation preserves model and optimizer state. The first sampling request can take several minutes while serverless capacity starts, a practical consideration for short experiments.
A sampler checkpoint deploys as the base model with the trained LoRA adapter loaded. Checkpoints outlive their runs and can be deployed through the CLI, REST API or dashboard using a Hugging Face access token, because the deployment downloads base weights. The resulting OpenAI-compatible endpoint bills for its GPU while live. Deleting that deployment stops the GPU charge without deleting the checkpoint; deactivating the Loops run does not remove the deployment.
For GA, Baseten improved handling of transient network errors and GPU failures, reduced startup overhead and accelerated weight transfers between trainers and samplers. Baseten says those changes support more concurrent customer workloads and that its production inference stack serves more than 15 billion requests daily. The production path includes autoscaling, deployment management and inference metrics, while users can download weights in Hugging Face Safetensors format.
Validation and Early Customer Workloads
Baseten says it validated the service with numerical tests and end-to-end reinforcement-learning experiments using τ-bench tasks across retail, airline and banking, followed by evaluation on held-out hard tasks. The τ-bench project describes a benchmark for whether agents can converse with users, call tools, retrieve knowledge and follow policy across enterprise domains, with outcomes checked against verifiable database results.
Baseten reports that NVIDIA post-trained Nemotron Super 3.5 VL on text and vision tasks, improving intent-classification accuracy from 76.8% to 94% and reaching 87% validation F1 on receipt extraction; the work also included reinforcement learning for visual object counting. Datadog is using multi-turn agentic RL to test training ideas for its SRE agent, and Ambience Healthcare post-trained models to correct clinical transcription errors and capture medical terminology and formatting accurately.
LangChain Labs is post-training models on agent traces for LangSmith Engine, including work that reads a GitHub repository to diagnose an issue and drafts the prompt or code change for a pull request. It can switch to a smaller open-weight model for classifying traces by failure mode and severity or mapping them to existing issues. LangChain’s workflow illustrates how application behavior becomes the training target, with model size and task specialization chosen around the job.
The customer accounts document use before the GA announcement and show the intended infrastructure loop: application data becomes training input, checkpoints are evaluated, and selected models move toward production. For infrastructure teams, the cost boundary is equally clear. Supported serverless runs avoid GPU reservations and idle-time token charges, while a live inference endpoint continues to bill for its GPU; the serverless catalog and workspace capacity determine where the serverless option applies.
Credit: Source link
