Z.ai on September 17, 2026 published a technical account describing how it built a complete production-grade inference service for its GLM-5.3-Flash model from scratch on a cluster of more than 100,000 Chinese-made AI accelerators. According to the company, much of the work was carried out by an Infra Agent powered by GLM-5.3 rather than by infrastructure engineers alone, and all production inference for GLM-5.3-Flash runs on the system.
Z.ai said no one had previously operated a cluster of Chinese-made accelerators at this scale. The company cited relatively limited on-chip memory capacity and bandwidth, a new model architecture, a 1M-token context window, and multimodal requests, alongside an immature ecosystem in which kernel support was incomplete and engineers had to guess at behavior that should have been documented.
GLM-5.3-Flash launched on August 26, 2026 as the first natively multimodal model in the GLM-5 series, with 320 billion total parameters and 18 billion active parameters under a hybrid architecture combining sparse and linear attention. Before release, Z.ai tested the model anonymously as ox-alpha on OpenCode and OpenRouter, and the company said it became the most-used model on both platforms within a week of launch, processing more than 62 trillion tokens in six days.
The Dense Feedback Method
At the center of the account is a systems problem: end-to-end metrics can tell an agent that results got worse, but not why. A failed numerical accuracy test, a 30% increase in time-to-first-token, or a 20% drop in output throughput does not show which layer is responsible or what to test next. Z.ai’s answer, which it calls dense feedback, folds correctness tests, runtime logs, execution traces, runtime events, microbenchmarks, and end-to-end metrics into repeatable workflows that let the agent validate each hypothesis locally rather than wait for a full deployment and load test after every change.
The company defines three required properties for such feedback. It must be local, tied wherever possible to specific launch parameters, code changes, kernels, input conditions, threads, execution intervals, or code paths. It must be inexpensive and timely to obtain. And it must support objective verification through reference implementations and controlled experiments, because observed correlations by themselves do not establish a root cause.
In the launch loop the account describes, engineers defined objectives and system boundaries and reviewed critical changes involving numerical semantics, concurrency behavior, and production risk, while the agent handled analysis, hypotheses, and code changes. The stack they jointly optimized combined intra-node tensor parallelism for linear attention and the LM Head, ReplaySSM, W8A8 quantization, mixed-precision INT8/FP8/BF16 cache quantization, and Layer Split, under an Encode-Prefill-Decode disaggregated architecture.
Three Engineering Cases
The first case concerns numerical correctness. Validation comparing partitioned and unpartitioned kernel execution paths exposed an accuracy problem in the KDA kernel’s Context Parallelism path: the tl.dot operation defaulted to TF32 computation even when its inputs were FP32, so errors accumulated during state merging and compounded as context length increased. The fix explicitly set the input precision to tf32x3, which uses three TF32 Tensor Core operations to yield a higher-precision result.
According to the account, the fixes were merged upstream into Flash Linear Attention. The pull request, opened and merged on August 27, 2026, applies the tf32x3 affine chain in the state-update and transformation-merging kernels as an opt-in accuracy path, adds Context Parallelism tests, and falls back explicitly to ieee precision on platforms without tf32 support (AMD, NPUs, and NVIDIA GPUs below compute capability 8.0).
The second case concerns a KV Transfer concurrency bottleneck. According to the account, engineers set an acceptance criterion that, under the same workload, Prefill plus KV Transfer should run within 5% of the Prefill-only baseline. The agent found gaps exceeding 20% in some scenarios and traced them to DeepEP v1.2.1, in which neither the intranodedispatch nor the intranodecombine call explicitly released the Python GIL. As long as those calls held the lock, the Mooncake Transfer Python thread in the same process could not acquire the GIL in time, so scheduling and submission of transfer tasks slipped and overlap with computation shrank. The account notes that internode_dispatch in the same version already released the GIL, with a code comment stating the intent was to avoid blocking KV Transfer in other threads while the CPU waited. After the fix released the GIL during the relevant C++ execution intervals, the account reports, the gap fell below 1% under the same test conditions.
The third case concerns kernel performance. Z.ai had the agent distill techniques from handwritten kernels in projects including SGLang, Flash Linear Attention, and DeepGEMM into reusable optimization skeletons carrying applicability conditions, transformation methods, resource constraints, and validation evidence. On a representative KDA Decode kernel, the company reports that the agent’s division optimization cut execution time by 9.6%. After feedback identified computation as the primary bottleneck, the agent merged the kernel’s V-dimension tiles, which had repeated the same FP32 normalization and gating computations four times, into a single thread block with register-resident intermediate results and one warp-level reduction, producing what the company reports as a 1.71× speedup over the prior version.
Stated Results and Recursive Self-Improvement
Z.ai reports that GLM-5.3-Flash went from initial model adaptation to production readiness in less than two weeks, with end-to-end throughput ultimately tripling relative to the initial baseline. The company also said hardware utilization efficiency and per-token cost reached levels comparable to mainstream NVIDIA GPUs.
The company frames the effort as an early example of recursive self-improvement, noting that the model participated in optimizing the inference system on which it runs. At the same time, Z.ai states it has not yet reached recursive self-improvement, and that choosing objectives, setting boundaries, and assessing risk remain human responsibilities it believes humans should continue to hold.
Credit: Source link
