GPU memory is the most expensive resource in production AI, and it’s also the one running out fastest.
Long context windows and multi-turn conversations force AI models to repeatedly recompute information they’ve already processed, consuming GPU memory and compute that could otherwise serve additional users or generate new responses.
Instead of treating GPU memory as the limiting resource, why not extend it with much cheaper storage technologies?
Weka, for one, believes that cheap flash storage can close that gap. The company’s NeuralMesh 6 software platform, launching alongside its first self-designed hardware line, Wekapod 3, extends what Weka calls Augmented Memory Grid, an approach that aggregates NAND flash to behave like GPU memory at a fraction of the cost.
This is an active and increasingly crowded category. Dell, NetApp, Pure Storage and VAST have all repositioned toward AI infrastructure over the past two years and Weka is one of several vendors arguing it’s built for this specific moment rather than adapting to it.
“What we’re seeing now with customers is they’re chasing availability of compute, and once they get new allocation from anyone, they want to be able to grab it and start running right away,” Weka co-founder and CEO Liran Zvibel, told VentureBeat.
The potential payoff is straightforward: better utilization of existing GPU investments, lower inference costs and faster deployment of new AI workloads without waiting months for additional GPU capacity.
The technology is most relevant for organizations already operating AI at scale or expecting rapid growth in usage, particularly enterprises building internal copilots, customer service agents, software engineering assistants or retrieval systems with long context windows. Smaller deployments may see less immediate benefit than organizations where GPU utilization has already become a limiting factor.
Inside Weka’s NeuralMesh 6
NeuralMesh 6 adds four capabilities aimed directly at a functionality gap Zvibel says has been costing Weka deals in competitive evaluations.
Composable and virtual multi-tenancy. Composable clusters give anchor tenants full hardware-level isolation, dedicated CPU, memory, and storage. Virtual multi-tenancy runs through Weka’s RDMA fabric, delivering network-level isolation that scales past 1,000 tenants per cluster, with provisioning in under 30 minutes. Combined, a single cluster running 50 composable clusters can support up to 50,000 tenants.
Unified file and object storage. Most storage systems keep two separate paths: a file-based path (the standard way servers and applications read and write files, used heavily in training and fine-tuning pipelines) and an object-based path (S3, the format inference and cloud-native tools typically expect). Normally a gateway translates between the two, meaning the data effectively exists twice. Weka’s claim is that the same physical data on disk is directly readable through either path at once, no translation layer, no second copy. Zvibel is targeting non-AWS GPU clouds specifically, naming Lambda, Nebius, G42, and CoreWeave, with what he described as roughly two orders of magnitude higher performance than conventional S3 and a capacity-based pricing model instead of per-API charges.
Metadata-first replication. Destination environments become browsable before a full data copy arrives, with data hydrating only when accessed.
“They had to wait for all of that to make it to the other side, and this takes days or weeks, in extreme cases a month,” Zvibel said. “We now allow our customers to grab some allocation of new GPUs and get up and running within an hour.”
AlloyFlash and Always-On data reduction. TLC and QLC are two types of NAND flash memory. TLC is faster and more durable but costs more per terabyte, while QLC is cheaper and holds more data per chip but is slower. AlloyFlash mixes both within a single cluster, automatically routing latency-sensitive work to TLC while running bulk-capacity workloads on QLC, cutting cost per terabyte without a performance penalty on the work that needs speed. Data reduction now runs by default rather than as an option.
Solving AI’s context problem
Multi-tenancy and object storage solve how enterprises and neo clouds operate the platform day to day. A harder problem sits underneath: as context windows and multi-turn interactions grow, so does the GPU compute wasted recalculating work a model has already done. Augmented Memory Grid, a NeuralMesh 6 feature built specifically for this, is Weka’s answer.
Every prompt triggers two stages. Prefill calculates attention, the core mechanism behind how large language models process input, and it’s computationally expensive. Decode converts that calculation into output and is comparatively lightweight.
The cost shows up hardest in multi-turn sessions like chat or coding, where each new turn re-triggers prefill for everything that came before it, unless that work has been cached.
“If you have 10 turns, you may overcalculate 100 times because you’re redoing all of them. If you have 20, you’ll overcalculate 400 times,” Zvibel said. “You can put two orders of magnitude more NAND than you could afford in shared memory, and we can cache 100% of the pre-calculated tokens, so you never need to redo it.”
Where Weka sits competitively
Storage vendors have spent the past year and a half repositioning around AI, and separating genuine capability from repositioned messaging is now a real evaluation problem for buyers.
“The storage world is shifting its focus from serving bits to enterprise workloads to managing data at the speed of AI. We’ve seen that most clearly over the past 18 months from Dell, NetApp, and Pure,” Steve McDowell, chief analyst at NAND Research, told VentureBeat. “The interesting thing is that companies like Weka, and VAST, are the true AI-native data companies, solving these problems since day one.”
McDowell singled out Augmented Memory Grid as Weka’s clearest technical lead.
“Weka continues to have the most technically capable KV cache implementation on the market with its Augmented Memory Grid,” he said. ” They were early with this technology, and continue to innovate. This is critical for AI inference, as it enables a level of GPU efficiency that, without question, saves money on GPUs and memory. That’s key for today’s memory and GPU constrained market.”
He also flagged Weka’s contractual guarantee on its data reduction claims as underappreciated.
“One flying a little under the radar: Weka is putting its money where its mouth is with its contractual guarantees for its data reduction promises,” he said.
McDowell’s advice to buyers evaluating competing claims from Weka, VAST, Pure and NetApp alike was pointed suggesting that enterprise buyers should look hard at what vendors are promising versus what they’re actually delivering.
“A smart buyer will look at how competing vendors are solving real-world problems today,” McDowell said. ” They do this by talking to organizations running similar workloads at similar scale. If a vendor can’t point to that, then it should be a warning sign.”
Credit: Source link
