Cerebras Systems known for building massive computer clusters that are used for all kinds of AI and scientific tasks. has yet again shattered records in the AI industry by unveiling its latest technological marvel, the Wafer-Scale Engine 3 (WSE-3), touted as the fastest AI chip the world has seen to date. With an astonishing 4 trillion transistors, this chip is designed to power the next generation of AI supercomputers, offering unprecedented performance levels.
The WSE-3, crafted using a cutting-edge 5nm process, stands as the backbone of the Cerebras CS-3 AI supercomputer. It boasts a groundbreaking 125 petaflops of peak AI performance, enabled by 900,000 AI-optimized compute cores. This development marks a significant leap forward, doubling the performance of its predecessor, the WSE-2, without increasing power consumption or cost.
Cerebras Systems’ ambition to revolutionize AI computing is evident in the WSE-3’s specifications. The chip features 44GB of on-chip SRAM, and supports external memory configurations ranging from 1.5TB to a colossal 1.2PB. This vast memory capacity enables the training of AI models up to 24 trillion parameters in size, facilitating the development of models ten times larger than those like GPT-4 and Gemini.
One of the most compelling aspects of the CS-3 is its scalability. The system can be clustered up to 2048 CS-3 units, achieving a staggering 256 exaFLOPs of computational power. This scalability is not just about raw power; it simplifies the AI training workflow, enhancing developer productivity by allowing large models to be trained without the need for complex partitioning or refactoring.
Cerebras’ commitment to advancing AI technology extends to its software framework, which now supports PyTorch 2.0 and the latest AI models and techniques. This includes native hardware acceleration for dynamic and unstructured sparsity, which can speed up training times by up to eight times.
The journey of Cerebras, as recounted by CEO Andrew Feldman, from the skepticism faced eight years ago to the launch of the WSE-3, embodies the company’s pioneering spirit and commitment to pushing the boundaries of AI technology.
“When we started on this journey eight years ago, everyone said wafer-scale processors were a pipe dream. We could not be more proud to be introducing the third-generation of our groundbreaking wafer-scale AI chip,” said Andrew Feldman, CEO and co-founder of Cerebras. “WSE-3 is the fastest AI chip in the world, purpose-built for the latest cutting-edge AI work, from mixture of experts to 24 trillion parameter models. We are thrilled to bring WSE-3 and CS-3 to market to help solve today’s biggest AI challenges.”
This innovation has not gone unnoticed, with a significant backlog of orders for the CS-3 from enterprises, government entities, and international clouds. The impact of Cerebras’ technology is further highlighted through strategic partnerships, such as with G42, which has led to the creation of some of the world’s largest AI supercomputers.
As Cerebras Systems continues to pave the way for future AI advancements, the launch of the WSE-3 serves as a testament to the incredible potential of wafer-scale engineering. This chip is not just a piece of technology; it’s a gateway to a future where the limits of AI are continually expanded, promising new possibilities for research, enterprise applications, and beyond.
Credit: Source link
