Replit brings open source AI developer tools to all users 

Developer tooling vendor Replit is out this week with a series of new efforts designed to help bring AI for all developers.

Replit’s cloud software development platform is widely used with the company claiming to have over 20 million users. Over the course of the last year, Replit has been incrementally building out generative AI capabilities for its users, with the GhostWriter AI code completion tool and a partnership with Google. To date, GhostWriter access has been limited to a subset of Replit users, but that’s now going to change.

As of Oct 9, Replit is directly integrating GhostWriter into its core platform and making the generative AI code completion tool available to all of its users, calling the effort “AI for all.” 

Alongside the GhostWriter integration, Replit also announced a new version of its own purpose-built open source generative AI large language model (LLM) for coding known as replit-code-v1.5-3b. 

Replit’s open source coding LLM is being positioned as a competitive alternative to the StarCoder LLM which is  jointly developed by ServiceNow and Hugging Face, as well as Meta’s Llama CodeLlama 7B.

“Replit’s mission has always been about access,” Amjad Masad, CEO of Replit said during a live streamed session at the AI Engineer Summit. “Our mission is to empower the next billion developers and so we really didn’t want to create this world where some people have access to GhostWriter and other people don’t have access to it.”

GhostWriter disappears into Replit as GenAI goes mainstream

In recent years, there has been an explosion of AI powered coding tools to help developers write code. 

Microsoft has its suite of GitHub Co-pilot services, Amazon has Codewhisper, and even Stack Overflow is in the game with it OverFlow AI effort. In Masad’s view, tools like GitHub Co-pilot are add ons to existing development tooling.

“We think that’s not the way forward,” he said. “We think that AI needs to be really infused in every programming interaction that you have and it needs to be part of the default experience in Replit and I’m sure other products in the future.”

With the new integration, Replit is dropping the name GhostWriter entirely, and is instead moving to to have AI as a core feature that is enabled for all its users. Masad said that Replit has people all over the world coding on all sorts of devices, including laptops and even mobile phones and now all those users can become AI enhanced developers.

“We think this is going to be the biggest deployment of AI enhanced coding in the world,” Masad said. “We’re going to be burning as much GPU as we’re burning CPU, so pray for us.”

Replit update open source code LLM

Replit’s generative AI capabilities are not wrappers on top of some other vendor’s LLM, but rather are based on open source technology the company has built.

“Our code completion feature in Replit is powered by our own bespoke large language model,” Michele Catasta, VP of AI at Replit explained during a live-streamed session at the AI Engineer Summit. “We trained on open source code, both published on GitHub. and also developed by the Replit user base.”

Back in May, Replit released its replit-code-v1-3b LLM, and is now out with its replit-code-v1.5-3b update which significantly expands the power and scope of the LLM.

Catasta noted that the model update was trained on 1 trillion tokens of code and supports 30 different programming languages. 

The ‘secret ingredient’ in the new LLM, according to Catasta, is all the work that Replit did working on the data. He emphasized that paying attention to data quality is critical and that’s what Replit has done.

The other key to enabling the model is the powerful hardware the LLM was trained on.

“We trained on 128 [Nvidia] H100-80G [GPUs], which are as rare as gold at this point,” Catasta said. “To our knowledge, this is the first model officially announced to be trained on the H100 that has been released as open source, so we’re very excited about it.”