In a groundbreaking development, Google has introduced a cutting-edge grammar correction feature in its search engine, powered by the innovative EdiT5 model. This revolutionary approach addresses the challenges posed by complex grammatical error correction (GEC), ensuring high precision and recall while delivering results with exceptional speed.
Traditionally, GEC has been approached as a translation problem, utilising autoregressive Transformer models. While effective, this method lacks efficiency as autoregressive decoding limits parallelisation. Recognising the need for a more streamlined process, the team behind EdiT5 reimagined GEC as a text editing problem. By employing the T5 Transformer encoder-decoder architecture, they significantly reduced decoding steps, minimising latency.
The EdiT5 model takes input with grammatical errors and employs an encoder to determine which tokens to keep or delete. The retained tokens form a draft output, which can be optionally reordered using a non-autoregressive pointer network. A decoder then inserts any missing tokens required to generate a grammatically correct output. Crucially, the decoder only operates on tokens absent in the draft, drastically reducing processing time compared to traditional translation-based GEC.
To further enhance decoding speed, the team streamlined the decoder to a single layer while augmenting the encoder’s size. This strategic adjustment effectively balances the workload and results in a significant reduction in latency. In practical terms, this means the EdiT5 model achieves remarkable results with an impressive mean latency of just 4.1 milliseconds.
Performance evaluations on the public BEA grammatical error correction benchmark demonstrate the superiority of EdiT5. A large EdiT5 model with 391 million parameters outperforms a T5 base model with 248 million parameters, yielding higher F0.5 scores, which measure correction accuracy. This improvement comes alongside a remarkable 9x speedup, showcasing the model’s exceptional efficiency.
Furthermore, the study underscores the pivotal role of model size in generating accurate grammatical corrections. Leveraging a technique known as hard distillation, the team combines the advantages of large language models (LLMs) with EdiT5’s low latency. By training a teacher LLM and utilising it to generate training data for the student EdiT5 model, they ensure a powerful synergy between accuracy and speed.
The development process also involved refining training data to ensure optimal performance. Training sets, comprised of ungrammatical source and grammatical target sentence pairs, were subjected to rigorous self-training and iterative refinement. This meticulous approach effectively eliminated unnecessary paraphrasing, artefacts, and grammatical errors, resulting in cleaner and more consistent training data.
In the final implementation, two EdiT5-based models were trained: a grammatical error correction model and a grammaticality classifier. When a user utilises the grammar check feature, the query undergoes correction by the model before being validated by the classifier. This two-step process ensures that only accurate corrections are presented to the user, mitigating the risk of erroneous or confusing suggestions.
With the introduction of the EdiT5-powered grammar check feature, Google Search sets a new standard for efficient and accurate grammar correction. Users can now confidently assess the grammaticality of their queries simply by including the “grammar check” phrase in their search. This milestone in natural language processing reaffirms Google’s commitment to enhancing user experience and providing reliable, precise search results.
Check out the Paper and Google blog. All Credit For This Research Goes To the Researchers on This Project. Also, don’t forget to join our 32k+ ML SubReddit, 40k+ Facebook Community, Discord Channel, and Email Newsletter, where we share the latest AI research news, cool AI projects, and more.
If you like our work, you will love our newsletter..
We are also on Telegram and WhatsApp.
Niharika is a Technical consulting intern at Marktechpost. She is a third year undergraduate, currently pursuing her B.Tech from Indian Institute of Technology(IIT), Kharagpur. She is a highly enthusiastic individual with a keen interest in Machine learning, Data science and AI and an avid reader of the latest developments in these fields.
Credit: Source link
