• bitcoinBitcoin(BTC)$64,749.00-0.28%
  • ethereumEthereum(ETH)$1,820.190.14%
  • tetherTether(USDT)$1.00-0.04%
  • binancecoinBNB(BNB)$592.63-0.19%
  • usd-coinUSDC(USDC)$1.00-0.01%
  • rippleXRP(XRP)$0.96-0.79%
  • solanaSolana(SOL)$71.76-0.13%
  • tronTRON(TRX)$0.3053220.55%
  • Figure HelocFigure Heloc(FIGR_HELOC)$1.040.00%
  • HyperliquidHyperliquid(HYPE)$69.154.25%
  • zcashZcash(ZEC)$906.732.04%
  • moneroMonero(XMR)$494.570.13%
  • dogecoinDogecoin(DOGE)$0.058972-0.07%
  • leo-tokenLEO Token(LEO)$9.271.19%
  • USDSUSDS(USDS)$1.00-0.03%
  • whitebitWhiteBIT Coin(WBT)$62.60-0.35%
  • chainlinkChainlink(LINK)$9.13-0.60%
  • cardanoCardano(ADA)$0.152897-0.05%
  • stellarStellar(XLM)$0.137655-1.12%
  • Ethena USDeEthena USDe(USDE)$1.00-0.03%
  • daiDai(DAI)$1.000.00%
  • RainRain(RAIN)$0.00600473.36%
  • bitcoin-cashBitcoin Cash(BCH)$208.30-0.37%
  • nearNEAR Protocol(NEAR)$3.170.45%
  • litecoinLitecoin(LTC)$44.74-0.27%
  • CantonCanton(CC)$0.082289-3.96%
  • avalanche-2Avalanche(AVAX)$7.11-1.25%
  • uniswapUniswap(UNI)$5.170.44%
  • tether-goldTether Gold(XAUT)$3,850.38-0.43%
  • the-open-networkGram (prev. Toncoin)(GRAM)$1.05-1.24%
  • USD1USD1(USD1)$1.00-0.02%
  • suiSui(SUI)$0.72-0.92%
  • Ripple USDRipple USD(RLUSD)$1.00-0.01%
  • Global DollarGlobal Dollar(USDG)$1.000.01%
  • hedera-hashgraphHedera(HBAR)$0.057155-2.92%
  • paypal-usdPayPal USD(PYUSD)$1.00-0.02%
  • Ondo US Dollar YieldOndo US Dollar Yield(USDY)$1.15-0.06%
  • shiba-inuShiba Inu(SHIB)$0.000004-1.42%
  • crypto-com-chainCronos(CRO)$0.0434540.42%
  • BittensorBittensor(TAO)$191.660.11%
  • quant-networkQuant(QNT)$153.37-2.53%
  • Circle USYCCircle USYC(USYC)$1.140.00%
  • MemeCoreMemeCore(M)$0.864.52%
  • Spiko Amundi Overnight Swap Fund (EUR)Spiko Amundi Overnight Swap Fund (EUR)(EURSAFO)$1.12-0.28%
  • okbOKB(OKB)$85.86-2.01%
  • pax-goldPAX Gold(PAXG)$3,852.05-0.49%
  • aaveAave(AAVE)$109.840.76%
  • pepePepe(PEPE)$0.0000042.65%
  • Falcon USDFalcon USD(USDF)$1.000.02%
  • United StablesUnited Stables(U)$1.000.02%
TradePoint.io
  • Main
  • AI & Technology
  • Stock Charts
  • Market & News
  • Business
  • Finance Tips
  • Trade Tube
  • Blog
  • Shop
No Result
View All Result
TradePoint.io
No Result
View All Result

CMU Researchers Propose XEUS: A Cross-lingual Encoder for Universal Speech trained in 4000+ Languages

July 4, 2024
in AI & Technology
Reading Time: 4 mins read
A A
CMU Researchers Propose XEUS: A Cross-lingual Encoder for Universal Speech trained in 4000+ Languages
ShareShareShareShareShare

Self-supervised learning (SSL) has expanded the reach of speech technologies to many languages by minimizing the need for labeled data. However, current models only support 100-150 of the world’s 7,000+ languages. This limitation is largely due to the scarcity of transcribed speech, as only about half of these languages have formal writing systems, and even fewer have the resources to generate the extensive annotated data needed for training. While SSL models can operate with unlabeled data, they typically cover a narrow range of languages. Projects like MMS have extended coverage to over 1,000 languages but need help with data noise and a lack of diverse recording conditions.

Researchers from Carnegie Mellon University, Shanghai Jiaotong University, and Toyota Technological Institute in Chicago have developed XEUS, a Cross-lingual Encoder for Universal Speech. XEUS is trained on over 1 million hours of data from 4,057 languages, significantly increasing the language coverage of SSL models. This includes a new corpus of 7,413 hours from 4,057 languages, which will be publicly released. XEUS incorporates a novel dereverberation objective for enhanced robustness. It outperforms state-of-the-art models in various benchmarks, including ML-SUPERB. To support further research, the researchers will release XEUS, its code, training configurations, checkpoints, and training logs.

YOU MAY ALSO LIKE

Sabre Announces Grok Bot Travel Connector and Proactive Agentic APIs – Unite.AI

Meta Is Testing Direct, A Standalone Messaging App For Instagram

SSL has advanced speech processing by enabling neural networks to learn from large amounts of unlabeled data, which can then be fine-tuned for various tasks. Multilingual SSL models can leverage cross-lingual transfer learning but only scale to cover a few languages. XEUS, however, scales to 4,057 languages, surpassing models like Meta’s MMS. XEUS includes a novel dereverberation objective during training to handle noisy and diverse speech. Unlike state-of-the-art models that often use closed datasets and lack transparency, XEUS is fully open, with publicly available data, training code, and extensive documentation, facilitating further research into large-scale multilingual SSL.

XEUS is pre-trained using a vast dataset of 1.081 million hours across 4,057 languages, compiled from 37 public speech datasets and additional sources like Global Recordings Network, WikiTongues, and Jesus Dramas. Unique data types enhance its robustness, such as accented speech and code-switching. XEUS incorporates new objectives, including dereverberation and noise reduction, during training. The model architecture is based on HuBERT but includes enhancements like E-Branchformer layers and a simplified loss function. The training on 64 NVIDIA A100 GPUs uses advanced augmentation techniques and spans significantly more data than previous models.

The XEUS model is evaluated across various downstream tasks to assess its multilingual and acoustic representation capabilities. It excels in multilingual speech tasks, outperforming state-of-the-art models like XLS-R, MMS, and w2v-BERT on benchmarks such as ML-SUPERB and FLEURS, especially in low-resource language settings. Additionally, XEUS demonstrates strong performance in task universality by matching or exceeding leading models in English-only tasks like emotion recognition and speaker diarization. In acoustic representation, XEUS surpasses models like WavLM and w2v-BERT in generating high-quality speech, which is evident through metrics like MOS and WER.

XEUS is a robust SSL speech encoder trained on over 1 million hours of data spanning 4,057 languages, demonstrating superior performance across a wide range of multilingual and low-resource tasks. XEUS’s dereverberation task enhances its robustness, and despite the limited data for many languages, it still provides valuable results. XEUS advances multilingual research by offering open access to its data and model. However, ethical considerations are crucial, especially in handling speech data from indigenous communities and preventing misuse, such as generating audio deepfakes. XEUS’s integration with accessible platforms aims to democratize speech model development.


Check out the Paper, Dataset, and Model. All credit for this research goes to the researchers of this project. Also, don’t forget to follow us on Twitter. 

Join our Telegram Channel and LinkedIn Group.

If you like our work, you will love our newsletter..

Don’t Forget to join our 46k+ ML SubReddit


Sana Hassan, a consulting intern at Marktechpost and dual-degree student at IIT Madras, is passionate about applying technology and AI to address real-world challenges. With a keen interest in solving practical problems, he brings a fresh perspective to the intersection of AI and real-life solutions.

🐝 Join the Fastest Growing AI Research Newsletter Read by Researchers from Google + NVIDIA + Meta + Stanford + MIT + Microsoft and many others…


Credit: Source link

ShareTweetSendSharePin

Related Posts

Sabre Announces Grok Bot Travel Connector and Proactive Agentic APIs – Unite.AI
AI & Technology

Sabre Announces Grok Bot Travel Connector and Proactive Agentic APIs – Unite.AI

November 17, 2026
Meta Is Testing Direct, A Standalone Messaging App For Instagram
AI & Technology

Meta Is Testing Direct, A Standalone Messaging App For Instagram

November 17, 2026
GoMirra Combines 18 AI Companions With Persistent Memory and Multiple Perspectives – Unite.AI
AI & Technology

GoMirra Combines 18 AI Companions With Persistent Memory and Multiple Perspectives – Unite.AI

November 16, 2026
United Airlines Is Experiencing A ‘Widespread System Outage’
AI & Technology

United Airlines Is Experiencing A ‘Widespread System Outage’

November 16, 2026
Next Post
Meet the Press full broadcast — June 30

Meet the Press full broadcast — June 30

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Search

No Result
View All Result
ALERT: Jensen Huang Returns As NVIDIA CEO?

ALERT: Jensen Huang Returns As NVIDIA CEO?

November 16, 2026
Claim the ,000 Trump Account Deposit – but Save Your Own Money in a 529

Claim the $1,000 Trump Account Deposit – but Save Your Own Money in a 529

November 14, 2026
Voters sound off about the economy ahead of critical midterm elections

Voters sound off about the economy ahead of critical midterm elections

November 16, 2026

About

Learn more

Our Services

Legal

Privacy Policy

Terms of Use

Bloggers

Learn more

Article Links

Contact

Advertise

Ask us anything

©2020- TradePoint.io - All rights reserved!

Tradepoint.io, being just a publishing and technology platform, is not a registered broker-dealer or investment adviser. So we do not provide investment advice. Rather, brokerage services are provided to clients of Tradepoint.io by independent SEC-registered broker-dealers and members of FINRA/SIPC. Every form of investing carries some risk and past performance is not a guarantee of future results. “Tradepoint.io“, “Instant Investing” and “My Trading Tools” are registered trademarks of Apperbuild, LLC.

This website is operated by Apperbuild, LLC. We have no link to any brokerage firm and we do not provide investment advice. Every information and resource we provide is solely for the education of our readers. © 2020 Apperbuild, LLC. All rights reserved.

No Result
View All Result
  • Main
  • AI & Technology
  • Stock Charts
  • Market & News
  • Business
  • Finance Tips
  • Trade Tube
  • Blog
  • Shop

© 2023 - TradePoint.io - All Rights Reserved!