• bitcoinBitcoin(BTC)$64,864.000.31%
  • ethereumEthereum(ETH)$1,817.16-0.71%
  • tetherTether(USDT)$1.00-0.02%
  • binancecoinBNB(BNB)$593.500.11%
  • usd-coinUSDC(USDC)$1.00-0.01%
  • rippleXRP(XRP)$1.004.43%
  • solanaSolana(SOL)$75.183.18%
  • tronTRON(TRX)$0.3088441.09%
  • Figure HelocFigure Heloc(FIGR_HELOC)$1.01-3.07%
  • HyperliquidHyperliquid(HYPE)$69.07-0.40%
  • zcashZcash(ZEC)$955.003.30%
  • moneroMonero(XMR)$505.170.80%
  • dogecoinDogecoin(DOGE)$0.0597981.22%
  • leo-tokenLEO Token(LEO)$9.350.56%
  • USDSUSDS(USDS)$1.000.00%
  • whitebitWhiteBIT Coin(WBT)$62.620.05%
  • chainlinkChainlink(LINK)$9.291.34%
  • cardanoCardano(ADA)$0.1548650.90%
  • stellarStellar(XLM)$0.1424063.61%
  • Ethena USDeEthena USDe(USDE)$1.00-0.02%
  • nearNEAR Protocol(NEAR)$3.383.68%
  • daiDai(DAI)$1.000.00%
  • bitcoin-cashBitcoin Cash(BCH)$208.820.05%
  • litecoinLitecoin(LTC)$44.840.26%
  • CantonCanton(CC)$0.0847274.11%
  • avalanche-2Avalanche(AVAX)$7.200.09%
  • uniswapUniswap(UNI)$5.25-1.14%
  • RainRain(RAIN)$0.004443-19.54%
  • the-open-networkGram (prev. Toncoin)(GRAM)$1.084.18%
  • suiSui(SUI)$0.741.60%
  • tether-goldTether Gold(XAUT)$3,763.45-2.40%
  • USD1USD1(USD1)$1.000.01%
  • Ripple USDRipple USD(RLUSD)$1.00-0.01%
  • Global DollarGlobal Dollar(USDG)$1.000.00%
  • hedera-hashgraphHedera(HBAR)$0.0575691.19%
  • paypal-usdPayPal USD(PYUSD)$1.00-0.01%
  • Ondo US Dollar YieldOndo US Dollar Yield(USDY)$1.15-0.04%
  • BittensorBittensor(TAO)$197.342.66%
  • crypto-com-chainCronos(CRO)$0.043998-0.53%
  • shiba-inuShiba Inu(SHIB)$0.0000040.27%
  • quant-networkQuant(QNT)$154.370.88%
  • MemeCoreMemeCore(M)$0.89-2.51%
  • Circle USYCCircle USYC(USYC)$1.140.01%
  • Spiko Amundi Overnight Swap Fund (EUR)Spiko Amundi Overnight Swap Fund (EUR)(EURSAFO)$1.12-0.24%
  • okbOKB(OKB)$86.91-0.73%
  • aaveAave(AAVE)$111.19-1.72%
  • pax-goldPAX Gold(PAXG)$3,765.33-2.44%
  • Falcon USDFalcon USD(USDF)$1.000.06%
  • United StablesUnited Stables(U)$1.00-0.10%
  • BlackRock USD Institutional Digital Liquidity FundBlackRock USD Institutional Digital Liquidity Fund(BUIDL)$1.000.00%
TradePoint.io
  • Main
  • AI & Technology
  • Stock Charts
  • Market & News
  • Business
  • Finance Tips
  • Trade Tube
  • Blog
  • Shop
No Result
View All Result
TradePoint.io
No Result
View All Result

Meet PolyLM (Polyglot Large Language Model): An Open Source Multilingual LLM trained on 640B Tokens, Available In Two Model Sizes 1.7B and 13B

July 17, 2023
in AI & Technology
Reading Time: 4 mins read
A A
Meet PolyLM (Polyglot Large Language Model): An Open Source Multilingual LLM trained on 640B Tokens, Available In Two Model Sizes 1.7B and 13B
ShareShareShareShareShare

With the recent introduction of Large Language Models (LLMs), its versatility and capabilities have drawn everyone’s interest in the Artificial Intelligence sector. These models have been trained on massive amounts of data and possess some brilliant human-imitating abilities in understanding, reasoning, and generating text based on natural language instructions. Having good performance in zero-shot and few-shot tasks, these models can handle unforeseen challenges based on instructions given in natural language by being fine-tuned on various sets of tasks.  

Current LLMs and their development focus on English and resource-rich languages. Most of the existing LLMs have been specifically designed and trained for the English language, resulting in a predominant bias towards English in the research and development of these models. To address this limitation, a team of researchers from DAMO Academy and Alibaba Group have proposed a multilingual LLM called POLYLM (Polyglot Large Language Model). Unlike existing multilingual LLMs that lack a 13B model, the team has released POLYLM-13B and POLYLM-1.7B to facilitate usage.

POLYLM has been built using a massive dataset of 640B tokens from publically accessible sources, including Wikipedia, mC4, and CC-100. The team has also suggested a curricular learning technique to address the issue of insufficient data for low-resource languages. This method involves gradually increasing the ratio of high-quality, low-resource languages during training while initially focusing more on English. Focus has been made on transferring general knowledge from English to other languages.

[Sponsored] 🔥 Build your personal brand with Taplio  🚀 The 1st all-in-one AI-powered tool to grow on LinkedIn. Create better LinkedIn content 10x faster, schedule, analyze your stats & engage. Try it for free!

The team has also developed MULTIALPACA, a multilingual instruction dataset, for the supervised fine-tuning (SFT) phase. Existing multilingual SFT datasets are either obtained through manual annotation, which is time-consuming and expensive, or through machine translation, which may result in translation errors and lacks cultural nuances. This multilingual self-instruct approach automatically provides high-quality multilingual instruction data to overcome these restrictions and makes use of English seeds, translations into many languages, instruction production, and filtering systems.

For evaluation and to assess the multilingual capabilities of LLMs, the team has developed a benchmark derived from existing multilingual tasks, including question answering, language understanding, text generation, and cross-lingual machine translation. The benchmark has been developed with meticulous prompting and covers ten tasks across 15 languages. The team has demonstrated through extensive experiments that their pretrained model outperforms open-source models of comparable size in non-English languages. The proposed curriculum training strategy improves multilingual performance while maintaining English proficiency. The use of multilingual instruction data also significantly enhances POLYLM’s ability to tackle multilingual zero-shot tasks.

The team has summarized the contributions as follows.

  1. A proficient 13B scale model has been performed that performs well in major non-English languages like Spanish, Russian, Arabic, Japanese, Korean, Thai, Indonesian, and Chinese. This model complements existing open-source models that either lack proficiency in these languages or have smaller versions without the same capabilities.
  1. An advanced curriculum learning approach has been proposed that facilitates the transfer of general knowledge, mainly acquired in English, to diverse non-English languages and specific natural language processing tasks, such as machine translation.
  1. A dataset called MULTIALPACA has been proposed that complements existing instruction datasets, allowing LLMs to better follow multilingual instructions, particularly from non-native English speakers.

Check out the Paper and Project. All Credit For This Research Goes To the Researchers on This Project. Also, don’t forget to join our 26k+ ML SubReddit, Discord Channel, and Email Newsletter, where we share the latest AI research news, cool AI projects, and more.

🚀 Check Out 800+ AI Tools in AI Tools Club


YOU MAY ALSO LIKE

Recast Brings AI Troubleshooting to Right Click Tools With MCP Server – Unite.AI

Nexalus and RENDA Launch Nova8 Liquid-Cooled AI Server for Data Center Retrofits – Unite.AI

Tanya Malhotra is a final year undergrad from the University of Petroleum & Energy Studies, Dehradun, pursuing BTech in Computer Science Engineering with a specialization in Artificial Intelligence and Machine Learning.
She is a Data Science enthusiast with good analytical and critical thinking, along with an ardent interest in acquiring new skills, leading groups, and managing work in an organized manner.


🔥 StoryBird.ai just dropped some amazing features. Generate an illustrated story from a prompt. Check it out here. (Sponsored)

Credit: Source link

ShareTweetSendSharePin

Related Posts

Recast Brings AI Troubleshooting to Right Click Tools With MCP Server – Unite.AI
AI & Technology

Recast Brings AI Troubleshooting to Right Click Tools With MCP Server – Unite.AI

November 18, 2026
Nexalus and RENDA Launch Nova8 Liquid-Cooled AI Server for Data Center Retrofits – Unite.AI
AI & Technology

Nexalus and RENDA Launch Nova8 Liquid-Cooled AI Server for Data Center Retrofits – Unite.AI

November 18, 2026
Flowtica Launches Verso, an E-Paper AI Recorder for Meeting Notes – Unite.AI
AI & Technology

Flowtica Launches Verso, an E-Paper AI Recorder for Meeting Notes – Unite.AI

November 18, 2026
Here’s What You Can Do If Your PC Won’t Read An External Hard Drive
AI & Technology

Here’s What You Can Do If Your PC Won’t Read An External Hard Drive

November 18, 2026
Next Post
Meta is DOMINATING AI! We haven’t seen ANYTHING Like this!

Meta is DOMINATING AI! We haven’t seen ANYTHING Like this!

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Search

No Result
View All Result
British adventurer Karl Bushby completes 28-year walk around the world

British adventurer Karl Bushby completes 28-year walk around the world

November 11, 2026
Two planes collide on taxiway at Miami airport one day after national grounding – The Guardian

Two planes collide on taxiway at Miami airport one day after national grounding – The Guardian

November 15, 2026
How Adobe Is Helping Disney with Custom AI Tools

How Adobe Is Helping Disney with Custom AI Tools

November 15, 2026

About

Learn more

Our Services

Legal

Privacy Policy

Terms of Use

Bloggers

Learn more

Article Links

Contact

Advertise

Ask us anything

©2020- TradePoint.io - All rights reserved!

Tradepoint.io, being just a publishing and technology platform, is not a registered broker-dealer or investment adviser. So we do not provide investment advice. Rather, brokerage services are provided to clients of Tradepoint.io by independent SEC-registered broker-dealers and members of FINRA/SIPC. Every form of investing carries some risk and past performance is not a guarantee of future results. “Tradepoint.io“, “Instant Investing” and “My Trading Tools” are registered trademarks of Apperbuild, LLC.

This website is operated by Apperbuild, LLC. We have no link to any brokerage firm and we do not provide investment advice. Every information and resource we provide is solely for the education of our readers. © 2020 Apperbuild, LLC. All rights reserved.

No Result
View All Result
  • Main
  • AI & Technology
  • Stock Charts
  • Market & News
  • Business
  • Finance Tips
  • Trade Tube
  • Blog
  • Shop

© 2023 - TradePoint.io - All Rights Reserved!