• bitcoinBitcoin(BTC)$64,918.000.90%
  • ethereumEthereum(ETH)$1,821.360.98%
  • tetherTether(USDT)$1.000.00%
  • binancecoinBNB(BNB)$591.560.60%
  • usd-coinUSDC(USDC)$1.00-0.01%
  • rippleXRP(XRP)$0.972.12%
  • solanaSolana(SOL)$73.393.38%
  • tronTRON(TRX)$0.3062570.64%
  • Figure HelocFigure Heloc(FIGR_HELOC)$1.040.00%
  • HyperliquidHyperliquid(HYPE)$69.345.76%
  • zcashZcash(ZEC)$945.289.33%
  • moneroMonero(XMR)$511.512.13%
  • dogecoinDogecoin(DOGE)$0.0592741.48%
  • leo-tokenLEO Token(LEO)$9.280.66%
  • USDSUSDS(USDS)$1.000.00%
  • whitebitWhiteBIT Coin(WBT)$62.710.61%
  • chainlinkChainlink(LINK)$9.302.02%
  • cardanoCardano(ADA)$0.1543181.73%
  • stellarStellar(XLM)$0.1401192.50%
  • Ethena USDeEthena USDe(USDE)$1.000.01%
  • daiDai(DAI)$1.000.01%
  • bitcoin-cashBitcoin Cash(BCH)$212.483.08%
  • nearNEAR Protocol(NEAR)$3.243.11%
  • RainRain(RAIN)$0.005421-0.06%
  • litecoinLitecoin(LTC)$45.001.61%
  • CantonCanton(CC)$0.082476-1.40%
  • uniswapUniswap(UNI)$5.386.32%
  • avalanche-2Avalanche(AVAX)$7.231.73%
  • tether-goldTether Gold(XAUT)$3,832.68-0.59%
  • suiSui(SUI)$0.732.78%
  • USD1USD1(USD1)$1.000.00%
  • the-open-networkGram (prev. Toncoin)(GRAM)$1.03-1.25%
  • Ripple USDRipple USD(RLUSD)$1.000.00%
  • Global DollarGlobal Dollar(USDG)$1.00-0.01%
  • hedera-hashgraphHedera(HBAR)$0.0578230.14%
  • Ondo US Dollar YieldOndo US Dollar Yield(USDY)$1.15-0.03%
  • paypal-usdPayPal USD(PYUSD)$1.000.00%
  • crypto-com-chainCronos(CRO)$0.0445134.05%
  • BittensorBittensor(TAO)$193.803.55%
  • shiba-inuShiba Inu(SHIB)$0.000004-0.18%
  • quant-networkQuant(QNT)$155.141.17%
  • MemeCoreMemeCore(M)$0.908.82%
  • Circle USYCCircle USYC(USYC)$1.140.00%
  • Spiko Amundi Overnight Swap Fund (EUR)Spiko Amundi Overnight Swap Fund (EUR)(EURSAFO)$1.12-0.33%
  • okbOKB(OKB)$86.823.50%
  • aaveAave(AAVE)$112.894.73%
  • pax-goldPAX Gold(PAXG)$3,835.69-0.55%
  • Falcon USDFalcon USD(USDF)$1.00-0.02%
  • United StablesUnited Stables(U)$1.000.00%
  • pepePepe(PEPE)$0.0000040.94%
TradePoint.io
  • Main
  • AI & Technology
  • Stock Charts
  • Market & News
  • Business
  • Finance Tips
  • Trade Tube
  • Blog
  • Shop
No Result
View All Result
TradePoint.io
No Result
View All Result

This AI Paper from MIT Explores the Scaling of Deep Learning Models for Chemistry Research

November 17, 2023
in AI & Technology
Reading Time: 4 mins read
A A
This AI Paper from MIT Explores the Scaling of Deep Learning Models for Chemistry Research
ShareShareShareShareShare

Researchers from MIT investigated the scaling behavior of large chemical language models, focusing on both generative pre-trained transformers (GPT) for chemistry (ChemGPT) and graph neural network force fields (GNNs). They introduce the concept of neural scaling, where the performance of models is characterized by empirical scaling laws, particularly in terms of loss scaling as a power law concerning the number of model parameters, dataset size, or compute resources. The study delves into the challenges and opportunities associated with scaling large chemical models, aiming to provide insights into the optimal allocation of resources for improving pre-training loss.

For chemical language modeling, the researchers design ChemGPT, a GPT-3-style model based on GPT-Neo, with a tokenizer for self-referencing embedded strings (SELFIES) representations of molecules. The model is pre-trained on molecules from PubChem, and the study explores the impact of dataset and model size on pre-training loss.

In addition to language models, the paper addresses graph neural network force fields (GNNs) for tasks requiring molecular geometry and three-dimensional structure. Four types of GNNs are considered, ranging from models with internal layers manipulating only E(3) invariant quantities to those using E(3) equivariant quantities with increasing physics-informed model architectures. The authors evaluate the capacity of these GNNs, defined in terms of depth and width, during neural-scaling experiments.

To efficiently handle hyperparameter optimization (HPO) for deep chemical models, the paper introduces a technique called Training Performance Estimation (TPE), adapting it from a method used in computer vision architectures. TPE utilizes training speed to enable performance estimation across different domains and model/dataset sizes. The paper details the experimental settings, including the use of NVIDIA Volta V100 GPUs, PyTorch, and distributed data-parallel acceleration for model implementation and training.

Overall, the study provides a comprehensive exploration of neural scaling in the context of large chemical language models, considering both generative pre-trained transformers and graph neural network force fields, and introduces an efficient method for hyperparameter optimization. The experimental results and insights contribute to understanding the resource efficiency of different model architectures in scientific deep learning applications.


Check out the Paper. All credit for this research goes to the researchers of this project. Also, don’t forget to join our 33k+ ML SubReddit, 41k+ Facebook Community, Discord Channel, and Email Newsletter, where we share the latest AI research news, cool AI projects, and more.

If you like our work, you will love our newsletter..

We are also on Telegram and WhatsApp.


YOU MAY ALSO LIKE

The Smartest AI Decision Might Be Not Sending the Message – Unite.AI

Sarvam Launches Sarvam Token Factory: Bringing India-Hosted Inference to 8 Open Models

Pragati Jhunjhunwala is a consulting intern at MarktechPost. She is currently pursuing her B.Tech from the Indian Institute of Technology(IIT), Kharagpur. She is a tech enthusiast and has a keen interest in the scope of software and data science applications. She is always reading about the developments in different field of AI and ML.


🔥 Join The AI Startup Newsletter To Learn About Latest AI Startups

Credit: Source link

ShareTweetSendSharePin

Related Posts

The Smartest AI Decision Might Be Not Sending the Message – Unite.AI
AI & Technology

The Smartest AI Decision Might Be Not Sending the Message – Unite.AI

November 17, 2026
Sarvam Launches Sarvam Token Factory: Bringing India-Hosted Inference to 8 Open Models
AI & Technology

Sarvam Launches Sarvam Token Factory: Bringing India-Hosted Inference to 8 Open Models

November 17, 2026
Adam Mosseri Thinks It Will Be ‘Pretty Challenging’ To Design Instagram For The iPhone Duo
AI & Technology

Adam Mosseri Thinks It Will Be ‘Pretty Challenging’ To Design Instagram For The iPhone Duo

November 17, 2026
TRMNL Frame Is A 00 Full-Color E-Paper Display That Looks Like Real Art
AI & Technology

TRMNL Frame Is A $1300 Full-Color E-Paper Display That Looks Like Real Art

November 17, 2026
Next Post
(MARKET UPDATE) Don’t Miss This Tomorrow…

(MARKET UPDATE) Don't Miss This Tomorrow...

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Search

No Result
View All Result
This Morning’s Top Headlines – Oct. 26 | Morning News NOW

This Morning’s Top Headlines – Oct. 26 | Morning News NOW

November 16, 2026
What Is Nvidia Without Jensen Huang?

What Is Nvidia Without Jensen Huang?

November 15, 2026
Fans cheer Milwaukee Brewers at World Series victory parade

Fans cheer Milwaukee Brewers at World Series victory parade

November 13, 2026

About

Learn more

Our Services

Legal

Privacy Policy

Terms of Use

Bloggers

Learn more

Article Links

Contact

Advertise

Ask us anything

©2020- TradePoint.io - All rights reserved!

Tradepoint.io, being just a publishing and technology platform, is not a registered broker-dealer or investment adviser. So we do not provide investment advice. Rather, brokerage services are provided to clients of Tradepoint.io by independent SEC-registered broker-dealers and members of FINRA/SIPC. Every form of investing carries some risk and past performance is not a guarantee of future results. “Tradepoint.io“, “Instant Investing” and “My Trading Tools” are registered trademarks of Apperbuild, LLC.

This website is operated by Apperbuild, LLC. We have no link to any brokerage firm and we do not provide investment advice. Every information and resource we provide is solely for the education of our readers. © 2020 Apperbuild, LLC. All rights reserved.

No Result
View All Result
  • Main
  • AI & Technology
  • Stock Charts
  • Market & News
  • Business
  • Finance Tips
  • Trade Tube
  • Blog
  • Shop

© 2023 - TradePoint.io - All Rights Reserved!