• bitcoinBitcoin(BTC)$64,231.00-0.73%
  • ethereumEthereum(ETH)$1,810.940.15%
  • tetherTether(USDT)$1.00-0.02%
  • binancecoinBNB(BNB)$585.64-0.80%
  • usd-coinUSDC(USDC)$1.00-0.01%
  • rippleXRP(XRP)$0.95-1.18%
  • solanaSolana(SOL)$70.20-1.33%
  • tronTRON(TRX)$0.304907-0.33%
  • Figure HelocFigure Heloc(FIGR_HELOC)$1.060.00%
  • HyperliquidHyperliquid(HYPE)$65.93-1.88%
  • zcashZcash(ZEC)$850.16-4.60%
  • moneroMonero(XMR)$497.13-0.31%
  • dogecoinDogecoin(DOGE)$0.058300-0.04%
  • leo-tokenLEO Token(LEO)$9.200.63%
  • USDSUSDS(USDS)$1.000.05%
  • whitebitWhiteBIT Coin(WBT)$62.18-0.65%
  • chainlinkChainlink(LINK)$9.06-1.50%
  • cardanoCardano(ADA)$0.151030-1.33%
  • RainRain(RAIN)$0.007715108.22%
  • Ethena USDeEthena USDe(USDE)$1.00-0.02%
  • stellarStellar(XLM)$0.136576-1.38%
  • daiDai(DAI)$1.000.01%
  • bitcoin-cashBitcoin Cash(BCH)$205.00-1.30%
  • nearNEAR Protocol(NEAR)$3.12-0.57%
  • litecoinLitecoin(LTC)$44.05-1.46%
  • CantonCanton(CC)$0.081967-0.76%
  • avalanche-2Avalanche(AVAX)$7.06-1.50%
  • tether-goldTether Gold(XAUT)$3,855.091.04%
  • uniswapUniswap(UNI)$5.06-1.27%
  • USD1USD1(USD1)$1.000.06%
  • the-open-networkGram (prev. Toncoin)(GRAM)$1.03-1.21%
  • suiSui(SUI)$0.71-0.99%
  • Ripple USDRipple USD(RLUSD)$1.000.01%
  • Global DollarGlobal Dollar(USDG)$1.000.01%
  • hedera-hashgraphHedera(HBAR)$0.057363-1.57%
  • paypal-usdPayPal USD(PYUSD)$1.000.02%
  • Ondo US Dollar YieldOndo US Dollar Yield(USDY)$1.150.02%
  • shiba-inuShiba Inu(SHIB)$0.000004-1.78%
  • quant-networkQuant(QNT)$153.98-0.47%
  • crypto-com-chainCronos(CRO)$0.042950-1.65%
  • BittensorBittensor(TAO)$186.73-1.97%
  • Circle USYCCircle USYC(USYC)$1.140.00%
  • Spiko Amundi Overnight Swap Fund (EUR)Spiko Amundi Overnight Swap Fund (EUR)(EURSAFO)$1.12-0.11%
  • MemeCoreMemeCore(M)$0.853.40%
  • okbOKB(OKB)$84.83-2.64%
  • pax-goldPAX Gold(PAXG)$3,856.861.03%
  • aaveAave(AAVE)$107.591.17%
  • Falcon USDFalcon USD(USDF)$1.00-0.02%
  • BlackRock USD Institutional Digital Liquidity FundBlackRock USD Institutional Digital Liquidity Fund(BUIDL)$1.000.00%
  • United StablesUnited Stables(U)$1.00-0.03%
TradePoint.io
  • Main
  • AI & Technology
  • Stock Charts
  • Market & News
  • Business
  • Finance Tips
  • Trade Tube
  • Blog
  • Shop
No Result
View All Result
TradePoint.io
No Result
View All Result

The Kolmogorov-Arnold Theorem Revisited: Why Averaging Functions Work Better

August 4, 2024
in AI & Technology
Reading Time: 5 mins read
A A
The Kolmogorov-Arnold Theorem Revisited: Why Averaging Functions Work Better
ShareShareShareShareShare

Kolmogorov-Arnold Networks (KANs) have emerged as a promising alternative to traditional Multi-Layer Perceptrons (MLPs). Inspired by the Kolmogorov-Arnold representation theorem, these networks utilize neurons that perform simple summation operations. However, the current implementation of KANs poses some challenges in practical applications. Currently, researchers are investigating the possibility of identifying alternative multivariate functions for KAN neurons that could offer enhanced practical utility across several benchmarks related to machine-learning tasks.

Research has highlighted the potential of KANs in various fields, like computer vision, time series analysis, and quantum architecture search. Some studies show that KANs can outperform MLPs in data fitting and PDE tasks while using fewer parameters. However, some research has raised concerns about the robustness of KANs to noise and their performance compared to MLPs. Variations and improvements to the standard KAN architecture are also explored, such as graph-based designs, convolutional KANs, and transformer-based KANs to solve the issues. Moreover, alternative activation functions like wavelets, radial basis functions, and sinusoidal functions are investigated to improve KAN efficiency. Despite these works, there is a need for further improvements to enhance KAN performance.

YOU MAY ALSO LIKE

AMD Closes World Labs Acquisition and Names Fei-Fei Li Chief Scientist – Unite.AI

Hands-On (And Off) With Mercedes-Benz’s Upcoming MB.Drive Assist Pro System

A Researcher from the Center for Applied Intelligent Systems Research at Halmstad University, Sweden, has proposed a novel approach to enhance the performance of Kolmogorov-Arnold Networks (KANs). This method aims to identify the optimal multivariate function for KAN neurons across various machine learning classification tasks. The traditional use of addition as the node-level function is often non-ideal, especially for high-dimensional datasets with multiple features. This can cause the inputs to exceed the effective range of subsequent activation functions, leading to training instability and reduced generalization performance. To solve this problem, the researcher suggests using the mean instead of the sum as the node function. 

To evaluate the proposed KAN modifications, 10 popular datasets from the UCI Machine Learning Database Repository are utilized, covering multiple domains and varying sizes. These datasets are divided into training (60%), validation (20%), and testing (20%) partitions. A standardized preprocessing method is applied across all datasets, which includes categorical feature encoding, missing value imputation, and instance randomization. Models are trained for 2000 iterations using the Adam optimizer with a learning rate of 0.01 and a batch size of 32. Model accuracy on the testing set serves as the primary evaluation metric. The parameter count is managed by setting the grid to 3 and using default hyperparameters for the KAN models.

The results support the hypothesis that using the mean function in KAN neurons is more effective than the traditional sum function. This enhancement is due to the mean’s ability to keep input values within the optimal range of the spline activation function, which is [-1.0, +1.0]. Standard KANs struggled to keep values within this range in intermediate layers as the number of features increased. However, adopting the mean function in neurons leads to enhanced performance, keeping values within the desired range across datasets with 20 or more features. For datasets with fewer features, values stayed within the range more than 99.0% of the time, except for the ‘abalone’ dataset, which had a slightly lower adherence rate of 96.51%.

In this paper, a Researcher from the Center for Applied Intelligent Systems Research at Halmstad University, Sweden, has proposed a method to enhance the performance of KANs. An important modification to KANs is introduced in this paper by replacing the traditional summation in KAN neurons with an averaging function. Experimental results show that this change leads to more stable training processes and keeps inputs within the effective range of spline activations. This adjustment to KAN architecture solves previous challenges related to input range and training stability. In the future, this work offers a promising direction for future KAN implementations, potentially enhancing their performance and applicability in various machine-learning tasks.


Check out the Paper. All credit for this research goes to the researchers of this project. Also, don’t forget to follow us on Twitter and join our Telegram Channel and LinkedIn Group. If you like our work, you will love our newsletter..

Don’t Forget to join our 47k+ ML SubReddit

Find Upcoming AI Webinars here



Sajjad Ansari is a final year undergraduate from IIT Kharagpur. As a Tech enthusiast, he delves into the practical applications of AI with a focus on understanding the impact of AI technologies and their real-world implications. He aims to articulate complex AI concepts in a clear and accessible manner.


Credit: Source link

ShareTweetSendSharePin

Related Posts

AMD Closes World Labs Acquisition and Names Fei-Fei Li Chief Scientist – Unite.AI
AI & Technology

AMD Closes World Labs Acquisition and Names Fei-Fei Li Chief Scientist – Unite.AI

November 16, 2026
Hands-On (And Off) With Mercedes-Benz’s Upcoming MB.Drive Assist Pro System
AI & Technology

Hands-On (And Off) With Mercedes-Benz’s Upcoming MB.Drive Assist Pro System

November 16, 2026
Zoox Brings Purpose-Built Robotaxis to Uber Riders in Las Vegas – Unite.AI
AI & Technology

Zoox Brings Purpose-Built Robotaxis to Uber Riders in Las Vegas – Unite.AI

November 16, 2026
Moth’s Tessa Encodes and Recovers 1,568 Values on a 120-Qubit Chip – Unite.AI
AI & Technology

Moth’s Tessa Encodes and Recovers 1,568 Values on a 120-Qubit Chip – Unite.AI

November 16, 2026
Next Post
Dave Ramsey’s advice for finding financial balance

Dave Ramsey’s advice for finding financial balance

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Search

No Result
View All Result
Reinstatement Letter for Jensen Huang Surpasses 100,000 Signatures – Unite.AI

Reinstatement Letter for Jensen Huang Surpasses 100,000 Signatures – Unite.AI

November 14, 2026
Parisians protest in Halloween costumes

Parisians protest in Halloween costumes

November 13, 2026
Sen. Schumer: ‘We have seen a generational remake of the Senate map’

Sen. Schumer: ‘We have seen a generational remake of the Senate map’

November 10, 2026

About

Learn more

Our Services

Legal

Privacy Policy

Terms of Use

Bloggers

Learn more

Article Links

Contact

Advertise

Ask us anything

©2020- TradePoint.io - All rights reserved!

Tradepoint.io, being just a publishing and technology platform, is not a registered broker-dealer or investment adviser. So we do not provide investment advice. Rather, brokerage services are provided to clients of Tradepoint.io by independent SEC-registered broker-dealers and members of FINRA/SIPC. Every form of investing carries some risk and past performance is not a guarantee of future results. “Tradepoint.io“, “Instant Investing” and “My Trading Tools” are registered trademarks of Apperbuild, LLC.

This website is operated by Apperbuild, LLC. We have no link to any brokerage firm and we do not provide investment advice. Every information and resource we provide is solely for the education of our readers. © 2020 Apperbuild, LLC. All rights reserved.

No Result
View All Result
  • Main
  • AI & Technology
  • Stock Charts
  • Market & News
  • Business
  • Finance Tips
  • Trade Tube
  • Blog
  • Shop

© 2023 - TradePoint.io - All Rights Reserved!