Nums AI has released Causilo, a pretrained tabular foundation model for classification and regression. Causilo comes with a scikit-learn interface, Apache-2.0 code and pretrained weights on Hugging Face. On TabArena, it has the highest Elo among single models for both classification and regression.
Is it deployable? Yes, for research and evaluation today, on CUDA or CPU. Commercial, production and hosted API use need a separate license from Nums AI.
What Causilo Does
Causilo is an in-context learning model. Calling fit does not update the pretrained weights. It stores the training rows as context and predicts query rows in one forward pass. According to its TabArena submission, Nums AI pretrained Causilo only on synthetic data, with no TabArena datasets.
Inputs can be NumPy arrays or pandas DataFrames, including categorical features and missing values. Classification supports up to 10 classes. Regression returns mean predictions by default. Version 1.0.1 adds median and quantile outputs, based on 999 native quantiles.
Architecture: Refinement, Compression, In-Context Learning
Nums AI splits the network into 3 phases: refinement, compression and in-context learning. The released code and configs show how each phase works.
Features are grouped in sets of 3. Each value is embedded with 16 learned sine and cosine frequencies. Missing values get their own learned vector.
2 column stages summarize each feature group. In each, 128 latent slots read only the training rows and pass that summary to every row. Between the 2 column stages, a row stage lets feature groups interact through 4 latent tokens. It uses cross-attention instead of full self-attention, which Nums AI says keeps cost linear in feature count.
A pooling block then compresses each row into a fixed 512-dimensional vector. Labels are added to the training rows. A 12-layer prediction block lets query rows attend to those labeled rows. Query rows cannot change the training context or each other.
By default, 8 ensemble members share the same weights. Each one cycles through none, rank2gaussian, robust or power normalization, with seeded feature and class permutations.
TabArena Results
Nums AI used the official TabArena pipeline: 51 datasets and 816 Full splits, with 8 estimators and seed 42. A TabArena maintainer re-ran the full evaluation and got the same overall Elo of 1794.
| Task | Causilo Elo | Next best single model | Causilo improvability |
|---|---|---|---|
| Overall | 1792.9 | TabFM, 1764.4 | 0.0684 |
| Classification | 1771.8 | EXAONE Tabular, 1758.8 | 0.0875 |
| Regression | 2032.6 | TabFM, 1992.8 | 0.0125 |
The field includes Google Research’s TabFM, LG AI Research’s EXAONE Tabular and Prior Labs’ TabPFN-3 (1636.2 overall).
A few points help when reading these numbers:
- The #1 positions exclude system entries. With systems included, the maintainer re-run placed Causilo 3rd of 88 overall.
- On improvability, TabFM still leads overall and on classification. Causilo leads on regression.
- The Elo confidence intervals at the top overlap, so the lead over TabFM and EXAONE Tabular is narrow.
- Nums AI also lists Xiaomi-TabLDM and Amazon’s Mitra-v2 behind Causilo. Neither model appears in the benchmark files in Causilo’s repo.
ScoringBench Results
ScoringBench scores regression models with proper scoring rules such as CRPS, alongside RMSE and R². Nums AI submitted Causilo 1.0.1 on 101 datasets, 5 folds each, capped at 3,000 samples. Nums AI reports that Causilo ranks 1st by CRPS, R² and RMSE. The ScoringBench maintainer independently checked the results before committing them.
Speed and Memory
Nums AI also reran 3 models on 1 H100 80 GB GPU, with 8 CPU cores per job.
| Model | Fit (s per 1k rows) | Predict (s per 1k rows) | GPU memory (GiB) |
|---|---|---|---|
| Causilo | 2.504 | 0.251 | 8.15 |
| TabICLv2 | 3.449 | 0.303 | 8.37 |
| TabPFN-3 | 4.18 | 0.686 | 0.88 |
In this test, Causilo is fastest on both fit and predict. TabPFN-3 uses far less GPU memory. Setting use_kv_cache=True moves context work into fit, using more memory to speed up repeated predictions.
Getting Started
Causilo needs Python 3.10 to 3.12 and PyTorch 2.13 or newer. The first fit downloads the checkpoint automatically.
# pip install causilo
from causilo import CausiloClassifier, CausiloRegressor
clf = CausiloClassifier(n_estimators=8, random_state=42)
clf.fit(X_train, y_train)
proba = clf.predict_proba(X_test)
reg = CausiloRegressor()
reg.fit(X_train, y_train)
bands = reg.predict(X_test, output_type="quantiles", quantiles=[0.05, 0.5, 0.95])You can also try the Hugging Face demo Space.
Credit: Source link
