| Cmp | # | Model | Params | Int Index▼ | HellaSwag | ARC-E | ARC-C | PIQA | ArithMark-3 | Avg | Fit Std Dev | ArithMark-2 | Released |
|---|
Dashed line: Non-linear scaling fit. Shaded zone: Outperforming parameter expectation.
Top models for the selected benchmark.
Normalized multi-task comparison.
| # | ORGANIZATION | MODELS TRACKED | MEAN FIT STD DEV | MEAN SCORE | TOP MODEL |
|---|
You can now filter models by architecture type. Standard models work out of the box with GGUF, vLLM, and standard runtimes with zero remote code execution.
ArithMark-3 is now the primary math benchmark for ranking, weighting subject to change. ArithMark-2 remains available in the expanded columns.
UCR's 2.74M SLM uses custom tokenization and digit features to reach 69.4% ArithMark accuracy, setting the leaderboard's #1 score.
Built on Axiomic Labs' T-X4 refresh-gated XSA stack, GPT-S2-5M pushes into the sub-10M bracket and lands at the top, dethroning SLM-10M.
Standard raw accuracy averages reward models for trivial chance performance on multiple-choice benchmarks. The Intelligence Index normalizes each benchmark against its theoretical random baseline, ensuring that chance performance equals 0 and perfect performance equals 100.
* Combined ARC is defined as (ARC-Easy + ARC-Challenge) / 2 prior to normalization. All benchmarks (HellaSwag, ARC, PIQA, and ArithMark-3) are strictly required. Models with incomplete evaluation suites will not receive an official Intelligence Index ranking.
Quick guidelines to get your model evaluated and listed on the board.
Keep your total parameters under 150 million. If you are building a Mixture of Experts (MoE), both the total weights on disk and the active weights per token during inference must stay under this 150M limit.
The base model lineage must be your own. Training from scratch, distilling from larger teachers with logits or synthetic data, and SFT chat fine-tunes of your own models are all welcome. Please do not submit fine-tunes, pruned versions, or weight merges built on top of someone else's pretrained model.
Your Hugging Face repo needs to be public with working weights, tokenizer files, and config. If you are using a custom architecture with custom modeling code, make sure it imports cleanly and runs inference without breaking.
Run your benchmarks through standard LM-Eval Harness (make sure to report acc_norm when available) and use the official ArithMark-3.0 eval script for math.
We cannot stress this enough: evaluations must be strictly zero-shot. That means num_fewshot: 0 across the board. Not 1-shot, not 5-shot, zero. Do not submit scores obtained with few-shot prompting expecting them to pass. We re-run every model strictly at 0-shot on our own evaluation GPUs, and if your reported numbers do not match because they relied on few-shot examples, your submission will be rejected. 0 means 0.
We re-run and audit every submission on our own evaluation GPUs before merging. Keep your datasets clean: do not include benchmark test splits in pretraining or SFT, and do not use synthetic pipelines built to copy or paraphrase evaluation questions. Prompt formatting hacks or gaming zero-shot setups will get the model rejected.
We reproduce evaluations on our own hardware. Tiny rounding differences from CUDA kernels or float precision are fine, but your numbers must land within standard error (stderr). If any reproduced score drifts outside that margin from what you submitted, we are going to have a problem and the run will be rejected.
Open a Discussion or Pull Request on this Space with your Hugging Face model link and raw eval JSON files. We will run our verification checks and add you to the leaderboard.
AtomixLabs indexes and evaluates open-weight models independently and through community submissions. If your model is public on Hugging Face, it can be benchmarked and listed on this board. We do not allow public models to opt out, because selective reporting defeats the purpose of an open, objective leaderboard.
Discretionary review and enforcement: Rankings are strictly merit-based and depend on reproduced evaluation runs. AtomixLabs reserves full authority to flag, unlist, or remove any model if there is reasonable suspicion of data leakage, benchmark gaming, or bad-faith behavior, even if the specific trick or loophole is not explicitly covered in the written rules.
Independence and affiliation: Listing a model or organization on this leaderboard does not mean AtomixLabs is affiliated with, partnered with, or knows the creators. We are an independent benchmarking initiative that tests publicly released models across the open-source community.