An open source LLM inference engine written in pure Rust and CUDA. It serves an OpenAI-compatible API from a single binary, with no Python and no PyTorch in the serving path. One codebase covers the range, from edge-class accelerators through workstations to expert-parallel deployments across nodes.

What hardware does Atlas run on?

NVIDIA DGX Spark (GB10) is verified today, and AMD Strix Halo (gfx1151) runs the same CUDA source compiled through SCALE by Spectral Compute — one codebase, no HIP port. Both were submitted to MLPerf Inference v6.1 in the closed edge division.

Is Atlas faster than vLLM on a DGX Spark?

On the published concurrency ladder, yes at every rung from C=1 to C=128, by 1.012x to 1.225x against whichever vLLM configuration is faster at that concurrency. The margin is widest at the top, because between C=64 and C=128 Atlas keeps scaling and the vLLM configuration that leads the mid-ladder stops. Same box, same checkpoint, same client, same prompts, greedy sampling with matched penalties. The full campaign log, including the rungs we lost on the way, is in the repo.

One command: curl -fsSL https://atlasinference.io/install.sh | sh. It downloads a prebuilt atlasctl, verifies its checksum, and installs to ~/.local/bin. If you would rather not pipe curl to a shell, cargo install atlasctl does the same thing from source.

Which models can I run?

Every model on this page maps to a recipe in the atlas-recipes repository, which is the single source of truth — the site cannot list a model that has no recipe. Qwen3.6 is the flagship family, alongside Gemma, Nemotron, Mistral, MiniMax and DeepSeek.

What does “verified” mean here?

An image ships only after the serve matrix passes: every model boots, stays coherent under greedy determinism with no token leakage and reliable tool calls, and holds throughput within 10% of its committed baseline. A release that ships slower than its baseline fails the gate.

Why does concurrency matter more than single-stream speed?

Because agentic systems do not send one request at a time. A fleet of tool-calling agents sharing a context bus arrives as many concurrent streams, so the engine is judged where the requests pile up. On the published ladder Atlas keeps gaining throughput from C=64 to C=128 while the leading vLLM configuration does not, and an engine that flattens under load caps how many agents a given box can actually run.

What license is Atlas under, and can I use it commercially?

The Community Edition is AGPL-3.0-only. Contributions are covered by a CLA that permits re-licensing for the Enterprise Edition. If you are running Atlas in production or need different terms, email us.

Does Atlas run multi-node?

Yes. EP=2 expert parallelism across two DGX Sparks is supported and shipped as recipes; those cards are marked EP=2 in the model list. A three-node GB10 topology is being wired up now.

Atlas, pure Rust inference for DGX Spark

Throughput in tok/s. Ratio is Atlas over the faster vLLM configuration at that rung; where the two disagree the losing one is not silently dropped.
C	Atlas	vLLM + MTP	vLLM, no speculation	Ratio
1	23.59	19.72	11.04	1.196×
2	41.02	37.11	21.34	1.105×
4	74.21	71.61	41.20	1.036×
8	125.95	124.48	78.18	1.012×
16	203.36	197.03	137.11	1.032×
32	291.01	283.48	219.50	1.027×
64	386.63	361.39	312.26	1.070×
128	478.11	358.57	390.42	1.225×

Checkpoint: unsloth/Qwen3.8-27B-NVFP4
Hardware: NVIDIA GB10 Grace Blackwell, 121.7 GB unified — dgx2 (spark-43fa)
Workload: ISL 128 / OSL 1024, 3 timed reps + 1 warmup, temperature 0, seed 42
Parity: disabled on both engines via chat_template_kwargs.enable_thinking=false; presence_penalty and frequency_penalty pinned to 0.0 on both engines

Atlas Atlas 1.0.0-beta-preview

Round 11 throughout, except C=2. C=2 is the 2026-08-18 hardening run (c2_atlas_dgx2_20260818.json), not round 8: the certified table originally carried round 8's 38.95 there, which won by 1.004x -- a margin inside either engine's run-to-run spread. Both legs were re-measured back to back on dgx2 on 2026-08-18 (Atlas 15:05, vLLM+MTP 15:20) and that PAIR replaced the earlier pair, which is why the Atlas number rose to 41.02 while the paired vLLM number fell to 37.11 and the margin moved to 1.105x with non-overlapping distributions. A round-11 C=2 exists at 40.42 (l38_r11_c2.json) but has no same-day vLLM leg, so it is not publishable as a pair. See RESULTS.md, C=2 HARDENED.

Buildbce152b18

Certified at 4012c9b7e1, which differs only in doc comments and gate machinery -- no executable change. Merged to main as 60370b9532.

SpeculationMTP, K=4 (--num-drafts 3); self-disables above 32 concurrent sequences

EnvATLAS_PREFILL_CODISPATCH=1 ATLAS_FP8_ROWWISE=1 ATLAS_MTP_DCUT_RATIO=1.0 ATLAS_MTP_K_LADDER=1:3,2:1,4:2,8:2,16:1

Command

spark serve unsloth/Qwen3.8-27B-NVFP4 --host 0.0.0.0 --port 8888 --model-name unsloth/Qwen3.8-27B-NVFP4 --max-seq-len 2048 --max-batch-size 128 --gpu-memory-utilization 0.85 --kv-cache-dtype fp8 --enable-prefix-caching true --ssm-cache-slots 8 --ssm-checkpoint-interval 32 --speculative --num-drafts 3 --mtp-quantization bf16 --scheduling-policy fifo --tool-call-parser qwen3_coder --disable-tool-grammar true --disable-thinking --request-timeout 0 --ssm-h-dtype f16-pool --gdn-fused-norm --ssm-batched-recurrent --ssm-tail-midchunk false --mtp-gate force --prefill-varlen-batch --no-tui

vLLM + MTP vLLM 0.27.1

The apples-to-apples reference: ctx 2048, batch cap 128, util 0.85, fp8 KV, prefix caching, thinking off and MTP K=4 -- every axis matched to Atlas. vLLM at its best, not a handicapped baseline.

Buildvllm/vllm-openai:latest @ sha256:0a51ea5b4ae2dc5d81890e5173f54203d2a3ae0cfffe51b8fd2afd4391bfd967

SpeculationvLLM's own Qwen3_5MTP, K=4 (num_speculative_tokens=3)

EnvHF_HUB_OFFLINE=1

Command

vllm serve --model unsloth/Qwen3.8-27B-NVFP4 --served-model-name unsloth/Qwen3.8-27B-NVFP4 --host 0.0.0.0 --port 8001 --max-model-len 2048 --max-num-seqs 128 --gpu-memory-utilization 0.85 --enable-prefix-caching --dtype bfloat16 --kv-cache-dtype fp8 --speculative-config '{"method":"mtp","num_speculative_tokens":3}'

vLLM, no speculation vLLM 0.27.1

Not matched to Atlas: bf16 KV cache (Atlas and the MTP leg use fp8); ctx 4096 (the matched legs use 2048); no speculative decoding. Shown because it is the faster vLLM configuration at C=128.

Published because it beats vLLM+MTP at C=128 (390.42 vs 358.57): MTP verification costs vLLM more than it gains 128-wide. Quoting only the MTP leg there would overstate our margin, so the headline ratio is taken against whichever vLLM configuration is faster at each rung.

Buildvllm/vllm-openai:latest @ sha256:0a51ea5b4ae2dc5d81890e5173f54203d2a3ae0cfffe51b8fd2afd4391bfd967

Speculationnone

EnvHF_HUB_OFFLINE=1

Command

vllm serve --model unsloth/Qwen3.8-27B-NVFP4 --served-model-name unsloth/Qwen3.8-27B-NVFP4 --host 0.0.0.0 --port 8001 --max-model-len 4096 --max-num-seqs 128 --gpu-memory-utilization 0.85 --enable-prefix-caching --dtype bfloat16 --kv-cache-dtype bfloat16

Per-rung detail

Series	C	tok/s	median	spread	TTFT p50	TPOT p50	source file
Atlas	1	23.59	23.55	2.78%	682.54 ms	41.84 ms	l38_r11_c1.json
Atlas	2	41.02	41.37	4.15%	795.2 ms	47.53 ms	c2_atlas_dgx2_20260818.json
Atlas	4	74.21	74.29	2.79%	1296.3 ms	51.76 ms	l38_r11_c4.json
Atlas	8	125.95	125.83	2.92%	2331.47 ms	59.01 ms	l38_r11_c8.json
Atlas	16	203.36	203.45	1.23%	4252.58 ms	72.67 ms	l38_r11_c16.json
Atlas	32	291.01	292.50	2.43%	8281.81 ms	98.91 ms	l38_r11_c32.json
Atlas	64	386.63	387.48	1.68%	16052.93 ms	148.81 ms	l38_r11_c64.json
Atlas	128	478.11	480.05	2.05%	31760.85 ms	234.6 ms	l38_r11_c128.json
vLLM + MTP	1	19.72	19.72	1.89%	286.71 ms	50.37 ms	vllm_fp8_mtp_reference.json
vLLM + MTP	2	37.11	37.18	2.94%	390.84 ms	53.22 ms	c2_vllm_mtp_dgx2_20260818.json
vLLM + MTP	4	71.61	72.05	2.82%	604.84 ms	54.51 ms	vllm_fp8_mtp_reference.json
vLLM + MTP	8	124.48	124.31	1.25%	936.14 ms	60.8 ms	vllm_fp8_mtp_reference.json
vLLM + MTP	16	197.03	197.85	3.51%	1620.35 ms	76.25 ms	vllm_fp8_mtp_reference.json
vLLM + MTP	32	283.48	284.20	1.27%	3066.96 ms	107.45 ms	vllm_fp8_mtp_reference.json
vLLM + MTP	64	361.39	361.18	0.21%	5278.74 ms	167.48 ms	vllm_fp8_mtp_reference.json
vLLM + MTP	128	358.57	358.61	0.44%	8500.7 ms	269.54 ms	vllm_fp8_mtp_reference.json
vLLM, no speculation	1	11.04	11.05	0.32%	155.95 ms	90.4 ms	vllm_latest_reference.json
vLLM, no speculation	2	21.34	21.34	0.04%	235.35 ms	93.52 ms	vllm_latest_reference.json
vLLM, no speculation	4	41.20	41.20	0.03%	419.59 ms	96.77 ms	vllm_latest_reference.json
vLLM, no speculation	8	78.18	78.18	0.02%	729.44 ms	101.71 ms	vllm_latest_reference.json
vLLM, no speculation	16	137.11	137.11	0.04%	1176.06 ms	115.56 ms	vllm_latest_reference.json
vLLM, no speculation	32	219.50	219.45	0.09%	2240.16 ms	143.58 ms	vllm_latest_reference.json
vLLM, no speculation	64	312.26	312.31	0.12%	4009.21 ms	200.87 ms	vllm_latest_reference.json
vLLM, no speculation	128	390.42	390.15	0.31%	6710.78 ms	320.84 ms	vllm_latest_reference.json

Harness bench/ladder38/harness_w55_conc_ladder.py. Two harness revisions appear above: 6412b12d4d — vLLM legs. Does not send the penalty keys.; 1c77e1d8e9 — Atlas legs (rounds 7-11). Adds explicit presence_penalty/frequency_penalty = 0.0. The only difference is those two keys. vLLM already defaults both to 0.0, so its sampling is byte-identical either way; the keys exist to stop Atlas's non_thinking preset injecting presence_penalty=1.5. Both engines therefore ran identical sampling.

Full campaign log, including every rung we lost on the way and the three claims we retracted: bench/ladder38/RESULTS.md. Generated 2026-08-27T11:06:59Z from the committed measurements.

One inference engine, from the device in your hand to the datacenter rack.

Every number is a receipt.

Faster than vLLM at every concurrency, C=1 to 128

Atlas Atlas 1.0.0-beta-preview

vLLM + MTP vLLM 0.27.1

vLLM, no speculation vLLM 0.27.1

Per-rung detail

The top of the ladder is the part that matters.

What the gate checks

What just happened.

We helped build the MLPerf Edge Agentic benchmark

Atlas running on AMD Strix Halo

Our MLPerf submission is in

One engine, every tier.

NVIDIA DGX Spark

AMD Strix Halo

Every model here has a recipe.

Qwen · Qwen3.6

Up and running in one command.

Prefer to inspect first?

Built in the open, starred in the open.

The action is in Discord.

Your machine is the test fleet.

Run the serve matrix

Add or tune a recipe

Kernels in Rust and CUDA

Docs, triage, ideas

What we are building next.

Three node GB10 topology

Intel Arc Pro B70

AMD Strix Halo

MLPerf Inference v6.1

Qwen GDN kernel upstream

Bigger model support

AI worth having, on hardware you own.

The questions we actually get asked.

Come work with us.

Business

Partnerships

Hardware