An open source LLM inference engine written in pure Rust and CUDA, tuned for prosumer AI workstations rather than datacenter racks. It serves an OpenAI-compatible API from a single binary, with no Python and no PyTorch in the serving path.

What hardware does Atlas run on?

NVIDIA DGX Spark (GB10) is verified today, and AMD Strix Halo (gfx1151) runs the same CUDA source compiled through SCALE by Spectral Compute — one codebase, no HIP port. Both were submitted to MLPerf Inference v6.1 in the closed edge division.

Is Atlas faster than vLLM on a DGX Spark?

On the published concurrency ladder, yes at every rung from C=1 to C=128, by 1.012x to 1.225x against whichever vLLM configuration is faster at that concurrency. Same box, same checkpoint, same client, same prompts, greedy sampling with matched penalties. The full campaign log, including the rungs we lost on the way, is in the repo.

One command: curl -fsSL https://atlasinference.io/install.sh | sh. It downloads a prebuilt atlasctl, verifies its checksum, and installs to ~/.local/bin. If you would rather not pipe curl to a shell, cargo install atlasctl does the same thing from source.

Which models can I run?

Every model on this page maps to a recipe in the atlas-recipes repository, which is the single source of truth — the site cannot list a model that has no recipe. Qwen3.6 is the flagship family, alongside Gemma, Nemotron, Mistral, MiniMax and DeepSeek.

What does “verified” mean here?

An image ships only after the serve matrix passes: every model boots, stays coherent under greedy determinism with no token leakage and reliable tool calls, and holds throughput within 10% of its committed baseline. A release that ships slower than its baseline fails the gate.

What license is Atlas under, and can I use it commercially?

The Community Edition is AGPL-3.0-only. Contributions are covered by a CLA that permits re-licensing for the Enterprise Edition. If you are running Atlas in production or need different terms, email us.

Does Atlas run multi-node?

Yes. EP=2 expert parallelism across two DGX Sparks is supported and shipped as recipes; those cards are marked EP=2 in the model list.

Atlas, pure Rust inference for DGX Spark

Throughput in tok/s. Ratio is Atlas over the faster vLLM configuration at that rung; where the two disagree the losing one is not silently dropped.
C	Atlas	vLLM + MTP	vLLM, no speculation	Ratio
1	23.59	19.72	11.04	1.196×
2	41.02	37.11	21.34	1.105×
4	74.21	71.61	41.20	1.036×
8	125.95	124.48	78.18	1.012×
16	203.36	197.03	137.11	1.032×
32	291.01	283.48	219.50	1.027×
64	386.63	361.39	312.26	1.070×
128	478.11	358.57	390.42	1.225×

Checkpoint: unsloth/Qwen3.8-27B-NVFP4
Hardware: NVIDIA GB10 Grace Blackwell, 121.7 GB unified — dgx2 (spark-43fa)
Workload: ISL 128 / OSL 1024, 3 timed reps + 1 warmup, temperature 0, seed 42
Parity: disabled on both engines via chat_template_kwargs.enable_thinking=false; presence_penalty and frequency_penalty pinned to 0.0 on both engines

Atlas Atlas 1.0.0-beta-preview

Round 11 throughout. C=2 is round 8 on the identical configuration -- round 11 re-measured that rung at 40.42, and we publish the lower, older number rather than the better one.

Buildbce152b18

Certified at 4012c9b7e1, which differs only in doc comments and gate machinery -- no executable change. Merged to main as 60370b9532.

SpeculationMTP, K=4 (--num-drafts 3); self-disables above 32 concurrent sequences

EnvATLAS_PREFILL_CODISPATCH=1 ATLAS_FP8_ROWWISE=1 ATLAS_MTP_DCUT_RATIO=1.0 ATLAS_MTP_K_LADDER=1:3,2:1,4:2,8:2,16:1

Command

spark serve unsloth/Qwen3.8-27B-NVFP4 --host 0.0.0.0 --port 8888 --model-name unsloth/Qwen3.8-27B-NVFP4 --max-seq-len 2048 --max-batch-size 128 --gpu-memory-utilization 0.85 --kv-cache-dtype fp8 --enable-prefix-caching true --ssm-cache-slots 8 --ssm-checkpoint-interval 32 --speculative --num-drafts 3 --mtp-quantization bf16 --scheduling-policy fifo --tool-call-parser qwen3_coder --disable-tool-grammar true --disable-thinking --request-timeout 0 --ssm-h-dtype f16-pool --gdn-fused-norm --ssm-batched-recurrent --ssm-tail-midchunk false --mtp-gate force --prefill-varlen-batch --no-tui

vLLM + MTP vLLM 0.27.1

The apples-to-apples reference: ctx 2048, batch cap 128, util 0.85, fp8 KV, prefix caching, thinking off and MTP K=4 -- every axis matched to Atlas. vLLM at its best, not a handicapped baseline.

Buildvllm/vllm-openai:latest @ sha256:0a51ea5b4ae2dc5d81890e5173f54203d2a3ae0cfffe51b8fd2afd4391bfd967

SpeculationvLLM's own Qwen3_5MTP, K=4 (num_speculative_tokens=3)

EnvHF_HUB_OFFLINE=1

Command

vllm serve --model unsloth/Qwen3.8-27B-NVFP4 --served-model-name unsloth/Qwen3.8-27B-NVFP4 --host 0.0.0.0 --port 8001 --max-model-len 2048 --max-num-seqs 128 --gpu-memory-utilization 0.85 --enable-prefix-caching --dtype bfloat16 --kv-cache-dtype fp8 --speculative-config '{"method":"mtp","num_speculative_tokens":3}'

vLLM, no speculation vLLM 0.27.1

Not matched to Atlas: bf16 KV cache (Atlas and the MTP leg use fp8); ctx 4096 (the matched legs use 2048); no speculative decoding. Shown because it is the faster vLLM configuration at C=128.

Published because it beats vLLM+MTP at C=128 (390.42 vs 358.57): MTP verification costs vLLM more than it gains 128-wide. Quoting only the MTP leg there would overstate our margin, so the headline ratio is taken against whichever vLLM configuration is faster at each rung.

Buildvllm/vllm-openai:latest @ sha256:0a51ea5b4ae2dc5d81890e5173f54203d2a3ae0cfffe51b8fd2afd4391bfd967

Speculationnone

EnvHF_HUB_OFFLINE=1

Command

vllm serve --model unsloth/Qwen3.8-27B-NVFP4 --served-model-name unsloth/Qwen3.8-27B-NVFP4 --host 0.0.0.0 --port 8001 --max-model-len 4096 --max-num-seqs 128 --gpu-memory-utilization 0.85 --enable-prefix-caching --dtype bfloat16 --kv-cache-dtype bfloat16

Per-rung detail

Series	C	tok/s	median	spread	TTFT p50	TPOT p50	source file
Atlas	1	23.59	23.55	2.78%	682.54 ms	41.84 ms	l38_r11_c1.json
Atlas	2	41.02	41.37	4.15%	795.2 ms	47.53 ms	c2_atlas_dgx2_20260818.json
Atlas	4	74.21	74.29	2.79%	1296.3 ms	51.76 ms	l38_r11_c4.json
Atlas	8	125.95	125.83	2.92%	2331.47 ms	59.01 ms	l38_r11_c8.json
Atlas	16	203.36	203.45	1.23%	4252.58 ms	72.67 ms	l38_r11_c16.json
Atlas	32	291.01	292.50	2.43%	8281.81 ms	98.91 ms	l38_r11_c32.json
Atlas	64	386.63	387.48	1.68%	16052.93 ms	148.81 ms	l38_r11_c64.json
Atlas	128	478.11	480.05	2.05%	31760.85 ms	234.6 ms	l38_r11_c128.json
vLLM + MTP	1	19.72	19.72	1.89%	286.71 ms	50.37 ms	vllm_fp8_mtp_reference.json
vLLM + MTP	2	37.11	37.18	2.94%	390.84 ms	53.22 ms	c2_vllm_mtp_dgx2_20260818.json
vLLM + MTP	4	71.61	72.05	2.82%	604.84 ms	54.51 ms	vllm_fp8_mtp_reference.json
vLLM + MTP	8	124.48	124.31	1.25%	936.14 ms	60.8 ms	vllm_fp8_mtp_reference.json
vLLM + MTP	16	197.03	197.85	3.51%	1620.35 ms	76.25 ms	vllm_fp8_mtp_reference.json
vLLM + MTP	32	283.48	284.20	1.27%	3066.96 ms	107.45 ms	vllm_fp8_mtp_reference.json
vLLM + MTP	64	361.39	361.18	0.21%	5278.74 ms	167.48 ms	vllm_fp8_mtp_reference.json
vLLM + MTP	128	358.57	358.61	0.44%	8500.7 ms	269.54 ms	vllm_fp8_mtp_reference.json
vLLM, no speculation	1	11.04	11.05	0.32%	155.95 ms	90.4 ms	vllm_latest_reference.json
vLLM, no speculation	2	21.34	21.34	0.04%	235.35 ms	93.52 ms	vllm_latest_reference.json
vLLM, no speculation	4	41.20	41.20	0.03%	419.59 ms	96.77 ms	vllm_latest_reference.json
vLLM, no speculation	8	78.18	78.18	0.02%	729.44 ms	101.71 ms	vllm_latest_reference.json
vLLM, no speculation	16	137.11	137.11	0.04%	1176.06 ms	115.56 ms	vllm_latest_reference.json
vLLM, no speculation	32	219.50	219.45	0.09%	2240.16 ms	143.58 ms	vllm_latest_reference.json
vLLM, no speculation	64	312.26	312.31	0.12%	4009.21 ms	200.87 ms	vllm_latest_reference.json
vLLM, no speculation	128	390.42	390.15	0.31%	6710.78 ms	320.84 ms	vllm_latest_reference.json

Harness bench/ladder38/harness_w55_conc_ladder.py. Two harness revisions appear above: 6412b12d4d — vLLM legs. Does not send the penalty keys.; 1c77e1d8e9 — Atlas legs (rounds 7-11). Adds explicit presence_penalty/frequency_penalty = 0.0. The only difference is those two keys. vLLM already defaults both to 0.0, so its sampling is byte-identical either way; the keys exist to stop Atlas's non_thinking preset injecting presence_penalty=1.5. Both engines therefore ran identical sampling.

Full campaign log, including every rung we lost on the way and the three claims we retracted: bench/ladder38/RESULTS.md. Generated 2026-08-26T18:04:54Z from the committed measurements.

The inference engine for the machine on your desk.

Every number is a receipt.

Faster than vLLM at every concurrency, C=1 to 128

Atlas Atlas 1.0.0-beta-preview

vLLM + MTP vLLM 0.27.1

vLLM, no speculation vLLM 0.27.1

Per-rung detail

What the gate checks

What just happened.

We helped build the MLPerf Edge Agentic benchmark

AMD sent us a Strix Halo desktop

Our MLPerf submission is in

Prosumer first. Desk machines, not clusters.

NVIDIA DGX Spark🎁

AMD Strix Halo🎁

Every model here has a recipe.

Qwen · Qwen3.6

Up and running in one command.

Prefer to inspect first?

Built in the open, starred in the open.

The action is in Discord.

Your machine is the test fleet.

Run the serve matrix

Add or tune a recipe

Kernels in Rust and CUDA

Docs, triage, ideas

What we are building next.

🎁Trifecta, three Sparks

Intel Arc Pro B70

🎁AMD Strix Halo

MLPerf Inference v6.1

Qwen GDN kernel upstream

Bigger model support

Local AI worth having, open to all.

The questions we actually get asked.

Come work with us.

Business

Partnerships

Hardware