// self-hosted inference, zero cloud

Run frontier models
on your own hardware.

AI Lab Home is an open toolkit for building a private LLM lab — reproducible benchmarks across consumer GPUs and Apple Silicon, one-command installers, and battle-tested configs for local inference.

↓ Download Toolkit View Benchmarks

42+

models profiled

hardware configs

100%

runs locally

MIT

licensed

Reproducible results

Inference benchmarks

Single-request decode throughput, measured with our harness at 2k context. Numbers are medians over 5 runs. Higher tokens/sec is better.

Model	Params	Quant	Hardware	Tokens/sec	VRAM
Llama 3.1 Instruct	8B	Q4_K_M	RTX 4090	138.4	6.1 GB
Llama 3.1 Instruct	8B	Q4_K_M	RTX 3090	96.7	6.1 GB
Qwen2.5 Instruct	14B	Q4_K_M	RTX 4090	71.2	9.8 GB
Qwen2.5 Instruct	32B	Q4_K_M	2× RTX 3090	38.9	20.4 GB
Mixtral 8x7B	47B	Q4_K_M	2× RTX 3090	44.1	26.7 GB
DeepSeek-Coder V2 Lite	16B	Q5_K_M	RTX 4090	62.5	12.3 GB
Llama 3.1 Instruct	8B	Q4_K_M	M2 Max (38c)	54.3	unified
Qwen2.5 Instruct	72B	Q4_K_M	M3 Ultra 512GB	17.8	unified

⚠ Throughput depends on driver, batch size and context length. Re-run ailab bench on your box for numbers that match your setup.

Get started

Downloads

Grab the toolkit for your platform, or pull the raw benchmark presets and model manifests.

Toolkit · Linux

CLI installer, CUDA/ROCm autodetect, benchmark harness and a local web console.

v0.9.4 · x86_64 · 18 MB

Download .tar.gz

Toolkit · macOS

Universal build with Metal acceleration for Apple Silicon and Intel Macs.

v0.9.4 · universal · 21 MB

Download .dmg

Model Presets

Ready-to-run quantization presets and sampling configs for 42 tested models.

JSON · updated weekly · 340 KB

Browse presets

In under 5 minutes

Quick start

From a clean machine to your first local completion. Linux/macOS shown; Windows users can run the same steps under WSL2.

Install the toolkit

# one-line installer (asks before touching your system)
curl -fsSL https://ai-lab-home.tech/install.sh | sh

Pull a model

ailab pull llama3.1:8b-instruct-q4_K_M

Run your first prompt

ailab run llama3.1:8b \
  --prompt "Explain KV-cache in one paragraph."

Benchmark your hardware

ailab bench --models llama3.1:8b,qwen2.5:14b \
  --context 2048 --runs 5 --report ./results.json

Good to know

FAQ

What hardware do I need?

Any GPU with 8 GB+ VRAM runs 7–8B models comfortably. 24 GB cards handle 14B; dual 3090s cover 32B–47B MoE models.

Does anything leave my machine?

No. Weights, prompts and results stay local. The toolkit only talks to model registries when you explicitly pull.

Which backends are supported?

llama.cpp, vLLM and MLX out of the box, with CUDA, ROCm and Metal acceleration auto-detected at install time.

Run frontier modelson your own hardware.