# How to choose an MLX model

A practical v0.6.2 table for architecture, local discovery, parameter count, quantization, context, concurrency, and unified-memory headroom.

## Direct answer

Start with the task and exact architecture support, then verify tokenizer and checkpoint compatibility, including models discovered in Hugging Face cache roots. Size weights, context cache, runtime overhead, and macOS headroom; use compatible KV-cache quantization only where appropriate. If the estimate approaches physical memory, choose a smaller or more strongly weight-quantized checkpoint.

- Canonical: https://macmlx.app/models/choosing-a-model/
- Last verified: 2026-07-15

## Page facts

- **Apple Silicon unified memory** — Released; since 0.1.0; last verified 2026-07-15. MLX arrays use the Mac's shared CPU/GPU memory system. Unified memory reduces explicit transfers between CPU orchestration and integrated-GPU compute, but model weights, activations, and KV cache still consume finite physical memory.
- **Bounded model pool** — Released; since 0.5.0; last verified 2026-07-15. Budgets, LRU eviction, pinning, cold swap, idle TTL, and probes bound multi-model use. The pool shipped in v0.5.0 and was hardened in v0.5.3. It is not a unified adaptive controller.
- **Exact-prefix RAM and SSD cache** — Released; since 0.5.0; last verified 2026-07-15. A hot RAM tier and content-addressed SSD cold tier support promotion and demotion. The released v0.5.0 cache reuses exact full prefixes. It does not provide released block sharing or paged KV allocation.
- **KV-cache quantization** — Released; since 0.6.0; last verified 2026-07-15. v0.6.0 exposes kv_bits, kv_group_size, and quantized_kv_start for compatible requests. These controls change KV-cache precision, not model-weight precision. Nonpositive kv_bits disables the feature, and the compatibility matrix rejects unsupported model or request combinations.
- **Hugging Face cache discovery** — Released; since 0.6.0; last verified 2026-07-15. v0.6.0 can discover models in configured Hugging Face cache roots without downloading duplicate weights. Discovery confirms a local candidate, not a universal load guarantee; architecture, tokenizer, processor, and checkpoint compatibility still apply.
- **Temperature and top-p** — Released; since 0.1.0; last verified 2026-07-15. Temperature and nucleus top-p sampling are released controls. These are the current exposed core sampling controls.
- **Expanded sampling controls** — Planned; since future; last verified 2026-07-15. top-k, min-p, presence, frequency, and repetition penalties, plus per-request seed are planned. DeepSeek expert-routing top-k is an internal architecture operation and is unrelated to user sampling top-k.

## Sources

- https://ml-explore.github.io/mlx/build/html/usage/unified_memory.html
- https://github.com/magicnight/mac-mlx/blob/v0.6.2/MacMLXCore/Sources/MacMLXCore/Engine/MLXSwiftEngine.swift
- https://github.com/magicnight/mac-mlx/releases/tag/v0.6.2
- https://github.com/magicnight/mac-mlx/blob/v0.6.2/CHANGELOG.md
- https://github.com/magicnight/mac-mlx/blob/v0.6.2/MacMLXCore/Sources/MacMLXCore/ModelPool/ModelPool.swift
- https://github.com/magicnight/mac-mlx/blob/v0.6.2/MacMLXCore/Sources/MacMLXCore/PromptCache/PromptCacheStore.swift
- https://github.com/magicnight/mac-mlx/blob/v0.6.2/MacMLXCore/Sources/MacMLXCore/Engine/GenerateRequest.swift
- https://github.com/magicnight/mac-mlx/blob/v0.6.2/MacMLXCore/Sources/MacMLXCore/Managers/ModelLibraryManager.swift
- https://github.com/magicnight/mac-mlx/blob/v0.6.2/MacMLXCore/Sources/MacMLXCore/Managers/ModelParametersStore.swift
- https://github.com/magicnight/mac-mlx/blob/main/docs/superpowers/specs/2026-07-10-engine-scroll-story-design.md
- https://github.com/magicnight/mac-mlx/blob/main/MacMLXCore/Sources/MacMLXCore/Managers/ModelParametersStore.swift

## Related pages

- [Choose models by task and memory](https://macmlx.app/models/)
- [Vision-language model support](https://macmlx.app/models/vision-language-models/)
- [How macMLX runs models on Apple Silicon](https://macmlx.app/architecture/)