Current v0.10.0 release
macMLX v0.10.0
The current release lets structured output accept the schemas Apple's Foundation Models client sends, makes unloading a model actually return its memory, moves /v1/rerank onto MLXRerankers inside the model pool's budget, and builds the DMG from the committed lockfiles its tests ran on.
Compatibility and upgrade notes
- Compatibility
- The new surfaces state their limits: schema and usage compatibility with Apple's client is not a verified Xcode 27 or Foundation Models end-to-end connection; the XLM-RoBERTa (BGE) and Jina v3 rerank paths were not run on a real checkpoint; no test runs audio inference; null, unions, pattern, and multipleOf stay unsupported in schemas; and POST /x/models/unload with an id that is neither a resident reranker or embedder nor the active chat model is now a 404 instead of unloading the chat model.
- Upgrade
- Review the tagged v0.10.0 changelog before upgrading. The cold prompt-cache tier's format version is bumped, so cold entries written by earlier builds are reclaimed and only cache is lost; the first load of an audio model cached by v0.9.0 asks the Hub for a listing; and the copies v0.9.0 left in ~/.cache/huggingface are not removed.
Shipped
Apple Silicon macOS installation
macMLX supports Apple Silicon Macs running macOS 14 or later.
Use the current project installation and Gatekeeper guidance; do not disable system-wide security protections solely to open the app.
Verified
Swift in-process inference
The default engine loads and runs MLX models inside the Swift process.
Model loading, generation, caching, and serving use Apple MLX through MacMLXCore; the default inference path does not require a Python runtime.
Verified
Apple Silicon unified memory
MLX arrays use the Mac's shared CPU/GPU memory system.
Unified memory reduces explicit transfers between CPU orchestration and integrated-GPU compute, but model weights, activations, and KV cache still consume finite physical memory.
Verified
Shared code, process-local engines
The app and CLI both import MacMLXCore, which owns inference and the server.
The products share implementation and behavior. When the app and CLI run as separate processes, they do not share one in-memory engine instance.
Verified
No Python on the default path
The released default engine is Swift-native and needs no Python runtime.
Optional compatibility engines may use subprocesses and other runtimes. This is not a claim that Python is absent everywhere in the project.
Verified
OpenAI endpoint compatibility
Chat, legacy completions, model listing, and embeddings use compatible request and response shapes.
Compatibility is endpoint-specific. macMLX model load and unload routes under /x/models are project extensions, not OpenAI-compatible model management.
Verified
Anthropic Messages compatibility
POST /v1/messages, including streaming, is available in v0.5.3.
This is Messages API compatibility only, not compatibility with the full Anthropic API.
Verified
Selected Ollama endpoints
macMLX supports /api/version, /api/tags, /api/show, /api/chat, and /api/generate.
The compatibility layer has shipped since v0.3.7. It is a selected endpoint set, not a drop-in replacement for every Ollama API.
Verified
MCP server
The CLI can expose local inference to MCP clients.
The MCP server shipped in v0.5.0 and is separate from chat-side routing to external tools.
Verified
MCP client pool
v0.5.3 includes managed MCP client connections.
The pool manages external MCP processes and connections; integrated chat-side tool routing is a separate capability released in v0.6.0.
Verified
Integrated chat tool routing
v0.6.0 ships multi-turn tool routing for OpenAI, Anthropic, and the GUI MCP loop.
Protocol-specific validation keeps tool-call histories explicit; this routing is distinct from the MCP server and client-pool infrastructure.
Verified
Local embeddings
POST /v1/embeddings shipped in v0.5.3.
Encoder-family model detection exists, while using an unsuitable chat model can still produce vectors without semantic guarantees.
Verified
Exact-prefix RAM and SSD cache
A hot RAM tier and content-addressed SSD cold tier support promotion and demotion.
The released v0.5.0 cache reuses exact full prefixes. It does not provide released block sharing or paged KV allocation.
Verified
Trie longest-prefix reuse
v0.6.0 reuses the longest compatible cached token prefix.
Multi-turn prompts can trim to the longest common prefix and incrementally prefill only the new suffix while usage retains full-prompt accounting.
Verified
Bounded model pool
Budgets, LRU eviction, pinning, cold swap, idle TTL, and probes bound multi-model use.
The pool shipped in v0.5.0 and was hardened in v0.5.3. It is not a unified adaptive controller.
Verified
Supported LoRA adapters
The native engine can apply supported LoRA adapters.
Adapter compatibility depends on the base architecture and weights; universal LoRA compatibility is not claimed.
Verified
Fourteen detected VLM families
The model library detects 14 vision-language model_type families.
This is an evidence-backed family count, not a guarantee that every checkpoint or processor variant will load.
Verified
DeepSeek V3.2 Swift overlay
v0.5.3 includes pure-Swift component parity for the DeepSeek V3.2 architecture.
A real-checkpoint smoke test remains pending and FP8 dequantization is absent, so this is not an end-to-end or universal MoE claim.
Verified
Eligibility-gated continuous batching
v0.6.0 batches only eligible dense-text requests under real concurrency, with automatic serial fallback.
The tagged 4-client benchmark measured 2.5–3.2× aggregate throughput. VLM, speculative decoding, Ollama, Anthropic, and embeddings remain serial.
Verified
Fixed prefill admission throttle
A fixed prefillBatchSize bounds rows admitted per scheduler step.
The released throttle is fixed configuration, not the planned adaptive memory controller.
Verified
Structured output
v0.6.0 supports response_format with json_object and an explicit JSON Schema subset.
Unsupported schema keywords return 400. VLM with structured output and tools with structured output are unsupported combinations and are explicitly rejected rather than silently degraded.
Verified
Speculative decoding
v0.6.0 ships the classic draft-model path through both the API and GUI.
Acceptance telemetry reports draft efficiency. Targets with non-trimmable hybrid or linear-attention caches are detected and fall back to standard decoding.
Verified
API compatibility pack
v0.6.0 adds logit_bias, logprobs and top_logprobs, XTC, per-request LoRA adapters, and tools.
An explicit compatibility matrix governs parameter combinations, and unsupported pairs return 400 instead of silently degrading.
Verified
KV-cache quantization
v0.6.0 exposes kv_bits, kv_group_size, and quantized_kv_start for compatible requests.
These controls change KV-cache precision, not model-weight precision. Nonpositive kv_bits disables the feature, and the compatibility matrix rejects unsupported model or request combinations.
Verified
Hugging Face cache discovery
v0.6.0 can discover models in configured Hugging Face cache roots without downloading duplicate weights.
Discovery confirms a local candidate, not a universal load guarantee; architecture, tokenizer, processor, and checkpoint compatibility still apply.
Verified
Track G tested models
v0.6.2 adds four checkpoint-tested native model families.
Measured real-checkpoint generation: Seed-OSS-36B 4-bit at 18.2 tok/s; Hunyuan V1 Dense 1.8B 4-bit at 80.3 tok/s; Cohere Command R7B 7B 4-bit at 21.7 tok/s; and MiniCPM3-4B 4-bit at 18.7 tok/s. Results are checkpoint-specific, not family-wide performance guarantees.
Verified
InternLM3 theoretical support
v0.6.2 ships parity-verified InternLM3 code at the theoretical support tier.
Real generation has not been demonstrated. Public checkpoints ship tokenizer.model but no tokenizer.json, while the Swift tokenizer stack requires tokenizer.json; support remains theoretical until that load-path boundary changes.
Verified
Temperature and top-p
Temperature and nucleus top-p sampling are released controls.
These are the current exposed core sampling controls.
Verified
Silicon Activity panel
A new main-window Activity tab shows live Apple Silicon readouts, a prefill/decode throughput split, and the current inference bottleneck with advice.
The panel reads GPU occupancy, memory bandwidth, thermal and memory pressure, and per-rail power without admin rights or a helper process. It is observability, not a performance guarantee: an unavailable counter renders an em dash with its reason, an idle engine shows no active generation, and estimated values are labeled rather than presented as measured.
Verified
Inference bottleneck classifier
v0.7.0 fuses the hardware samples with the engine's live prefill or decode phase to attribute what limits generation.
Because the classifier runs in-process it knows which inference phase is active, the signal an external GPU monitor cannot see. It is a smoothed heuristic, not a profiler: it prioritizes memory over thermal over compute or bandwidth, applies hysteresis and three-frame smoothing, and self-calibrates its bandwidth ceiling, so a verdict is guidance rather than a ground-truth measurement.
Verified
Sudoless silicon sampling
A runtime IOReport bridge plus samplers read GPU occupancy, memory bandwidth, thermal and memory pressure, and ANE power.
The IOReport bridge is resolved through dlopen, so a future macOS that renames or removes the private framework degrades the readouts to unavailable rather than failing to launch. Memory bandwidth is reported as an estimate, ANE exposes a power proxy with no fabricated utilization percentage, and the Media Engine, which has no duty-cycle signal, is omitted rather than guessed.
Verified
Benchmark bottleneck attribution
A benchmark run samples the silicon while generating and attaches what limited its decode steady state to the saved result.
The attribution names the decode limiter, whether memory, thermal, bandwidth, or compute, with a confidence and the representative hardware behind it. A run too short to attribute is reported as unavailable instead of inventing a verdict, and low-confidence or estimate-based calls are flagged; it describes one measured run, not a family-wide benchmark guarantee.
Verified
OCR model recognition
Dedicated OCR checkpoints receive an OCR badge distinct from the generic Vision badge, and GLM-OCR is verified end-to-end.
GLM-OCR loads through the stock VLM path with no new model code and reads image text back. The badge is intentionally narrow: it appears only on a model that actually loads, so an OCR family that is detected but not yet ported, such as dots_ocr or deepseek-ocr, earns no badge. This is model recognition, not a universal OCR-quality claim.
Verified
Byte-bounded cold KV tier
v0.8.0 bounds the on-disk cold KV cache to an explicit byte budget, pruned oldest-first, with a toggle to opt the cold tier out.
The previously unbounded cold directory now honors the Cold (SSD) budget with the Hot (RAM) budget wired alongside; an over-budget directory is trimmed on first launch, removing only regenerable cache rather than model files or settings. It does not add block sharing or paged allocation; the released cache still reuses exact full prefixes only.
Verified
Weight-identity-guarded cold entries
v0.8.0 fingerprints each cold entry against model weight identity so a weight swap cannot serve stale KV.
The fingerprint covers the model's config plus every safetensors shard; if the same path later holds re-downloaded, re-quantized, or swapped weights, the stale entry is rejected and deleted rather than restored into silently wrong output. It is an integrity guard for the cold tier's reuse, not a correctness proof for every cache path, and it does not change the released exact-prefix reuse semantics.
Verified
Persistent cross-session cold index
v0.8.0 persists a cold index so longest-prefix reuse survives a restart across sessions.
A follow-up turn that extends an earlier session's prompt reuses the shared prefix straight off disk, while a missing, unreadable, or version-mismatched index degrades to exact re-hits, never to wrong output. Reuse remains exact-prefix, not the planned paged or block-sharing virtualization; the persisted index only extends the existing longest-prefix reuse across process restarts.
Verified
Off-actor cold-tier writer
v0.8.0 moves cold-tier writes off the cache actor onto a dedicated serial background writer.
Persisting an evicted KV snapshot, hundreds of milliseconds for a large cache, no longer blocks another request's cache lookup; the write lands atomically through a temp file and rename, and the on-disk result is unchanged. It relocates disk writes to keep the store actor responsive and does not alter what is cached or the released reuse semantics.
Verified
In-process speech to text and text to speech
POST /v1/audio/transcriptions and POST /v1/audio/speech run on MLX inside the same Swift process.
The audio engine is mlx-audio-swift, so the default path still needs no Python runtime and no Core ML detour, and downloads land under ~/.mac-mlx/audio-models. A request for an output container macMLX cannot produce is refused rather than served as a different format. These paths are code-complete and unit-tested; they have not been validated against real audio checkpoints, so no transcription or synthesis quality is claimed.
Verified
Audio inside the app
Transcription is a cancellable chat attachment, assistant messages gained a speech button, and the model library lists audio checkpoints.
Transcribed text lands in the composer so it can be edited before sending, and starting one playback stops the other. The library needs its own scan because mlx-audio writes a flat layout and often ships no tokenizer.json, so the usual format detection cannot classify it; a checkpoint in neither the speech-to-text nor the text-to-speech registry is deliberately left unlisted rather than guessed at from its folder name. This surface has not been validated against real audio checkpoints.
Verified
Detected multi-token-prediction drafters
A draft model can be identified as an MTP block drafter from its config.json alone, before any weights load.
Registration and detection ship; the engine is deliberately not wired to use them for decoding. This is not released multi-token-prediction decoding, and it changes no generation path.
Verified
The app links the controlled MLX build
v0.9.0 is the first DMG whose GUI actually links macMLX's patched mlx-swift instead of upstream.
The Xcode project declared no mlx-swift dependency of its own, so on a package-identity collision the upstream copy pulled in transitively won and the app linked it; every earlier release therefore shipped a GUI without the correctness cherry-picks that MacMLXCore's own tests had verified. The CLI carried the same defect from a different root and was fixed alongside. This restores the intended engine rather than changing inference behavior beyond the fixes it now carries.
Verified
Foundation-Models-shaped JSON schemas
v0.10.0 structured output accepts the schema shapes Apple's Foundation Models client sends for @Generable types: nested objects, arrays with minItems and maxItems, $ref to the root's $defs or definitions, a string const, any type at the root, keys and literals in any Unicode string, and minimum or maximum bounds on numbers and integers.
A schema may nest at most 32 containers deep and hold at most 4,096 nodes after $ref expansion; a recursive schema is a 400, and null, unions, pattern, and multipleOf remain unsupported. A constrained request on a thinking model renders with enable_thinking off and is never reported as reasoning. Apple's TripPlanner and Person samples compile as the framework emits them, and Apple's client was driven against a live Qwen3.6 checkpoint for structured output, but no end-to-end Xcode 27 or Foundation Models connection is claimed as verified: this is schema and wire compatibility, not a verified integration.
Verified
Streaming usage, max_completion_tokens, and cached tokens
v0.10.0 honors stream_options.include_usage with one usage-only chunk before [DONE], accepts max_completion_tokens as the newer spelling of max_tokens, and reports how much of the prompt the prompt cache served.
Cached tokens appear as prompt_tokens_details.cached_tokens on OpenAI-shaped responses and as cache_read_input_tokens on /v1/messages, where input_tokens becomes the uncached remainder. The figure is 0 on a miss and on paths that never consult the cache, and it is withheld from requests that carry an Origin header because it is an exact prefix oracle. include_usage is ignored on a non-streaming request, whose body already carries usage. This covers these usage fields only, not every OpenAI streaming option.
Verified
Unloading returns memory
v0.10.0 drains MLX's buffer cache once a chat model is released, so an unload, an idle sweep, or a pool eviction gives the model's memory back instead of leaving it cached.
Before this release, unloading Mellum2-12B-A2.5B-Thinking-4bit left 7,021 MB of the 7,035 MB the model had used in MLX's cache; after the change that cache was empty and 15 MB stayed active, nearly all of it the in-memory prompt-cache tier, which an unload does not release (up to 512 MiB by default). That is one measured checkpoint, not a guarantee for every model. A cancel during speculative decoding can leave the weights for a later drain, and a first-ever load that fails after allocating is left to the next plain generation's clear.
Verified
Rerankers and embedders inside the pool budget
In the app, the server admits the /v1/rerank and /v1/embeddings models to the model pool before loading them, so they count against its budget, are evicted least-recently-used, and expire on the model's TTL like chat models.
A swap now releases the resident reranker or embedder and drains MLX's cache before loading the replacement, under the generation lock. POST /x/models/unload takes a model id and unloads a resident reranker or embedder by that exact id; an id that is neither one of them nor the active chat model is now a 404 instead of unloading the chat model regardless. The CLI's server has no pool and still loads them directly, and the Models tab no longer offers to pin them. /x/models remains a macMLX extension, not OpenAI-compatible model management.
Verified
Rerank on MLXRerankers
v0.10.0 serves POST /v1/rerank through mlx-swift-lm's MLXRerankers: BERT, RoBERTa, and XLM-RoBERTa cross-encoders, Qwen3 rerankers scored by their yes/no logit margin, and Jina reranker v3.
What ran on real checkpoints: ms-marco-MiniLM-L-6-v2 matches the PyTorch fp32 reference to 2e-6 on the logit scale with TF32 off, and to 0.025 under the TF32 matmuls M5 enables by default; Qwen3-Reranker-0.6B-4bit ranks the documents that answer the question above the ones that do not. The XLM-RoBERTa path (BGE) and Jina v3 were not run on a real checkpoint, and Jina v3 reports a cosine similarity rather than a probability. Classification heads qualify more narrowly than upstream's own rule, so sentiment heads stay plain models, and bi-encoder cosine stays the fallback for checkpoints that are not rerankers.
Verified
Native checkpoint chat templates
v0.10.0 renders every checkpoint's own chat template through swift-jinja 2.5.1 and removes the two built-in overrides; a per-model macmlx.chat_template.jinja file still takes precedence.
Prompts change in places users may notice: tojson now emits Python json.dumps-style output in tool descriptions, templates with {#- and -#} comments render their whitespace as Jinja does, and a Hunyuan history turn with answer tags is stripped as Python strips it. Command R7B now renders its tool_use template when a request carries tools, but macMLX has no parser for the action markup it asks for, so tool calling on that model is not supported. Template support is still not universal.
Verified
Controlled MLX fork on mlx-swift 0.32.3
v0.10.0 rebases the controlled MLX fork on mlx-swift 0.32.3 (core v0.32.2), which already contains twelve of the thirteen fixes the fork used to carry, so the fork now carries five of its own: mlx#3922, mlx#4009, mlx#4660, mlx#4431 and mlx#4775.
mlx#4009 repairs a sorted quantized MoE product that corrupted 92–97% of its output on M5 hardware at group size 32 with a ragged K. mlx#4660 stops the channel padding of a convolution, the 3-channel patch embedding of the Qwen-VL family and GLM-OCR among them, from picking up a stray value under asynchronous evaluation; mlx#4775 corrects the scale of all-negative groups in a quantized KV cache; mlx#4431 fixes a batched long-context attention kernel that no macMLX path selects today. Each was reproduced here with a synthetic test before it was carried, none through a real checkpoint. The new base also fixes the quantized MoE gather kernel names under JIT (mlx#4372) and compiles kernels under the Metal 4.1 language version macOS 27 requests. mlx-swift-lm moves to 3.32.3 with prefill chunking pinned to the legacy .remainder stride, checked by reading rather than by an output comparison, and Gemma3Text's separately reworked prefill is not covered. Carrying a fix is a correctness repair, not a performance claim.
Verified
Audio downloads stay in the app's cache
Through a controlled fork of mlx-audio-swift (upstream v0.1.5 plus cache and listing fixes), the speech families that used to fetch a second copy into ~/.cache/huggingface now load from ~/.mac-mlx/audio-models, and a cached snapshot is completed against the Hub's full listing.
Still fetched into the default Hugging Face cache: the tokenizer or codec of a model loaded from a local directory rather than by repo id, Kokoro's and Kitten's grapheme-to-phoneme model, and a non-English Kokoro voice's byT5 model or lexicon. Copies v0.9.0 left there are not removed, and a snapshot the Hub's listing has not yet confirmed cannot be completed offline. Audio loads now run under the generation lock, so a fetch a loader makes on its own can hold other requests. No test here runs audio inference; only the gated cache smokes for moonshine-tiny, VyvoTTS with its SNAC codec, and SenseVoiceSmall passed, so no transcription or synthesis quality is claimed.
Verified
The DMG ships the tested dependency graph
v0.10.0 commits the app's workspace lockfile, builds the release archive, the CLI tarball, and the app's CI build only from the versions their lockfiles record, and fails CI when the app's or the CLI's lockfile pins a MacMLXCore package differently.
The v0.9.0 archive resolved on a fresh runner and shipped swift-jinja 2.4.2 while every chat-template test had run on 2.3.6. This pins dependency versions to what the tests ran on; it is not a reproducible-build or code-signing claim, and the DMG remains unsigned.
Verified
Current limitations
Eligibility-gated continuous batching
v0.6.0 batches only eligible dense-text requests under real concurrency, with automatic serial fallback.
The tagged 4-client benchmark measured 2.5–3.2× aggregate throughput. VLM, speculative decoding, Ollama, Anthropic, and embeddings remain serial.
Verified
Structured output
v0.6.0 supports response_format with json_object and an explicit JSON Schema subset.
Unsupported schema keywords return 400. VLM with structured output and tools with structured output are unsupported combinations and are explicitly rejected rather than silently degraded.
Verified
InternLM3 theoretical support
v0.6.2 ships parity-verified InternLM3 code at the theoretical support tier.
Real generation has not been demonstrated. Public checkpoints ship tokenizer.model but no tokenizer.json, while the Swift tokenizer stack requires tokenizer.json; support remains theoretical until that load-path boundary changes.
Verified
DeepSeek V3.2 Swift overlay
v0.5.3 includes pure-Swift component parity for the DeepSeek V3.2 architecture.
A real-checkpoint smoke test remains pending and FP8 dequantization is absent, so this is not an end-to-end or universal MoE claim.
Verified
Sudoless silicon sampling
A runtime IOReport bridge plus samplers read GPU occupancy, memory bandwidth, thermal and memory pressure, and ANE power.
The IOReport bridge is resolved through dlopen, so a future macOS that renames or removes the private framework degrades the readouts to unavailable rather than failing to launch. Memory bandwidth is reported as an estimate, ANE exposes a power proxy with no fabricated utilization percentage, and the Media Engine, which has no duty-cycle signal, is omitted rather than guessed.
Verified
In-process speech to text and text to speech
POST /v1/audio/transcriptions and POST /v1/audio/speech run on MLX inside the same Swift process.
The audio engine is mlx-audio-swift, so the default path still needs no Python runtime and no Core ML detour, and downloads land under ~/.mac-mlx/audio-models. A request for an output container macMLX cannot produce is refused rather than served as a different format. These paths are code-complete and unit-tested; they have not been validated against real audio checkpoints, so no transcription or synthesis quality is claimed.
Verified
Audio inside the app
Transcription is a cancellable chat attachment, assistant messages gained a speech button, and the model library lists audio checkpoints.
Transcribed text lands in the composer so it can be edited before sending, and starting one playback stops the other. The library needs its own scan because mlx-audio writes a flat layout and often ships no tokenizer.json, so the usual format detection cannot classify it; a checkpoint in neither the speech-to-text nor the text-to-speech registry is deliberately left unlisted rather than guessed at from its folder name. This surface has not been validated against real audio checkpoints.
Verified
Detected multi-token-prediction drafters
A draft model can be identified as an MTP block drafter from its config.json alone, before any weights load.
Registration and detection ship; the engine is deliberately not wired to use them for decoding. This is not released multi-token-prediction decoding, and it changes no generation path.
Verified
Foundation-Models-shaped JSON schemas
v0.10.0 structured output accepts the schema shapes Apple's Foundation Models client sends for @Generable types: nested objects, arrays with minItems and maxItems, $ref to the root's $defs or definitions, a string const, any type at the root, keys and literals in any Unicode string, and minimum or maximum bounds on numbers and integers.
A schema may nest at most 32 containers deep and hold at most 4,096 nodes after $ref expansion; a recursive schema is a 400, and null, unions, pattern, and multipleOf remain unsupported. A constrained request on a thinking model renders with enable_thinking off and is never reported as reasoning. Apple's TripPlanner and Person samples compile as the framework emits them, and Apple's client was driven against a live Qwen3.6 checkpoint for structured output, but no end-to-end Xcode 27 or Foundation Models connection is claimed as verified: this is schema and wire compatibility, not a verified integration.
Verified
Rerank on MLXRerankers
v0.10.0 serves POST /v1/rerank through mlx-swift-lm's MLXRerankers: BERT, RoBERTa, and XLM-RoBERTa cross-encoders, Qwen3 rerankers scored by their yes/no logit margin, and Jina reranker v3.
What ran on real checkpoints: ms-marco-MiniLM-L-6-v2 matches the PyTorch fp32 reference to 2e-6 on the logit scale with TF32 off, and to 0.025 under the TF32 matmuls M5 enables by default; Qwen3-Reranker-0.6B-4bit ranks the documents that answer the question above the ones that do not. The XLM-RoBERTa path (BGE) and Jina v3 were not run on a real checkpoint, and Jina v3 reports a cosine similarity rather than a probability. Classification heads qualify more narrowly than upstream's own rule, so sentiment heads stay plain models, and bi-encoder cosine stays the fallback for checkpoints that are not rerankers.
Verified
Native checkpoint chat templates
v0.10.0 renders every checkpoint's own chat template through swift-jinja 2.5.1 and removes the two built-in overrides; a per-model macmlx.chat_template.jinja file still takes precedence.
Prompts change in places users may notice: tojson now emits Python json.dumps-style output in tool descriptions, templates with {#- and -#} comments render their whitespace as Jinja does, and a Hunyuan history turn with answer tags is stripped as Python strips it. Command R7B now renders its tool_use template when a request carries tools, but macMLX has no parser for the action markup it asks for, so tool calling on that model is not supported. Template support is still not universal.
Verified
Audio downloads stay in the app's cache
Through a controlled fork of mlx-audio-swift (upstream v0.1.5 plus cache and listing fixes), the speech families that used to fetch a second copy into ~/.cache/huggingface now load from ~/.mac-mlx/audio-models, and a cached snapshot is completed against the Hub's full listing.
Still fetched into the default Hugging Face cache: the tokenizer or codec of a model loaded from a local directory rather than by repo id, Kokoro's and Kitten's grapheme-to-phoneme model, and a non-English Kokoro voice's byT5 model or lexicon. Copies v0.9.0 left there are not removed, and a snapshot the Hub's listing has not yet confirmed cannot be completed offline. Audio loads now run under the generation lock, so a fetch a loader makes on its own can hold other requests. No test here runs audio inference; only the gated cache smokes for moonshine-tiny, VyvoTTS with its SNAC codec, and SenseVoiceSmall passed, so no transcription or synthesis quality is claimed.
Verified
Planned
Paged KV, block sharing, and CoW
Paged allocation, shared blocks, and copy-on-write branching are planned.
None of these cache-virtualization features is released; the shipped cache reuses exact token prefixes only, trimmed to the longest common prefix, without shared blocks.
Verified
Unified adaptive memory guard
A feedback controller across cache, model pool, and concurrency is planned.
Released memory probes and pool caps are separate mechanisms and must not be described as this guard.
Verified
Expanded sampling controls
top-k, min-p, presence, frequency, and repetition penalties, plus per-request seed are planned.
DeepSeek expert-routing top-k is an internal architecture operation and is unrelated to user sampling top-k.
Verified
Verified Xcode 27 and Foundation Models connection
An end-to-end check that Xcode 27's local-model provider and Apple's Foundation Models client drive a running macMLX server is planned.
The released schema and usage compatibility does not amount to this check: no end-to-end connection through Xcode 27 or a Foundation Models app has been verified, and no generation quality through either path is claimed.
Verified
Official sources
- macMLX v0.10.0 · github.com
- Apple Silicon macOS installation
- Unloading returns memory
- Apple Silicon unified memory
- Rerank on MLXRerankers
- macMLX v0.10.0 · github.com
- MCP client pool
- Integrated chat tool routing
- Local embeddings
- Weight-identity-guarded cold entries
- Trie longest-prefix reuse
- Rerankers and embedders inside the pool budget
- Track G tested models
- DeepSeek V3.2 Swift overlay
- Fixed prefill admission throttle
- Eligibility-gated continuous batching
- Foundation-Models-shaped JSON schemas
- Speculative decoding
- KV-cache quantization
- macMLX v0.10.0 · github.com
- InternLM3 theoretical support
- Temperature and top-p
- Silicon Activity panel
- Silicon Activity panel
- Inference bottleneck classifier
- Inference bottleneck classifier
- Sudoless silicon sampling
- Sudoless silicon sampling
- Benchmark bottleneck attribution
- Benchmark bottleneck attribution
- OCR model recognition
- Byte-bounded cold KV tier
- Weight-identity-guarded cold entries
- Persistent cross-session cold index
- Off-actor cold-tier writer
- Audio downloads stay in the app's cache
- In-process speech to text and text to speech
- Audio inside the app
- Audio inside the app
- Detected multi-token-prediction drafters
- The app links the controlled MLX build
- Audio downloads stay in the app's cache
- Foundation-Models-shaped JSON schemas
- Foundation-Models-shaped JSON schemas
- Rerank on MLXRerankers
- Native checkpoint chat templates
- The DMG ships the tested dependency graph
- The DMG ships the tested dependency graph
- Expanded sampling controls
- Expanded sampling controls
- Verified Xcode 27 and Foundation Models connection