Compatibility facts
OpenAI endpoint compatibility
Chat, legacy completions, model listing, and embeddings use compatible request and response shapes.
Compatibility is endpoint-specific. macMLX model load and unload routes under /x/models are project extensions, not OpenAI-compatible model management.
Verified
Anthropic Messages compatibility
POST /v1/messages, including streaming, is available in v0.5.3.
This is Messages API compatibility only, not compatibility with the full Anthropic API.
Verified
Selected Ollama endpoints
macMLX supports /api/version, /api/tags, /api/show, /api/chat, and /api/generate.
The compatibility layer has shipped since v0.3.7. It is a selected endpoint set, not a drop-in replacement for every Ollama API.
Verified
MCP server
The CLI can expose local inference to MCP clients.
The MCP server shipped in v0.5.0 and is separate from chat-side routing to external tools.
Verified
MCP client pool
v0.5.3 includes managed MCP client connections.
The pool manages external MCP processes and connections; integrated chat-side tool routing is a separate capability released in v0.6.0.
Verified
Integrated chat tool routing
v0.6.0 ships multi-turn tool routing for OpenAI, Anthropic, and the GUI MCP loop.
Protocol-specific validation keeps tool-call histories explicit; this routing is distinct from the MCP server and client-pool infrastructure.
Verified
Local embeddings
POST /v1/embeddings shipped in v0.5.3.
Encoder-family model detection exists, while using an unsuitable chat model can still produce vectors without semantic guarantees.
Verified
Bi-encoder rerank MVP
POST /v1/rerank scores independently embedded texts with cosine similarity.
This released MVP is not a cross-encoder reranker.
Verified
Eligibility-gated continuous batching
v0.6.0 batches only eligible dense-text requests under real concurrency, with automatic serial fallback.
The tagged 4-client benchmark measured 2.5–3.2× aggregate throughput. VLM, speculative decoding, Ollama, Anthropic, and embeddings remain serial.
Verified
Structured output
v0.6.0 supports response_format with json_object and an explicit JSON Schema subset.
Unsupported schema keywords return 400. VLM with structured output and tools with structured output are unsupported combinations and are explicitly rejected rather than silently degraded.
Verified
API compatibility pack
v0.6.0 adds logit_bias, logprobs and top_logprobs, XTC, per-request LoRA adapters, and tools.
An explicit compatibility matrix governs parameter combinations, and unsupported pairs return 400 instead of silently degrading.
Verified
KV-cache quantization
v0.6.0 exposes kv_bits, kv_group_size, and quantized_kv_start for compatible requests.
These controls change KV-cache precision, not model-weight precision. Nonpositive kv_bits disables the feature, and the compatibility matrix rejects unsupported model or request combinations.
Verified
Endpoint compatibility matrix
| Surface | Endpoints | Compatibility boundary |
|---|---|---|
| OpenAI chat | POST /v1/chat/completions | Compatible; streaming supported |
| OpenAI legacy completions | POST /v1/completions | Compatible legacy completion shape |
| OpenAI model listing | GET /v1/models | Compatible listing; load/unload are macMLX extensions under /x/models |
| OpenAI embeddings | POST /v1/embeddings | Compatible embeddings shape; model suitability still matters |
| Anthropic | POST /v1/messages | Messages API only, including streaming; not the full Anthropic API |
| Ollama | /api/version, /api/tags, /api/show, /api/chat, /api/generate | Selected endpoints since v0.3.7; not a drop-in replacement |
| Rerank | POST /v1/rerank | macMLX bi-encoder cosine MVP; not a cross-encoder |
| Tool loops | OpenAI, Anthropic, and GUI MCP routes | Multi-turn tool routing released in v0.6.0 |
| Structured output | response_format: json_object or supported JSON Schema | Unsupported schemas and tool/VLM combinations return 400 |
| API compatibility pack | logit_bias, logprobs, top_logprobs, XTC, per-request LoRA, tools | Unsupported parameter combinations return 400 |
| KV-cache quantization | kv_bits, kv_group_size, quantized_kv_start | Compatible requests only; this does not quantize model weights |
OpenAI-compatible chat example
Call the local server with an explicit model and message.
curl http://localhost:8000/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{"model":"your-mlx-model","messages":[{"role":"user","content":"Hello"}]}'