Download

API Documentation

Complete endpoint reference for the local AI server.

HTTP API

Overview

ai.local exposes two HTTP API surfaces:

  • OpenAI-compatible endpoints under /v1/*
  • Ollama-compatible endpoints under /api/*

Examples in this document use http://127.0.0.1:8080. Replace the host and port with the address used by your ai.local server.

Conventions

  • Request bodies are JSON unless noted otherwise.
  • Successful JSON endpoints return Content-Type: application/json.
  • POST /v1/audio/transcriptions requires multipart/form-data.
  • POST /v1/audio/speech returns binary audio, not JSON.
  • For OpenAI routes, use model IDs returned by GET /v1/models, for example llama-3.2-1b-instruct-4bit.
  • OpenAI text-generation routes resolve installed models as mlx-community/<model>. Registered local models use custom/<name>. apple-foundation-model is handled separately when Apple Foundation Models are available on the device.
  • Ollama routes accept installed model names with or without the mlx-community/ prefix. Registered local models use custom/<name>.
  • Text-generation routes strip <think> and <thinking> blocks from model output by default.
  • Thinking controls (reasoning_effort on /v1/chat/completions, reasoning.effort on /v1/responses, think on Ollama routes) pass enable_thinking to the model’s chat template. When a request omits them, or sends an unrecognized value, the model’s template default is used and output is unchanged. Models whose chat template does not read enable_thinking ignore the flag. apple-foundation-model ignores these controls.
  • When a request explicitly enables thinking, <think> blocks are always removed from the visible text. The reasoning is returned separately on /v1/responses as a reasoning output item, and on Ollama routes as thinking. /v1/chat/completions does not return reasoning text.
  • Usage counts on MLX models are the tokenizer’s real prompt and completion token counts. apple-foundation-model, and requests whose generation failed, fall back to character-based estimates.
  • MLX models keep the evaluated prompt (the KV cache) of the last request in memory. A request whose prompt starts with the same tokens, typically the same system prompt, only evaluates the tokens after the shared prefix. The reused token count is reported as usage.prompt_tokens_details.cached_tokens (usage.input_tokens_details.cached_tokens on /v1/responses). The cache holds one prompt for one model, is skipped for image input, and is released on a memory warning, when the model is evicted, or when an Ollama request sends keep_alive: 0.
  • Custom stop sequences (stop, or options.stop on Ollama routes) are matched on the generated text, including text that spans several tokens. Generation ends at the first match and the stop sequence itself is not returned. They apply to MLX models only.
  • Token limits (max_tokens, max_completion_tokens, max_output_tokens, options.num_predict) cap the generated tokens. Without one, or with a non-positive value, the limit is 4096 tokens. When the limit ends a generation, finish_reason (done_reason on Ollama routes) is length.
  • The server reserves /, /status, /v1/*, and /api/* for built-in routes.

Health and device control

HEAD /

Health probe. Returns 200 OK with no body.

GET /status

Returns basic server status.

{
  "status": "Running",
  "message": "Server is currently running."
}

POST /api/services/screen/brightness

Sets device brightness. brightness is clamped to 0...1.

Request:

{
  "brightness": 0.5
}

Response:

{
  "requestedBrightness": 0.5,
  "appliedBrightness": 0.5
}

GET /api/services/screen/brightness

Returns the current screen brightness.

{
  "brightness": 0.5
}

OpenAI-compatible API

GET /v1/models

Lists installed models, runtime-registered custom/<name> models, plus apple-foundation-model when the device reports Foundation Models availability.

{
  "object": "list",
  "data": [
    {
      "id": "llama-3.2-1b-instruct-4bit",
      "object": "model",
      "created": 0,
      "owned_by": "local-ai-server"
    }
  ]
}

Notes:

  • Returned model IDs are lowercased and stripped of the mlx-community/ prefix.
  • Runtime-registered local directories keep the custom/ prefix in the returned id.
  • The response uses the standard OpenAI list envelope and standard model fields only.
  • owned_by is apple for apple-foundation-model and local-ai-server for installed and custom/* models.
  • When apple-foundation-model is requested on OpenAI text routes, the server uses Apple Foundation Models instead of MLX.
  • apple-foundation-model currently supports plain-text generation only on /v1/*. Image input, tool calling, and tool-call history are rejected for that model.
  • custom/<name> model IDs are reserved for runtime-registered local directories. Requesting an unknown custom/<name> model returns an OpenAI-style 400 error with param == "model".

POST /v1/completions

Single-prompt text completion.

Request:

{
  "model": "llama-3.2-1b-instruct-4bit",
  "prompt": "Write one short sentence about local inference.",
  "temperature": 0.7,
  "n": 1
}

Response shape:

{
  "id": "UUID",
  "object": "text_completion",
  "created": 1741700000,
  "model": "llama-3.2-1b-instruct-4bit",
  "choices": [
    {
      "text": "Local inference keeps the model on your device.",
      "index": 0,
      "logprobs": null,
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 11,
    "completion_tokens": 12,
    "total_tokens": 23
  }
}

Current behavior:

  • Used: model, prompt, temperature, n, max_tokens, stop
  • stop is a string or an array of up to 4 strings. More than 4 returns 400.
  • finish_reason is length when max_tokens ended the generation, otherwise stop.
  • Non-streaming responses from MLX models include a timings object next to usage. It is an extension to the OpenAI schema and uses llama.cpp server field names: prompt_n (prompt tokens evaluated), cache_n (prompt tokens served from the prompt cache), prompt_ms, prompt_per_second, predicted_n, predicted_ms, predicted_per_second, load_ms, and total_ms.
  • Accepted but not enforced: top_p
  • Supported: stream=true returns text/event-stream with OpenAI-style text_completion chunks and a final [DONE]
  • If generation fails after the SSE stream has started, the server emits event: error with the standard OpenAI error envelope before [DONE]
  • n duplicates the same generated text across choices instead of sampling independent completions
  • model=apple-foundation-model runs through Apple Foundation Models when they are available on the device

POST /v1/chat/completions

OpenAI Chat Completions endpoint.

Basic request:

{
  "model": "llama-3.2-1b-instruct-4bit",
  "messages": [
    { "role": "system", "content": "Reply in one sentence." },
    { "role": "user", "content": "What is edge inference?" }
  ],
  "temperature": 0.7
}

Multimodal request shape also accepted:

{
  "model": "llama-3.2-11b-vision-instruct-4bit",
  "messages": [
    {
      "role": "user",
      "content": [
        { "type": "text", "text": "Describe this image." },
        {
          "type": "image_url",
          "image_url": {
            "url": "https://example.com/cat.jpg"
          }
        }
      ]
    }
  ]
}

Current behavior:

  • Used: model, messages, temperature, max_tokens, max_completion_tokens, stop, tools, tool_choice, parallel_tool_calls, reasoning_effort
  • max_completion_tokens wins when both it and max_tokens are sent.
  • stop is a string or an array of up to 4 strings. More than 4 returns 400.
  • finish_reason is tool_calls for a tool-call turn, length when the token limit ended the generation, otherwise stop.
  • Non-streaming responses from MLX models include a timings object next to usage. It is an extension to the OpenAI schema and uses llama.cpp server field names: prompt_n (prompt tokens evaluated), cache_n (prompt tokens served from the prompt cache), prompt_ms, prompt_per_second, predicted_n, predicted_ms, predicted_per_second, load_ms, and total_ms.
  • reasoning_effort: none or minimal disables model thinking. low, medium, high, or xhigh enables it. Effort levels are not distinguished. Unrecognized values are ignored instead of rejected.
  • Reasoning text is never returned on this route, because the OpenAI Chat Completions shape has no field for it. usage.completion_tokens_details.reasoning_tokens stays 0.
  • Accepted but not enforced: top_p, n
  • Supported: stream=true returns text/event-stream with chat.completion.chunk payloads and a final [DONE]
  • If generation fails after the SSE stream has started, the server emits event: error with the standard OpenAI error envelope before [DONE]
  • Rejected: legacy functions, legacy function_call, and non-standard chat request fields on /v1
  • Response always contains a single choice
  • Only POST /v1/chat/completions is registered. /v1/chat is not available.
  • Only standard OpenAI chat content parts are documented on this route. For image input, send an image_url object with a url string.
  • Tool calling follows the standard OpenAI client-side flow. The server never executes tools. When the model calls a tool, the response carries message.tool_calls and finish_reason: "tool_calls". The client runs the tool and sends the result back as a role: "tool" message with the matching tool_call_id.
  • Only tools of type function are supported. Function names must match ^[A-Za-z0-9_-]{1,64}$, and parameters must be a JSON object schema. Full JSON Schema enforcement is not implemented. strict is accepted but ignored.
  • Tools are rendered through the model’s own chat template, and tool calls are parsed from the model’s native tool-call markup. A reply that is only a JSON object such as {"name": "...", "arguments": {...}} or {"tool_calls": [...]} is also accepted as a tool call. Whether a model calls tools reliably depends on the model.
  • Tool calls that name a function that is not in tools are dropped.
  • tool_choice: auto is the default when tools is present. none hides the tools from the model. required and {"type": "function", "function": {"name": "..."}} are passed to the model as an instruction. If the model then produces no matching tool call, the request fails with a 500 server_error.
  • parallel_tool_calls defaults to true. With false, only the first tool call is returned.
  • content is null when the model returns only tool calls. Text the model writes before a tool call is kept in content.
  • With stream=true, text deltas stream as they are generated. Tool calls are sent whole in one delta.tool_calls chunk after any text (each entry has index, id, type, and function.name with the complete function.arguments string). The final chunk has finish_reason: "tool_calls". A reply that starts with { or a code fence is buffered until generation ends, so a JSON tool call is never streamed as text.
  • model=apple-foundation-model only accepts plain-text system, user, and assistant messages. Requests with tools, image input, tool-call history, and conversations whose final non-system message is not a user turn are rejected.

Function calling request:

{
  "model": "qwen3-4b-4bit",
  "messages": [
    { "role": "user", "content": "Add 2 and 3." }
  ],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "add_numbers",
        "description": "Add two integers.",
        "parameters": {
          "type": "object",
          "properties": {
            "a": { "type": "integer" },
            "b": { "type": "integer" }
          },
          "required": ["a", "b"]
        }
      }
    }
  ],
  "tool_choice": "auto"
}

Tool call response:

{
  "id": "chatcmpl-123",
  "object": "chat.completion",
  "created": 1741700000,
  "model": "qwen3-4b-4bit",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": null,
        "refusal": null,
        "annotations": [],
        "tool_calls": [
          {
            "id": "call_123",
            "type": "function",
            "function": {
              "name": "add_numbers",
              "arguments": "{\"a\":2,\"b\":3}"
            }
          }
        ]
      },
      "logprobs": null,
      "finish_reason": "tool_calls"
    }
  ],
  "usage": { "...": "..." }
}

Follow-up request with the tool result:

{
  "model": "qwen3-4b-4bit",
  "messages": [
    { "role": "user", "content": "Add 2 and 3." },
    {
      "role": "assistant",
      "content": null,
      "tool_calls": [
        {
          "id": "call_123",
          "type": "function",
          "function": { "name": "add_numbers", "arguments": "{\"a\":2,\"b\":3}" }
        }
      ]
    },
    { "role": "tool", "tool_call_id": "call_123", "content": "{\"sum\":5}" }
  ],
  "tools": [ "...same tools as before..." ]
}

Streaming tool call events:

data: {"id":"chatcmpl-123","object":"chat.completion.chunk","created":1741700000,"model":"qwen3-4b-4bit","choices":[{"index":0,"delta":{"role":"assistant"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-123","object":"chat.completion.chunk","created":1741700000,"model":"qwen3-4b-4bit","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"id":"call_123","type":"function","function":{"name":"add_numbers","arguments":"{\"a\":2,\"b\":3}"}}]},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-123","object":"chat.completion.chunk","created":1741700000,"model":"qwen3-4b-4bit","choices":[{"index":0,"delta":{},"logprobs":null,"finish_reason":"tool_calls"}]}

data: [DONE]

POST /v1/responses

OpenAI Responses-style wrapper over the same generation engine.

Simple request:

{
  "model": "llama-3.2-1b-instruct-4bit",
  "input": "Summarize on-device AI in one sentence.",
  "instructions": "Be concise.",
  "temperature": 0.7
}

Structured input with image:

{
  "model": "llama-3.2-11b-vision-instruct-4bit",
  "input": [
    {
      "type": "message",
      "role": "user",
      "content": [
        { "type": "input_text", "text": "Describe this image." },
        { "type": "input_image", "image_url": "https://example.com/cat.jpg" }
      ]
    }
  ]
}

Accepted input shapes:

  • plain string
  • array of OpenAI Responses input items

Current behavior:

  • Used for generation: model, input, instructions, temperature, tools, tool_choice, parallel_tool_calls, reasoning.effort
  • reasoning.effort follows the same mapping as reasoning_effort on /v1/chat/completions. A recognized value is echoed in the response reasoning.effort (lowercased). Otherwise that field is null. reasoning.summary is accepted but ignored, and the response reasoning.summary is always null.
  • When reasoning.effort enables thinking and the model produced reasoning, output contains a reasoning item before the item it preceded (the assistant message, or a function call):

    {
      "id": "rs_...",
      "type": "reasoning",
      "status": "completed",
      "summary": [],
      "content": [{ "type": "reasoning_text", "text": "..." }]
    }
    

    summary is always empty. usage.output_tokens_details.reasoning_tokens is an approximate estimate and is included in usage.output_tokens.

  • With stream=true, the reasoning item is streamed before the message as response.output_item.added, response.content_part.added (part type reasoning_text), response.reasoning_text.delta, response.reasoning_text.done, response.content_part.done, and response.output_item.done. Reasoning that the model emits after visible text has started appears only in the final response.completed payload.
  • max_output_tokens caps the generated tokens and is echoed in the response. The response status stays completed when the limit is reached; incomplete_details is not reported.
  • Echoed in the response metadata: top_p, previous_response_id, store, metadata, truncation
  • Supported: stream=true returns text/event-stream with OpenAI Responses events such as response.created, response.in_progress, response.output_item.added, response.content_part.added, response.output_text.delta, and response.completed. Function calls stream after any message item as response.output_item.added, one response.function_call_arguments.delta with the complete arguments, response.function_call_arguments.done, and response.output_item.done.
  • Rejected: single-object input, and legacy non-OpenAI aliases such as tool_call_id
  • Supported input-item types: message, input_text, input_image, function_call, function_call_output
  • Not implemented: input_file, file_id-based file resolution, reasoning summaries
  • Tool calling is client-side, as on /v1/chat/completions. The server never executes tools. When the model calls a tool, output contains a function_call item for each call, after any reasoning and message items. A turn with only tool calls has no message item. The client runs the tool and sends the result back in input as a function_call_output item with the matching call_id, usually after echoing the function_call item.
  • parallel_tool_calls defaults to true. With false, only the first tool call is returned. tool_choice behaves as on /v1/chat/completions.
  • model=apple-foundation-model only accepts plain-text requests on this route. tools, input_image, function_call, and function_call_output items are rejected for that model because image input and tool calling are not implemented on the Apple Foundation backend.

Function calling request:

{
  "model": "qwen3-4b-4bit",
  "input": "Add 2 and 3.",
  "tools": [
    {
      "type": "function",
      "name": "add_numbers",
      "description": "Add two integers.",
      "parameters": {
        "type": "object",
        "properties": {
          "a": { "type": "integer" },
          "b": { "type": "integer" }
        },
        "required": ["a", "b"]
      },
      "strict": true
    }
  ],
  "tool_choice": "auto"
}

Representative response shape:

{
  "id": "resp_123",
  "object": "response",
  "created_at": 1741700000,
  "completed_at": 1741700000,
  "background": false,
  "status": "completed",
  "error": null,
  "incomplete_details": null,
  "instructions": null,
  "max_output_tokens": null,
  "model": "llama-3.2-1b-instruct-4bit",
  "output": [
    {
      "id": "fc_123",
      "type": "function_call",
      "status": "completed",
      "call_id": "call_123",
      "name": "add_numbers",
      "arguments": "{\"a\":2,\"b\":3}"
    }
  ],
  "parallel_tool_calls": true,
  "previous_response_id": null,
  "reasoning": {
    "effort": null,
    "summary": null
  },
  "store": true,
  "temperature": 1,
  "text": {
    "format": {
      "type": "text"
    }
  },
  "tool_choice": "auto",
  "tools": [
    {
      "type": "function",
      "name": "add_numbers",
      "description": "Add two integers.",
      "parameters": {
        "type": "object"
      },
      "strict": true
    }
  ],
  "top_p": 1,
  "truncation": "disabled",
  "usage": {
    "input_tokens": 10,
    "input_tokens_details": {
      "cached_tokens": 0
    },
    "output_tokens": 6,
    "output_tokens_details": {
      "reasoning_tokens": 0
    },
    "total_tokens": 16
  },
  "user": null,
  "metadata": {}
}

Follow-up request with the tool result:

{
  "model": "qwen3-4b-4bit",
  "input": [
    { "role": "user", "content": "Add 2 and 3." },
    {
      "type": "function_call",
      "call_id": "call_123",
      "name": "add_numbers",
      "arguments": "{\"a\":2,\"b\":3}"
    },
    { "type": "function_call_output", "call_id": "call_123", "output": "{\"sum\":5}" }
  ],
  "tools": [ "...same tools as before..." ]
}

Notes:

  • Only tools of type function are supported.
  • /v1/responses uses the OpenAI Responses tool schema with top-level name, description, and parameters. The chat-style nested function wrapper is rejected on this route.
  • Duplicate function names, invalid names, and a tool_choice naming a function that is not in tools return an OpenAI-style validation error.
  • Parameter schemas must be JSON objects. Full JSON Schema enforcement is not implemented. strict is accepted and echoed but not enforced.

POST /v1/audio/transcriptions

Multipart transcription endpoint.

Example:

curl -X POST http://127.0.0.1:8080/v1/audio/transcriptions \
  -F "file=@sample.wav" \
  -F "model=apple" \
  -F "response_format=verbose_json" \
  -F "timestamp_granularities[]=segment" \
  -F "timestamp_granularities[]=word"

Validation rules:

  • required: file, model
  • built-in transcription engine label: apple
  • model=apple uses Apple’s built-in speech recognition
  • any other non-empty model value is treated as a Hugging Face repository ID for an MLX speech recognition model
  • supported speech recognition model families: Qwen3-ASR, Voxtral Realtime, Cohere Transcribe, Parakeet, GLM-ASR
  • forced-alignment models are rejected because /v1/audio/transcriptions has no reference-text field
  • max file size: 25 MB
  • allowed extensions: flac, m4a, mp3, mp4, mpeg, mpga, ogg, wav, webm
  • temperature must be between 0 and 1
  • timestamp_granularities only works with response_format=verbose_json

Supported response_format values:

  • json
  • text
  • srt
  • verbose_json
  • vtt

Response types:

  • json and verbose_json: JSON body
  • text: plain text
  • srt: SubRip text
  • vtt: WebVTT text

POST /v1/audio/speech

Synthesizes speech and returns audio data.

Request:

{
  "model": "apple",
  "input": "Hello from ai.local.",
  "voice": "com.apple.voice.compact.en-US.Samantha",
  "response_format": "mp3",
  "speed": 1.0
}

Example:

curl -X POST http://127.0.0.1:8080/v1/audio/speech \
  -H "Content-Type: application/json" \
  -d '{"model":"apple","input":"Hello from ai.local.","voice":"com.apple.voice.compact.en-US.Samantha","response_format":"mp3"}' \
  --output speech.mp3

Validation rules:

  • required: model, input, voice
  • built-in speech engine label: apple
  • model=apple uses Apple’s built-in speech synthesis
  • any other non-empty model value is treated as a Hugging Face repository ID for an MLX speech synthesis model
  • max input length: 4096 characters
  • model=apple requires the identifier of an Apple voice installed on the current device, such as com.apple.voice.compact.en-US.Samantha in the example above. Available voice identifiers depend on the device’s installed voices.
  • OpenAI-compatible voice aliases such as alloy, echo, fable, onyx, nova, and shimmer are not accepted for model=apple
  • MLX speech synthesis models with voice support accept their catalog voices, such as Kokoro af_alloy; models without catalog voices accept but ignore voice
  • supported response_format: mp3, wav
  • speed must be between 0.25 and 4.0
  • MLX speech synthesis currently ignores speed; Apple speech applies both voice and speed

Ollama-compatible API

POST /api/chat

Chat completion in Ollama-style request format.

Request:

{
  "model": "llama-3.2-1b-instruct-4bit",
  "messages": [
    { "role": "user", "content": "Say hello in one sentence." }
  ],
  "options": {
    "temperature": 0.7
  }
}

Current behavior:

  • If model is omitted, the server falls back to the currently selected model, then the first installed model, then the default model
  • Used: model, messages, options.temperature, options.num_predict, options.stop, think, keep_alive
  • options.num_predict caps the generated tokens. Non-positive values, including Ollama’s -1, use the default limit of 4096.
  • options.stop is a string or an array of strings.
  • keep_alive: 0 (or a zero duration such as "0s") releases the cached prompt prefix after the request. The model itself stays loaded. Other values are ignored.
  • think: true or false enables or disables model thinking. The strings "low", "medium", and "high" enable it. Other values are ignored.
  • Accepted but ignored: stream, format, tools, options.top_k, options.top_p
  • When think enables thinking, message.thinking contains the model’s reasoning, and message.content contains only the answer. Otherwise message.thinking is omitted, and reasoning is filtered from message.content by default.
  • eval_count includes the reasoning text.
  • Response uses the standard final Ollama chat object keys: model, created_at, message, done, done_reason, total_duration, load_duration, prompt_eval_count, prompt_eval_duration, eval_count, and eval_duration
  • prompt_eval_count and eval_count are real token counts. As in Ollama, prompt_eval_count excludes prompt tokens served from the prompt cache.
  • total_duration, load_duration, prompt_eval_duration, and eval_duration are measured, in nanoseconds. load_duration is close to 0 when the model was already in memory.
  • done_reason is length when options.num_predict (or the default limit) ended the generation, otherwise stop.
  • If generation fails, counts fall back to character-based estimates and durations are 0.
  • Unknown custom/<name> models return 404 with { "error": "model '<name>' not found" }

POST /api/generate

Single-prompt generation endpoint.

Request:

{
  "model": "llama-3.2-1b-instruct-4bit",
  "prompt": "Explain local-first AI in one sentence."
}

Optional images may be provided as an array of URL strings, local file paths, data: URLs, or base64 strings.

Current behavior:

  • Used: model, prompt, images, think, options.temperature, options.num_predict, options.stop, keep_alive
  • options and keep_alive follow the same rules as on /api/chat
  • think follows the same mapping as on /api/chat
  • Accepted but ignored: stream, format, system, raw, options.top_k, options.top_p, logprobs, top_logprobs
  • Response uses the standard final Ollama generate object keys: model, created_at, response, done, done_reason, total_duration, load_duration, prompt_eval_count, prompt_eval_duration, eval_count, and eval_duration
  • When think enables thinking, thinking contains the model’s reasoning and response contains only the answer. Otherwise thinking is omitted.
  • prompt_eval_count and eval_count are real token counts. As in Ollama, prompt_eval_count excludes prompt tokens served from the prompt cache.
  • total_duration, load_duration, prompt_eval_duration, and eval_duration are measured, in nanoseconds. load_duration is close to 0 when the model was already in memory.
  • done_reason is length when options.num_predict (or the default limit) ended the generation, otherwise stop.
  • If generation fails, counts fall back to character-based estimates and durations are 0.
  • Unknown custom/<name> models return 404 with { "error": "model '<name>' not found" }

POST /api/show

Returns inferred model metadata for an installed model.

Request:

{
  "model": "llama-3.2-1b-instruct-4bit",
  "verbose": true
}

Notes:

  • Returns 404 with { "error": "model '<name>' not found" } if the model is unavailable
  • Response keys follow the Ollama snake_case shape, including modified_at, details.parent_model, and model_info
  • license, family, parameter_size, quantization_level, template, and model_info are inferred from cached files or registered custom-model directories and model names
  • Verbose output is synthesized for compatibility; it is not backed by a native Ollama runtime or Modelfile

GET /api/tags

Lists available models in an Ollama-like response format.

{
  "models": [
    {
      "name": "llama-3.2-1b-instruct-4bit",
      "model": "llama-3.2-1b-instruct-4bit",
      "modified_at": "2026-03-11T14:00:00Z",
      "size": 1234567890,
      "digest": "c180fa9df3a6b73d6f4bf2af9eaef7ca51e9167e8d1812c1fc8de12d5d00d992",
      "details": {
        "format": "gguf",
        "family": "llama",
        "families": null,
        "parameter_size": "13B",
        "quantization_level": "Q4_0"
      }
    }
  ]
}

The details payload is compatibility metadata assembled by the server, not a direct export from Ollama. size is the resolved model-directory size in bytes for installed or registered custom/* models when the directory can be resolved, including Hugging Face snapshot layouts that store model payloads behind symlinks; otherwise it falls back to 0. digest is a stable SHA-256 hash of the normalized model name because the local cache does not expose native Ollama digests.

Errors

OpenAI-style error envelope

/v1/completions, /v1/chat/completions, /v1/responses, and /v1/audio/* convert validation and runtime failures into:

{
  "error": {
    "message": "Unsupported voice 'alloy'. Supported voices: <installed Apple voice identifiers>.",
    "type": "invalid_request_error",
    "param": "voice",
    "code": "invalid_request_error"
  }
}

For stream=true on /v1/completions and /v1/chat/completions, preflight validation failures still return the same JSON error envelope as a normal 4xx or 5xx response. If a failure happens after the SSE stream has started, the server emits event: error with that same error object and then terminates the stream with [DONE].

Current compatibility gaps

  • top_p may be accepted on text routes but is not enforced by the local generation engine.
  • stop, token limits, measured usage, prompt caching, and timings apply to MLX models. apple-foundation-model honors token limits only, and its usage counts are approximate.
  • Thinking controls toggle the chat template’s enable_thinking flag. Effort levels are not distinguished. Reasoning text is returned only on /v1/responses and the Ollama routes, and reasoning_tokens is an approximate estimate.
  • Tool calling is client-side only. Tool calls are streamed whole (one delta.tool_calls chunk or one response.function_call_arguments.delta per call), not token by token. JSON Schema is not enforced on arguments, and tool_choice: "required" or a named choice cannot force a call; the request fails if the model does not make one.
  • apple-foundation-model on /v1/* currently supports plain-text generation only; image input, tool calling, and tool-call history are rejected on that backend.
  • custom/<name> model IDs are reserved for runtime-registered local directories.
  • Legacy OpenAI compatibility aliases such as /v1/chat, functions, and function_call are intentionally not supported.
  • Ollama /api/chat and /api/generate currently return only the final non-streaming JSON object. Usage counters and durations are measured by the local evaluator.
  • Ollama /api/show and /api/tags return synthesized metadata derived from the local model cache, not from a native Ollama runtime.