HTTP API
Overview
ai.local exposes two HTTP API surfaces:
- OpenAI-compatible endpoints under
/v1/* - Ollama-compatible endpoints under
/api/*
Examples in this document use http://127.0.0.1:8080. Replace the host and port with the address used by your ai.local server.
Conventions
- Request bodies are JSON unless noted otherwise.
- Successful JSON endpoints return
Content-Type: application/json. POST /v1/audio/transcriptionsrequiresmultipart/form-data.POST /v1/audio/speechreturns binary audio, not JSON.- For OpenAI routes, use model IDs returned by
GET /v1/models, for examplellama-3.2-1b-instruct-4bit. - OpenAI text-generation routes resolve installed models as
mlx-community/<model>. Registered local models usecustom/<name>.apple-foundation-modelis handled separately when Apple Foundation Models are available on the device. - Ollama routes accept installed model names with or without the
mlx-community/prefix. Registered local models usecustom/<name>. - Text-generation routes strip
<think>and<thinking>blocks from model output by default. - Thinking controls (
reasoning_efforton/v1/chat/completions,reasoning.efforton/v1/responses,thinkon Ollama routes) passenable_thinkingto the model’s chat template. When a request omits them, or sends an unrecognized value, the model’s template default is used and output is unchanged. Models whose chat template does not readenable_thinkingignore the flag.apple-foundation-modelignores these controls. - When a request explicitly enables thinking,
<think>blocks are always removed from the visible text. The reasoning is returned separately on/v1/responsesas areasoningoutput item, and on Ollama routes asthinking./v1/chat/completionsdoes not return reasoning text. - Usage counts on MLX models are the tokenizer’s real prompt and completion token counts.
apple-foundation-model, and requests whose generation failed, fall back to character-based estimates. - MLX models keep the evaluated prompt (the KV cache) of the last request in memory. A request whose prompt starts with the same tokens, typically the same system prompt, only evaluates the tokens after the shared prefix. The reused token count is reported as
usage.prompt_tokens_details.cached_tokens(usage.input_tokens_details.cached_tokenson/v1/responses). The cache holds one prompt for one model, is skipped for image input, and is released on a memory warning, when the model is evicted, or when an Ollama request sendskeep_alive: 0. - Custom stop sequences (
stop, oroptions.stopon Ollama routes) are matched on the generated text, including text that spans several tokens. Generation ends at the first match and the stop sequence itself is not returned. They apply to MLX models only. - Token limits (
max_tokens,max_completion_tokens,max_output_tokens,options.num_predict) cap the generated tokens. Without one, or with a non-positive value, the limit is 4096 tokens. When the limit ends a generation,finish_reason(done_reasonon Ollama routes) islength. - The server reserves
/,/status,/v1/*, and/api/*for built-in routes.
Health and device control
HEAD /
Health probe. Returns 200 OK with no body.
GET /status
Returns basic server status.
{
"status": "Running",
"message": "Server is currently running."
}
POST /api/services/screen/brightness
Sets device brightness. brightness is clamped to 0...1.
Request:
{
"brightness": 0.5
}
Response:
{
"requestedBrightness": 0.5,
"appliedBrightness": 0.5
}
GET /api/services/screen/brightness
Returns the current screen brightness.
{
"brightness": 0.5
}
OpenAI-compatible API
GET /v1/models
Lists installed models, runtime-registered custom/<name> models, plus apple-foundation-model when the device reports Foundation Models availability.
{
"object": "list",
"data": [
{
"id": "llama-3.2-1b-instruct-4bit",
"object": "model",
"created": 0,
"owned_by": "local-ai-server"
}
]
}
Notes:
- Returned model IDs are lowercased and stripped of the
mlx-community/prefix. - Runtime-registered local directories keep the
custom/prefix in the returnedid. - The response uses the standard OpenAI list envelope and standard model fields only.
owned_byisappleforapple-foundation-modelandlocal-ai-serverfor installed andcustom/*models.- When
apple-foundation-modelis requested on OpenAI text routes, the server uses Apple Foundation Models instead of MLX. apple-foundation-modelcurrently supports plain-text generation only on/v1/*. Image input, tool calling, and tool-call history are rejected for that model.custom/<name>model IDs are reserved for runtime-registered local directories. Requesting an unknowncustom/<name>model returns an OpenAI-style400error withparam == "model".
POST /v1/completions
Single-prompt text completion.
Request:
{
"model": "llama-3.2-1b-instruct-4bit",
"prompt": "Write one short sentence about local inference.",
"temperature": 0.7,
"n": 1
}
Response shape:
{
"id": "UUID",
"object": "text_completion",
"created": 1741700000,
"model": "llama-3.2-1b-instruct-4bit",
"choices": [
{
"text": "Local inference keeps the model on your device.",
"index": 0,
"logprobs": null,
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 11,
"completion_tokens": 12,
"total_tokens": 23
}
}
Current behavior:
- Used:
model,prompt,temperature,n,max_tokens,stop stopis a string or an array of up to 4 strings. More than 4 returns400.finish_reasonislengthwhenmax_tokensended the generation, otherwisestop.- Non-streaming responses from MLX models include a
timingsobject next tousage. It is an extension to the OpenAI schema and uses llama.cpp server field names:prompt_n(prompt tokens evaluated),cache_n(prompt tokens served from the prompt cache),prompt_ms,prompt_per_second,predicted_n,predicted_ms,predicted_per_second,load_ms, andtotal_ms. - Accepted but not enforced:
top_p - Supported:
stream=truereturnstext/event-streamwith OpenAI-styletext_completionchunks and a final[DONE] - If generation fails after the SSE stream has started, the server emits
event: errorwith the standard OpenAIerrorenvelope before[DONE] nduplicates the same generated text across choices instead of sampling independent completionsmodel=apple-foundation-modelruns through Apple Foundation Models when they are available on the device
POST /v1/chat/completions
OpenAI Chat Completions endpoint.
Basic request:
{
"model": "llama-3.2-1b-instruct-4bit",
"messages": [
{ "role": "system", "content": "Reply in one sentence." },
{ "role": "user", "content": "What is edge inference?" }
],
"temperature": 0.7
}
Multimodal request shape also accepted:
{
"model": "llama-3.2-11b-vision-instruct-4bit",
"messages": [
{
"role": "user",
"content": [
{ "type": "text", "text": "Describe this image." },
{
"type": "image_url",
"image_url": {
"url": "https://example.com/cat.jpg"
}
}
]
}
]
}
Current behavior:
- Used:
model,messages,temperature,max_tokens,max_completion_tokens,stop,tools,tool_choice,parallel_tool_calls,reasoning_effort max_completion_tokenswins when both it andmax_tokensare sent.stopis a string or an array of up to 4 strings. More than 4 returns400.finish_reasonistool_callsfor a tool-call turn,lengthwhen the token limit ended the generation, otherwisestop.- Non-streaming responses from MLX models include a
timingsobject next tousage. It is an extension to the OpenAI schema and uses llama.cpp server field names:prompt_n(prompt tokens evaluated),cache_n(prompt tokens served from the prompt cache),prompt_ms,prompt_per_second,predicted_n,predicted_ms,predicted_per_second,load_ms, andtotal_ms. reasoning_effort:noneorminimaldisables model thinking.low,medium,high, orxhighenables it. Effort levels are not distinguished. Unrecognized values are ignored instead of rejected.- Reasoning text is never returned on this route, because the OpenAI Chat Completions shape has no field for it.
usage.completion_tokens_details.reasoning_tokensstays0. - Accepted but not enforced:
top_p,n - Supported:
stream=truereturnstext/event-streamwithchat.completion.chunkpayloads and a final[DONE] - If generation fails after the SSE stream has started, the server emits
event: errorwith the standard OpenAIerrorenvelope before[DONE] - Rejected: legacy
functions, legacyfunction_call, and non-standard chat request fields on/v1 - Response always contains a single choice
- Only
POST /v1/chat/completionsis registered./v1/chatis not available. - Only standard OpenAI chat content parts are documented on this route. For image input, send an
image_urlobject with aurlstring. - Tool calling follows the standard OpenAI client-side flow. The server never executes tools. When the model calls a tool, the response carries
message.tool_callsandfinish_reason: "tool_calls". The client runs the tool and sends the result back as arole: "tool"message with the matchingtool_call_id. - Only tools of type
functionare supported. Function names must match^[A-Za-z0-9_-]{1,64}$, andparametersmust be a JSON object schema. Full JSON Schema enforcement is not implemented.strictis accepted but ignored. - Tools are rendered through the model’s own chat template, and tool calls are parsed from the model’s native tool-call markup. A reply that is only a JSON object such as
{"name": "...", "arguments": {...}}or{"tool_calls": [...]}is also accepted as a tool call. Whether a model calls tools reliably depends on the model. - Tool calls that name a function that is not in
toolsare dropped. tool_choice:autois the default whentoolsis present.nonehides the tools from the model.requiredand{"type": "function", "function": {"name": "..."}}are passed to the model as an instruction. If the model then produces no matching tool call, the request fails with a500server_error.parallel_tool_callsdefaults totrue. Withfalse, only the first tool call is returned.contentisnullwhen the model returns only tool calls. Text the model writes before a tool call is kept incontent.- With
stream=true, text deltas stream as they are generated. Tool calls are sent whole in onedelta.tool_callschunk after any text (each entry hasindex,id,type, andfunction.namewith the completefunction.argumentsstring). The final chunk hasfinish_reason: "tool_calls". A reply that starts with{or a code fence is buffered until generation ends, so a JSON tool call is never streamed as text. model=apple-foundation-modelonly accepts plain-textsystem,user, andassistantmessages. Requests withtools, image input, tool-call history, and conversations whose final non-system message is not auserturn are rejected.
Function calling request:
{
"model": "qwen3-4b-4bit",
"messages": [
{ "role": "user", "content": "Add 2 and 3." }
],
"tools": [
{
"type": "function",
"function": {
"name": "add_numbers",
"description": "Add two integers.",
"parameters": {
"type": "object",
"properties": {
"a": { "type": "integer" },
"b": { "type": "integer" }
},
"required": ["a", "b"]
}
}
}
],
"tool_choice": "auto"
}
Tool call response:
{
"id": "chatcmpl-123",
"object": "chat.completion",
"created": 1741700000,
"model": "qwen3-4b-4bit",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": null,
"refusal": null,
"annotations": [],
"tool_calls": [
{
"id": "call_123",
"type": "function",
"function": {
"name": "add_numbers",
"arguments": "{\"a\":2,\"b\":3}"
}
}
]
},
"logprobs": null,
"finish_reason": "tool_calls"
}
],
"usage": { "...": "..." }
}
Follow-up request with the tool result:
{
"model": "qwen3-4b-4bit",
"messages": [
{ "role": "user", "content": "Add 2 and 3." },
{
"role": "assistant",
"content": null,
"tool_calls": [
{
"id": "call_123",
"type": "function",
"function": { "name": "add_numbers", "arguments": "{\"a\":2,\"b\":3}" }
}
]
},
{ "role": "tool", "tool_call_id": "call_123", "content": "{\"sum\":5}" }
],
"tools": [ "...same tools as before..." ]
}
Streaming tool call events:
data: {"id":"chatcmpl-123","object":"chat.completion.chunk","created":1741700000,"model":"qwen3-4b-4bit","choices":[{"index":0,"delta":{"role":"assistant"},"logprobs":null,"finish_reason":null}]}
data: {"id":"chatcmpl-123","object":"chat.completion.chunk","created":1741700000,"model":"qwen3-4b-4bit","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"id":"call_123","type":"function","function":{"name":"add_numbers","arguments":"{\"a\":2,\"b\":3}"}}]},"logprobs":null,"finish_reason":null}]}
data: {"id":"chatcmpl-123","object":"chat.completion.chunk","created":1741700000,"model":"qwen3-4b-4bit","choices":[{"index":0,"delta":{},"logprobs":null,"finish_reason":"tool_calls"}]}
data: [DONE]
POST /v1/responses
OpenAI Responses-style wrapper over the same generation engine.
Simple request:
{
"model": "llama-3.2-1b-instruct-4bit",
"input": "Summarize on-device AI in one sentence.",
"instructions": "Be concise.",
"temperature": 0.7
}
Structured input with image:
{
"model": "llama-3.2-11b-vision-instruct-4bit",
"input": [
{
"type": "message",
"role": "user",
"content": [
{ "type": "input_text", "text": "Describe this image." },
{ "type": "input_image", "image_url": "https://example.com/cat.jpg" }
]
}
]
}
Accepted input shapes:
- plain string
- array of OpenAI Responses input items
Current behavior:
- Used for generation:
model,input,instructions,temperature,tools,tool_choice,parallel_tool_calls,reasoning.effort reasoning.effortfollows the same mapping asreasoning_efforton/v1/chat/completions. A recognized value is echoed in the responsereasoning.effort(lowercased). Otherwise that field isnull.reasoning.summaryis accepted but ignored, and the responsereasoning.summaryis alwaysnull.-
When
reasoning.effortenables thinking and the model produced reasoning,outputcontains areasoningitem before the item it preceded (the assistant message, or a function call):{ "id": "rs_...", "type": "reasoning", "status": "completed", "summary": [], "content": [{ "type": "reasoning_text", "text": "..." }] }summaryis always empty.usage.output_tokens_details.reasoning_tokensis an approximate estimate and is included inusage.output_tokens. - With
stream=true, the reasoning item is streamed before the message asresponse.output_item.added,response.content_part.added(part typereasoning_text),response.reasoning_text.delta,response.reasoning_text.done,response.content_part.done, andresponse.output_item.done. Reasoning that the model emits after visible text has started appears only in the finalresponse.completedpayload. max_output_tokenscaps the generated tokens and is echoed in the response. The responsestatusstayscompletedwhen the limit is reached;incomplete_detailsis not reported.- Echoed in the response metadata:
top_p,previous_response_id,store,metadata,truncation - Supported:
stream=truereturnstext/event-streamwith OpenAI Responses events such asresponse.created,response.in_progress,response.output_item.added,response.content_part.added,response.output_text.delta, andresponse.completed. Function calls stream after any message item asresponse.output_item.added, oneresponse.function_call_arguments.deltawith the complete arguments,response.function_call_arguments.done, andresponse.output_item.done. - Rejected: single-object
input, and legacy non-OpenAI aliases such astool_call_id - Supported input-item types:
message,input_text,input_image,function_call,function_call_output - Not implemented:
input_file,file_id-based file resolution, reasoning summaries - Tool calling is client-side, as on
/v1/chat/completions. The server never executes tools. When the model calls a tool,outputcontains afunction_callitem for each call, after any reasoning and message items. A turn with only tool calls has nomessageitem. The client runs the tool and sends the result back ininputas afunction_call_outputitem with the matchingcall_id, usually after echoing thefunction_callitem. parallel_tool_callsdefaults totrue. Withfalse, only the first tool call is returned.tool_choicebehaves as on/v1/chat/completions.model=apple-foundation-modelonly accepts plain-text requests on this route.tools,input_image,function_call, andfunction_call_outputitems are rejected for that model because image input and tool calling are not implemented on the Apple Foundation backend.
Function calling request:
{
"model": "qwen3-4b-4bit",
"input": "Add 2 and 3.",
"tools": [
{
"type": "function",
"name": "add_numbers",
"description": "Add two integers.",
"parameters": {
"type": "object",
"properties": {
"a": { "type": "integer" },
"b": { "type": "integer" }
},
"required": ["a", "b"]
},
"strict": true
}
],
"tool_choice": "auto"
}
Representative response shape:
{
"id": "resp_123",
"object": "response",
"created_at": 1741700000,
"completed_at": 1741700000,
"background": false,
"status": "completed",
"error": null,
"incomplete_details": null,
"instructions": null,
"max_output_tokens": null,
"model": "llama-3.2-1b-instruct-4bit",
"output": [
{
"id": "fc_123",
"type": "function_call",
"status": "completed",
"call_id": "call_123",
"name": "add_numbers",
"arguments": "{\"a\":2,\"b\":3}"
}
],
"parallel_tool_calls": true,
"previous_response_id": null,
"reasoning": {
"effort": null,
"summary": null
},
"store": true,
"temperature": 1,
"text": {
"format": {
"type": "text"
}
},
"tool_choice": "auto",
"tools": [
{
"type": "function",
"name": "add_numbers",
"description": "Add two integers.",
"parameters": {
"type": "object"
},
"strict": true
}
],
"top_p": 1,
"truncation": "disabled",
"usage": {
"input_tokens": 10,
"input_tokens_details": {
"cached_tokens": 0
},
"output_tokens": 6,
"output_tokens_details": {
"reasoning_tokens": 0
},
"total_tokens": 16
},
"user": null,
"metadata": {}
}
Follow-up request with the tool result:
{
"model": "qwen3-4b-4bit",
"input": [
{ "role": "user", "content": "Add 2 and 3." },
{
"type": "function_call",
"call_id": "call_123",
"name": "add_numbers",
"arguments": "{\"a\":2,\"b\":3}"
},
{ "type": "function_call_output", "call_id": "call_123", "output": "{\"sum\":5}" }
],
"tools": [ "...same tools as before..." ]
}
Notes:
- Only tools of type
functionare supported. /v1/responsesuses the OpenAI Responses tool schema with top-levelname,description, andparameters. The chat-style nestedfunctionwrapper is rejected on this route.- Duplicate function names, invalid names, and a
tool_choicenaming a function that is not intoolsreturn an OpenAI-style validation error. - Parameter schemas must be JSON objects. Full JSON Schema enforcement is not implemented.
strictis accepted and echoed but not enforced.
POST /v1/audio/transcriptions
Multipart transcription endpoint.
Example:
curl -X POST http://127.0.0.1:8080/v1/audio/transcriptions \
-F "file=@sample.wav" \
-F "model=apple" \
-F "response_format=verbose_json" \
-F "timestamp_granularities[]=segment" \
-F "timestamp_granularities[]=word"
Validation rules:
- required:
file,model - built-in transcription engine label:
apple model=appleuses Apple’s built-in speech recognition- any other non-empty
modelvalue is treated as a Hugging Face repository ID for an MLX speech recognition model - supported speech recognition model families: Qwen3-ASR, Voxtral Realtime, Cohere Transcribe, Parakeet, GLM-ASR
- forced-alignment models are rejected because
/v1/audio/transcriptionshas no reference-text field - max file size: 25 MB
- allowed extensions:
flac,m4a,mp3,mp4,mpeg,mpga,ogg,wav,webm temperaturemust be between0and1timestamp_granularitiesonly works withresponse_format=verbose_json
Supported response_format values:
jsontextsrtverbose_jsonvtt
Response types:
jsonandverbose_json: JSON bodytext: plain textsrt: SubRip textvtt: WebVTT text
POST /v1/audio/speech
Synthesizes speech and returns audio data.
Request:
{
"model": "apple",
"input": "Hello from ai.local.",
"voice": "com.apple.voice.compact.en-US.Samantha",
"response_format": "mp3",
"speed": 1.0
}
Example:
curl -X POST http://127.0.0.1:8080/v1/audio/speech \
-H "Content-Type: application/json" \
-d '{"model":"apple","input":"Hello from ai.local.","voice":"com.apple.voice.compact.en-US.Samantha","response_format":"mp3"}' \
--output speech.mp3
Validation rules:
- required:
model,input,voice - built-in speech engine label:
apple model=appleuses Apple’s built-in speech synthesis- any other non-empty
modelvalue is treated as a Hugging Face repository ID for an MLX speech synthesis model - max input length: 4096 characters
model=applerequires the identifier of an Apple voice installed on the current device, such ascom.apple.voice.compact.en-US.Samanthain the example above. Available voice identifiers depend on the device’s installed voices.- OpenAI-compatible voice aliases such as
alloy,echo,fable,onyx,nova, andshimmerare not accepted formodel=apple - MLX speech synthesis models with voice support accept their catalog voices, such as Kokoro
af_alloy; models without catalog voices accept but ignorevoice - supported
response_format:mp3,wav speedmust be between0.25and4.0- MLX speech synthesis currently ignores
speed; Apple speech applies bothvoiceandspeed
Ollama-compatible API
POST /api/chat
Chat completion in Ollama-style request format.
Request:
{
"model": "llama-3.2-1b-instruct-4bit",
"messages": [
{ "role": "user", "content": "Say hello in one sentence." }
],
"options": {
"temperature": 0.7
}
}
Current behavior:
- If
modelis omitted, the server falls back to the currently selected model, then the first installed model, then the default model - Used:
model,messages,options.temperature,options.num_predict,options.stop,think,keep_alive options.num_predictcaps the generated tokens. Non-positive values, including Ollama’s-1, use the default limit of 4096.options.stopis a string or an array of strings.keep_alive:0(or a zero duration such as"0s") releases the cached prompt prefix after the request. The model itself stays loaded. Other values are ignored.think:trueorfalseenables or disables model thinking. The strings"low","medium", and"high"enable it. Other values are ignored.- Accepted but ignored:
stream,format,tools,options.top_k,options.top_p - When
thinkenables thinking,message.thinkingcontains the model’s reasoning, andmessage.contentcontains only the answer. Otherwisemessage.thinkingis omitted, and reasoning is filtered frommessage.contentby default. eval_countincludes the reasoning text.- Response uses the standard final Ollama chat object keys:
model,created_at,message,done,done_reason,total_duration,load_duration,prompt_eval_count,prompt_eval_duration,eval_count, andeval_duration prompt_eval_countandeval_countare real token counts. As in Ollama,prompt_eval_countexcludes prompt tokens served from the prompt cache.total_duration,load_duration,prompt_eval_duration, andeval_durationare measured, in nanoseconds.load_durationis close to0when the model was already in memory.done_reasonislengthwhenoptions.num_predict(or the default limit) ended the generation, otherwisestop.- If generation fails, counts fall back to character-based estimates and durations are
0. - Unknown
custom/<name>models return404with{ "error": "model '<name>' not found" }
POST /api/generate
Single-prompt generation endpoint.
Request:
{
"model": "llama-3.2-1b-instruct-4bit",
"prompt": "Explain local-first AI in one sentence."
}
Optional images may be provided as an array of URL strings, local file paths, data: URLs, or base64 strings.
Current behavior:
- Used:
model,prompt,images,think,options.temperature,options.num_predict,options.stop,keep_alive optionsandkeep_alivefollow the same rules as on/api/chatthinkfollows the same mapping as on/api/chat- Accepted but ignored:
stream,format,system,raw,options.top_k,options.top_p,logprobs,top_logprobs - Response uses the standard final Ollama generate object keys:
model,created_at,response,done,done_reason,total_duration,load_duration,prompt_eval_count,prompt_eval_duration,eval_count, andeval_duration - When
thinkenables thinking,thinkingcontains the model’s reasoning andresponsecontains only the answer. Otherwisethinkingis omitted. prompt_eval_countandeval_countare real token counts. As in Ollama,prompt_eval_countexcludes prompt tokens served from the prompt cache.total_duration,load_duration,prompt_eval_duration, andeval_durationare measured, in nanoseconds.load_durationis close to0when the model was already in memory.done_reasonislengthwhenoptions.num_predict(or the default limit) ended the generation, otherwisestop.- If generation fails, counts fall back to character-based estimates and durations are
0. - Unknown
custom/<name>models return404with{ "error": "model '<name>' not found" }
POST /api/show
Returns inferred model metadata for an installed model.
Request:
{
"model": "llama-3.2-1b-instruct-4bit",
"verbose": true
}
Notes:
- Returns
404with{ "error": "model '<name>' not found" }if the model is unavailable - Response keys follow the Ollama snake_case shape, including
modified_at,details.parent_model, andmodel_info license,family,parameter_size,quantization_level,template, andmodel_infoare inferred from cached files or registered custom-model directories and model names- Verbose output is synthesized for compatibility; it is not backed by a native Ollama runtime or
Modelfile
GET /api/tags
Lists available models in an Ollama-like response format.
{
"models": [
{
"name": "llama-3.2-1b-instruct-4bit",
"model": "llama-3.2-1b-instruct-4bit",
"modified_at": "2026-03-11T14:00:00Z",
"size": 1234567890,
"digest": "c180fa9df3a6b73d6f4bf2af9eaef7ca51e9167e8d1812c1fc8de12d5d00d992",
"details": {
"format": "gguf",
"family": "llama",
"families": null,
"parameter_size": "13B",
"quantization_level": "Q4_0"
}
}
]
}
The details payload is compatibility metadata assembled by the server, not a direct export from Ollama.
size is the resolved model-directory size in bytes for installed or registered custom/* models when the directory can be resolved, including Hugging Face snapshot layouts that store model payloads behind symlinks; otherwise it falls back to 0.
digest is a stable SHA-256 hash of the normalized model name because the local cache does not expose native Ollama digests.
Errors
OpenAI-style error envelope
/v1/completions, /v1/chat/completions, /v1/responses, and /v1/audio/* convert validation and runtime failures into:
{
"error": {
"message": "Unsupported voice 'alloy'. Supported voices: <installed Apple voice identifiers>.",
"type": "invalid_request_error",
"param": "voice",
"code": "invalid_request_error"
}
}
For stream=true on /v1/completions and /v1/chat/completions, preflight validation failures still return the same JSON error envelope as a normal 4xx or 5xx response. If a failure happens after the SSE stream has started, the server emits event: error with that same error object and then terminates the stream with [DONE].
Current compatibility gaps
top_pmay be accepted on text routes but is not enforced by the local generation engine.stop, token limits, measured usage, prompt caching, and timings apply to MLX models.apple-foundation-modelhonors token limits only, and its usage counts are approximate.- Thinking controls toggle the chat template’s
enable_thinkingflag. Effort levels are not distinguished. Reasoning text is returned only on/v1/responsesand the Ollama routes, andreasoning_tokensis an approximate estimate. - Tool calling is client-side only. Tool calls are streamed whole (one
delta.tool_callschunk or oneresponse.function_call_arguments.deltaper call), not token by token. JSON Schema is not enforced on arguments, andtool_choice: "required"or a named choice cannot force a call; the request fails if the model does not make one. apple-foundation-modelon/v1/*currently supports plain-text generation only; image input, tool calling, and tool-call history are rejected on that backend.custom/<name>model IDs are reserved for runtime-registered local directories.- Legacy OpenAI compatibility aliases such as
/v1/chat,functions, andfunction_callare intentionally not supported. - Ollama
/api/chatand/api/generatecurrently return only the final non-streaming JSON object. Usage counters and durations are measured by the local evaluator. - Ollama
/api/showand/api/tagsreturn synthesized metadata derived from the local model cache, not from a native Ollama runtime.