MLX Guide

MetalCloud hosts run jobs with Apple's MLX framework on the Mac. A customer starts that work by submitting an inference job and a model id.

Submit an inference job

The host menu bar app executes MLX inference, MLX training, and MLX fine-tuning. The job you send from the API uses type inference and a model_id from the model library. The Mac runs that job as an MLX inference workload.

Python
from metalcloud import Client

client = Client(api_key="mc_...")
job = client.jobs.submit(
    type="inference",
    model_id="mlx-community/Llama-3.2-1B",
    input={
        "prompt": "Summarise unified memory in two sentences.",
        "max_tokens": 200,
    },
    min_memory_gb=16,
    max_duration_minutes=30,
)
print(job.uuid)
completed = job.wait()

Pick model_id from GET /api/v1/models?hosts=.... The id above shows the repo shape. The host must have that repo cached and marked runnable. You can limit placement with host_uuids.

The same call over HTTP is POST /api/v1/jobs with a bearer token. Field rules, statuses, and the other job routes are in the REST API.

Prompt and token limit

Put the text in input.prompt. max_tokens is optional. If you set it equal to the model's published context length, the API drops it and the host applies its own cap. Any larger request is reduced to the job's max_duration_minutes at 25 tokens per second. Ask GET /api/v1/models/context?repo_id=... when you need the published length.

What the Mac runs

MLX inference is the customer path above.
The host app can also run MLX training and MLX fine-tuning workloads.
The API accepts training, fine_tuning, batch, and container as well as inference. Send inference for a model reply.

A host only receives work while it is online and inside the hours set in availability. If no machine matches the model and the memory requirement, create returns 503.