QUILIO

Razor SDK 0.4.0 · Local inference

Local LLM inference SDK for Mac and Android.

Razor is a C++ inference SDK for Apple Silicon and Android. Run models on device, with CPU inference on both platforms and GPU inference on Apple Silicon.

Apple SiliconAndroid arm64CPU and GPULocal execution

Razor for Mac

Install Razor Manager and Razor Terminal

The Mac apps put model setup and chat in two familiar windows. Razor Manager shows which model files are ready. Razor Terminal opens the selected model for chat and computer actions.

1. Get an approved evaluation build

Ask your organization for Razor Manager and Razor Terminal.

2. Move both apps to Applications

Open the downloaded folder and drag each app into Applications.

3. Check model files

Open Razor Manager, choose a chat profile and check its file status.

4. Start a chat

Select Open in Razor Terminal. Model weights are downloaded separately from their listed sources.

Current SDK terms allow evaluation within the licensee's organization. They do not allow public distribution or sale of a product containing Razor. A one-time Mac app purchase is planned after commercial distribution rights are available.

Request evaluation access

Supported model families

Validated model configurations

Published results cover five configurations across three model families. Other profiles are available in the CLI and may use external runtimes or remain experimental.

SmolLM2

135M, 360M and 1.7B

Published fp32 and int8 results. Quality and speed vary with model size and weight format.

Qwen3

0.6B

Instruction chat support with tool calling through the configured chat format.

Gemma 3

1B

Published results include fp32, int8 and int4 formats. Gemma model terms apply.

Measured performance

Speed depends on the model and device

SmolLM2 135M decode results below use greedy generation. Weight formats and hardware differ by row. Treat the values as workload measurements.

Device and runtimeModel and formatDecode
Apple M4 Max · Razor CPUSmolLM2 135M · int8155 tok/s
Apple M4 Max · Razor MLXSmolLM2 135M · MLX563 tok/s
Snapdragon 8 Gen 3 · Razor CPUSmolLM2 135M · int8, cold151 tok/s
Snapdragon 8 Gen 3 · LiteRTSmolLM2 135M · q864 tok/s

Apple GPU comparison: 64-token prompt, 128 generated tokens. Android result: cold device. Results change with prompt length, model format and device temperature.

Evaluation

Accuracy results stay beside speed results

On a 68-question factual suite with identical prompts and greedy decoding, Razor scored 58 correct and MLX-LM scored 60. The reported 95% confidence intervals overlap. Model and weight format affect output quality.

58 / 68Razor
60 / 68MLX-LM
32 / 32fp32 token match on listed parity runs

Modes

10 ways to run a local model

One command covers chat, tools, coding, shell work, memory across sessions, diffusion models, images and voice. Some profiles use external runtimes or remain experimental, and reply quality depends on the model.

Chat

A conversation in the terminal with more than 20 slash commands. Switch models in the middle of a conversation.

Tools

Eight built-in tools, plus your own in OpenAI format. Razor shows each command and its risk and asks before it runs.

Coder

Project-aware file and command tools for coding models. Edits and commands wait for approval.

Shell

Small models turn a plain request into a shell command. Piped input joins the prompt.

Long context

Razor archives a project's chat on disk and recalls relevant earlier turns, within the model's context limit.

Diffusion

Masked-diffusion language models with six pass budgets, exact-length replies and infill.

Image, video, voice

Multimodal profiles take a picture, a clip or a recording. A speech profile replies with audio.

Fast decoding

Optional fast decoding on eligible models and backends. Standard decoding stays available.

Planner

Give a workload and limits. Razor predicts which models fit on this device before you download them.

Model finder

Describe a task and Razor ranks the models you have, with the command to start each one.

Using Razor

Local inference for app developers

Razor provides a C++ SDK and command-line tools for running supported language models on Apple Silicon and Android arm64. Published validation covers SmolLM2 135M, 360M and 1.7B, Qwen3 0.6B, and Gemma 3 1B. Other profiles have separate availability and validation status.

Does Razor support tool calling?

Tool calls work with configured model formats. The model determines whether it can produce reliable tool requests.

Does disk-backed history remove context limits?

Disk-backed history retrieves relevant earlier turns into the prompt. The model's configured context window still applies.

Where does inference run?

Inference runs on the device. Razor supports CPU inference on Apple Silicon and Android, with MLX GPU inference on Apple Silicon.

Evaluate Razor on your hardware

The 30-day evaluation includes the SDK and benchmark harness. Model licences apply to their weights.

Request an evaluation