> ## Documentation Index
> Fetch the complete documentation index at: https://www.rumus.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# OpenAI-compatible endpoints

> Connect Rumus to OpenRouter, vLLM, LiteLLM, LocalAI, Azure OpenAI, or any service that speaks the OpenAI API.

The **OpenAI-compatible** provider is the universal escape hatch. If a service exposes the standard `/v1/chat/completions` endpoint, Rumus can use it. This unlocks:

* **OpenRouter** — one key, hundreds of models from many labs.
* **vLLM / LiteLLM / LocalAI / Text Generation Inference** — self-hosted inference.
* **Azure OpenAI** — Microsoft's hosted OpenAI deployment.
* **Internal corporate gateways** — your team's shared LLM proxy.

## Before you start

Have these on hand:

* The full **endpoint URL** for the service.
* An **API key** if the endpoint requires one.
* The **model ID** as the endpoint expects to receive it (this varies — `openai/gpt-4o`, `meta-llama/Llama-3.1-70B-Instruct`, your-deployment-name, etc.).

## Add an OpenAI-compatible endpoint

<Steps>
  <Step title="Open the model settings">
    Go to **Settings → AI → Models** and click **Add Model**.
  </Step>

  <Step title="Pick the provider">
    Set **Provider** to **OpenAI Compatible**.
  </Step>

  <Step title="Endpoint URL">
    Paste the full endpoint URL. Most services document this; common values:

    | Service       | Endpoint                       |
    | ------------- | ------------------------------ |
    | OpenRouter    | `https://openrouter.ai/api/v1` |
    | vLLM (local)  | `http://localhost:8000/v1`     |
    | LiteLLM proxy | `http://localhost:4000/v1`     |
    | LocalAI       | `http://localhost:8080/v1`     |

    For Azure OpenAI, the URL takes the form `https://<resource>.openai.azure.com/openai/deployments/<deployment>` — note that Azure also requires an `api-version` query parameter.
  </Step>

  <Step title="API key">
    Paste the key if required. Some local servers don't need one — leave it blank in that case. Keys are stored encrypted in your local [vault](/docs/account/vault).
  </Step>

  <Step title="Model ID">
    Enter the model ID exactly as the endpoint expects. Copy it from the service's docs:

    | Example service         | Example model ID                            |
    | ----------------------- | ------------------------------------------- |
    | OpenRouter              | `anthropic/claude-sonnet-4.5`               |
    | OpenRouter (auto-route) | `openrouter/auto`                           |
    | vLLM                    | The model name you launched the server with |
    | Azure OpenAI            | Your deployment name                        |
  </Step>

  <Step title="Capabilities">
    Set capability flags based on what the model supports — tool calling, vision, prompt cache. When in doubt, start with just **Tool Calling** on and adjust if you hit errors.
  </Step>

  <Step title="Save">
    The model appears in the picker under **Custom Models**.
  </Step>
</Steps>

## Service-specific notes

### OpenRouter

OpenRouter gives you a single key for hundreds of models with usage-based billing.

* Sign up and add credit at [openrouter.ai](https://openrouter.ai/).
* Create a key from **Keys** in your OpenRouter dashboard.
* Use **Endpoint URL** `https://openrouter.ai/api/v1`.
* Model IDs follow the format `<lab>/<model>` — see the [models page](https://openrouter.ai/models). Use `openrouter/auto` to let OpenRouter pick.

### vLLM, LiteLLM, LocalAI, TGI

These all serve the OpenAI API on a local or self-hosted port. Point Rumus at the URL and use the model ID you started the server with. None of them require a key by default, but most support adding one through their config.

### Azure OpenAI

Azure has two quirks compared to vanilla OpenAI:

1. The URL embeds your resource and deployment names.
2. You must pass `api-version` as a query parameter.

Append it to the endpoint URL when configuring the provider:

```
https://<resource>.openai.azure.com/openai/deployments/<deployment>?api-version=2024-10-21
```

The model ID can be the same as the deployment name. Authentication uses the `api-key` header rather than `Authorization: Bearer`, which Rumus handles automatically when the URL pattern matches Azure.

### Internal corporate gateway

If your company runs a shared LLM proxy (often LiteLLM), get the URL and a per-user key from your platform team. The setup is identical to LiteLLM above.

## Tips

* **One provider, many models.** You can add the same OpenRouter or Azure setup multiple times with different model IDs — Rumus treats each as its own entry in the picker.
* **Privacy flag.** Tick **Privacy Protection** on the model card if the upstream commits to no logging / no training. The chat picker shows a green shield so you can spot privacy-safe models at a glance.
* **Ollama already has a dedicated provider.** Use the [Ollama](/docs/models/ollama) provider, not OpenAI-compatible, for cleaner handling of its custom request shape.

## Troubleshooting

<AccordionGroup>
  <Accordion title="404 Not Found">
    Endpoint URL is wrong, or you forgot the `/v1` suffix some services require. Double-check by hitting the URL with `curl`.
  </Accordion>

  <Accordion title="401 / 403 Unauthorized">
    API key missing or wrong. For Azure, make sure you're sending the key as `api-key`, not bearer.
  </Accordion>

  <Accordion title="Model returns errors mid-stream">
    The endpoint may not implement every OpenAI feature. Try turning off **Tool Calling** or **Prompt Cache** on the model card to see whether one of them is unsupported.
  </Accordion>

  <Accordion title="OpenRouter says 'No credits'">
    Top up your balance at [openrouter.ai/credits](https://openrouter.ai/credits).
  </Accordion>
</AccordionGroup>

<Note>
  Hit a snag we didn't cover? Ask in the [Rumus community](https://www.rumus.ai/community).
</Note>

## Next steps

<CardGroup cols={2}>
  <Card title="Other providers" icon="grid" href="/docs/models/overview">
    Anthropic, OpenAI, Google, Z.AI, DeepSeek, Kimi, Ollama.
  </Card>

  <Card title="AI assistant" icon="robot" href="/docs/ai/overview">
    What you can do with the agent once a model is connected.
  </Card>
</CardGroup>