Skip to main content

Documentation Index

Fetch the complete documentation index at: https://www.rumus.ai/docs/llms.txt

Use this file to discover all available pages before exploring further.

The OpenAI-compatible provider is the universal escape hatch. If a service exposes the standard /v1/chat/completions endpoint, Rumus can use it. This unlocks:
  • OpenRouter — one key, hundreds of models from many labs.
  • vLLM / LiteLLM / LocalAI / Text Generation Inference — self-hosted inference.
  • Azure OpenAI — Microsoft’s hosted OpenAI deployment.
  • Internal corporate gateways — your team’s shared LLM proxy.

​
Before you start

Have these on hand:
  • The full endpoint URL for the service.
  • An API key if the endpoint requires one.
  • The model ID as the endpoint expects to receive it (this varies — openai/gpt-4o, meta-llama/Llama-3.1-70B-Instruct, your-deployment-name, etc.).

​
Add an OpenAI-compatible endpoint

1

Open the model settings

Go to Settings → AI → Models and click Add Model.
2

Pick the provider

Set Provider to OpenAI Compatible.
3

Endpoint URL

Paste the full endpoint URL. Most services document this; common values:
ServiceEndpoint
OpenRouterhttps://openrouter.ai/api/v1
vLLM (local)http://localhost:8000/v1
LiteLLM proxyhttp://localhost:4000/v1
LocalAIhttp://localhost:8080/v1
For Azure OpenAI, the URL takes the form https://<resource>.openai.azure.com/openai/deployments/<deployment> — note that Azure also requires an api-version query parameter.
4

API key

Paste the key if required. Some local servers don’t need one — leave it blank in that case. Keys are stored encrypted in your local vault.
5

Model ID

Enter the model ID exactly as the endpoint expects. Copy it from the service’s docs:
Example serviceExample model ID
OpenRouteranthropic/claude-sonnet-4.5
OpenRouter (auto-route)openrouter/auto
vLLMThe model name you launched the server with
Azure OpenAIYour deployment name
6

Capabilities

Set capability flags based on what the model supports — tool calling, vision, prompt cache. When in doubt, start with just Tool Calling on and adjust if you hit errors.
7

Save

The model appears in the picker under Custom Models.

​
Service-specific notes

​
OpenRouter

OpenRouter gives you a single key for hundreds of models with usage-based billing.
  • Sign up and add credit at openrouter.ai.
  • Create a key from Keys in your OpenRouter dashboard.
  • Use Endpoint URL https://openrouter.ai/api/v1.
  • Model IDs follow the format <lab>/<model> — see the models page. Use openrouter/auto to let OpenRouter pick.

​
vLLM, LiteLLM, LocalAI, TGI

These all serve the OpenAI API on a local or self-hosted port. Point Rumus at the URL and use the model ID you started the server with. None of them require a key by default, but most support adding one through their config.

​
Azure OpenAI

Azure has two quirks compared to vanilla OpenAI:
  1. The URL embeds your resource and deployment names.
  2. You must pass api-version as a query parameter.
Append it to the endpoint URL when configuring the provider: 
https://<resource>.openai.azure.com/openai/deployments/<deployment>?api-version=2024-10-21
The model ID can be the same as the deployment name. Authentication uses the api-key header rather than Authorization: Bearer, which Rumus handles automatically when the URL pattern matches Azure.

​
Internal corporate gateway

If your company runs a shared LLM proxy (often LiteLLM), get the URL and a per-user key from your platform team. The setup is identical to LiteLLM above.

​
Tips

  • One provider, many models. You can add the same OpenRouter or Azure setup multiple times with different model IDs — Rumus treats each as its own entry in the picker.
  • Privacy flag. Tick Privacy Protection on the model card if the upstream commits to no logging / no training. The chat picker shows a green shield so you can spot privacy-safe models at a glance.
  • Ollama already has a dedicated provider. Use the Ollama provider, not OpenAI-compatible, for cleaner handling of its custom request shape.

​
Troubleshooting

Endpoint URL is wrong, or you forgot the /v1 suffix some services require. Double-check by hitting the URL with curl.
API key missing or wrong. For Azure, make sure you’re sending the key as api-key, not bearer.
The endpoint may not implement every OpenAI feature. Try turning off Tool Calling or Prompt Cache on the model card to see whether one of them is unsupported.
Top up your balance at openrouter.ai/credits.
Hit a snag we didn’t cover? Ask in the Rumus community.

​
Next steps

Other providers

Anthropic, OpenAI, Google, Z.AI, DeepSeek, Kimi, Ollama.

AI assistant

What you can do with the agent once a model is connected.