OVHcloud AI Endpoints Provider

Hosted serverless inference for open models (Llama, Mistral, Qwen, gpt-oss, DeepSeek, …) on OVHcloud's European cloud, behind a single OpenAI-compatible gateway. Includes an explicit opt-in for the keyless, rate-limited free tier.

Provider Name	`ovhcloud`
Module	`shared.plugins.model_provider.ovhcloud`
SDK	`openai` (OpenAI-compatible API)
Auth	API key (OVHcloud Manager → Public Cloud → AI Endpoints → API keys), or anonymous free tier (explicit opt-in)
Default endpoint	`https://oai.endpoints.kepler.ai.cloud.ovh.net/v1`

Highlights

Serverless open models — Llama, Mistral, Qwen, gpt-oss, DeepSeek distills and more, with no GPU provisioning
OpenAI-compatible — standard /v1/chat/completions surface through one unified gateway
European hosting — inference runs in OVHcloud's EU data centers (GDPR-friendly data locality)
Catalog auto-detect — per-model context window (and modality, when reported) from GET /v1/models at connect(), with manual override knobs
Anonymous free tier — evaluate without a key via an explicit opt-in (JAATO_OVHCLOUD_ALLOW_ANONYMOUS); never a silent fallback
Extended thinking — reasoning extraction for models that emit it (e.g. DeepSeek-R1 distills, gpt-oss)

Quick start

# Discover the config + verify your setup (source of truth — never drifts):
jaato-scaffold explain provider ovhcloud      # typed knobs, modalities, context
jaato-doctor --workspace . --env-file .env    # confirm creds + endpoint resolve

Python quick start

from jaato import JaatoClient

client = JaatoClient(provider_name="ovhcloud")
client.connect(
    project=None,
    location=None,
    model="gpt-oss-120b"
)
client.configure_tools(registry)

response = client.send_message(
    "Hello from OVHcloud!",
    on_output=on_output
)

Evaluate keyless (rate-limited)

# Explicit opt-in — the free tier is for evaluation only
export JAATO_OVHCLOUD_ALLOW_ANONYMOUS=true

Authentication

Get Your API Key

Create a key in the OVHcloud Manager: Public Cloud → AI & Machine Learning → AI Endpoints → API keys
Set JAATO_OVHCLOUD_API_KEY (the vendor's own OVH_AI_ENDPOINTS_ACCESS_TOKEN also works)

For the exact credential variables and their precedence, run jaato-scaffold explain provider ovhcloud — the live registry is the source of truth, so it never drifts from the code. Then jaato-doctor confirms the credential resolves and the endpoint is reachable before your client calls connect().

Without a key, the provider fails loud — unless you explicitly opt into OVHcloud's anonymous free tier, which is heavily rate-limited and intended for evaluation only.

Authenticate + verify

# Environment variable (jaato namespace, highest priority)
export JAATO_OVHCLOUD_API_KEY=<your-key>

# ...or the vendor's own variable (honored as-is)
export OVH_AI_ENDPOINTS_ACCESS_TOKEN=<your-key>

# Confirm the daemon will resolve them
jaato-doctor --workspace . --env-file .env

Configuration

jaato doesn't duplicate the provider's environment variables here — they drift. The installed framework is the source of truth: run jaato-scaffold explain provider ovhcloud for the full typed configuration (credential variables, base_url / context_length / modalities / allow_anonymous knobs and their defaults), and jaato-doctor to verify your environment resolves them.

Inspect config + verify

jaato-scaffold explain provider ovhcloud      # typed knobs + env, from the live registry
jaato-doctor --workspace . --env-file .env

Popular model names

gpt-oss-120b
gpt-oss-20b
Meta-Llama-3_3-70B-Instruct
Mistral-Small-3.2-24B-Instruct-2506
Qwen2.5-Coder-32B-Instruct
DeepSeek-R1-Distill-Llama-70B
# IDs are case-sensitive — browse the catalog:
# https://endpoints.ai.cloud.ovh.net/catalog

Profile Configuration

OVHcloud has a small set of profile knobs under plugin_configs.ovhcloud — the catalog auto-detects the context window when it reports one, and the manual knobs cover the rest.

Key	Type	Description
`base_url`	str	Override the default gateway URL (e.g. a local proxy, or a legacy per-model endpoint)
`context_length`	int	Manual context-window override, used when the catalog doesn't report the model's window
`modalities`	list[str]	Assert / correct input modalities (e.g. `["text","image"]` for vision models like `Qwen2.5-VL-72B-Instruct`)
`allow_anonymous`	bool	Opt into the keyless rate-limited free tier (evaluation only)

Profile config example (YAML)

name: ovhcloud-coder
model: Qwen2.5-Coder-32B-Instruct
provider: ovhcloud

plugins:
  - cli
  - file_edit

plugin_configs:
  ovhcloud:
    # Needed when the catalog doesn't report the window.
    context_length: 32768

Vision model (assert modalities)

provider: ovhcloud
model: Qwen2.5-VL-72B-Instruct
plugin_configs:
  ovhcloud:
    context_length: 32768
    modalities: ["text", "image"]

Model Catalog

list_models() queries GET /v1/models on the unified gateway (authenticated when a key is available; the public catalog is also served anonymously).

At connect() the provider bootstraps the active model's context window from that catalog when it reports one (the lookup tolerates the common key spellings — context_length, max_model_len, max_context_length). There is no hardcoded fallback — if the catalog doesn't report a window, set context_length and the provider fails loud telling you so. Per-model context sizes are listed on the catalog page.

Browse the catalog

from shared.plugins.model_provider.ovhcloud.provider import (
    OVHcloudProvider
)
from shared.plugins.model_provider.base import ProviderConfig

provider = OVHcloudProvider()
provider.initialize(ProviderConfig())

# List models from the unified gateway
models = provider.list_models()

# Connect and check the resolved context limit
provider.connect("gpt-oss-120b")
print(provider.get_context_limit())

Error Handling

Exception	Cause
`APIKeyNotFoundError`	No API key in env or stored credentials, and anonymous access not opted in
`AuthenticationError`	API key rejected (401/403)
`RateLimitError`	Rate limit exceeded (429) — constant on the anonymous tier
`ModelNotFoundError`	Model not in the AI Endpoints catalog (IDs are case-sensitive)
`ContextLimitError`	Prompt exceeds the model's context window
`InfrastructureError`	Upstream server error (5xx) or connection failure

Handle errors

from shared.plugins.model_provider.ovhcloud.errors import (
    APIKeyNotFoundError,
    AuthenticationError,
    RateLimitError,
    ContextLimitError,
)

try:
    provider.initialize(config)
    provider.connect("gpt-oss-120b")
    result = provider.complete(messages, tools=tools)
except APIKeyNotFoundError:
    print("Set JAATO_OVHCLOUD_API_KEY (jaato-doctor confirms it resolves)")
except AuthenticationError:
    print("Invalid API key")
except RateLimitError as e:
    if e.retry_after:
        print(f"Rate limited; retry in {e.retry_after}s")
except ContextLimitError:
    print("Prompt too long for this model")