Nebius Token Factory Provider

Hosted serverless inference for open models (Llama, Qwen, DeepSeek-R1, Mistral, …) behind a single OpenAI-compatible API. Models use the vendor/model form; your own registered fine-tunes run on the same endpoint, addressed by name.

Provider Name	`nebius`
Module	`shared.plugins.model_provider.nebius`
SDK	`openai` (OpenAI-compatible API)
Auth	API key (from the Nebius Token Factory dashboard)
Default endpoint	`https://api.tokenfactory.nebius.com/v1`

Highlights

Serverless open models — Llama, Qwen, DeepSeek-R1, Mistral and more, with no GPU provisioning
OpenAI-compatible — standard /v1/chat/completions surface
Catalog auto-detect — the RichModel catalog (GET /v1/models) drives per-model context window and input modalities at connect()
Fine-tunes supported — registered custom models run on the same serverless endpoint; just point the profile at the model name
Extended thinking — reasoning extraction for models that emit it (e.g. DeepSeek-R1)
Cache reporting — cached_tokens parsed from the usage object when the upstream reports it

Quick start

# Discover the config + verify your setup (source of truth — never drifts):
jaato-scaffold explain provider nebius        # typed knobs, modalities, context
jaato-doctor --workspace . --env-file .env    # confirm creds + endpoint resolve

Python quick start

from jaato import JaatoClient

client = JaatoClient(provider_name="nebius")
client.connect(
    project=None,
    location=None,
    model="deepseek-ai/DeepSeek-R1"
)
client.configure_tools(registry)

response = client.send_message(
    "Hello from Nebius!",
    on_output=on_output
)

Use your own fine-tune

# A registered Token Factory fine-tune is addressed by name —
# same serverless endpoint, catalog-detected context + modality.
client.connect(None, None, "legislation-qa-private")

Authentication

Get Your API Key

Create a key in the Nebius Token Factory dashboard
Store it with nebius-auth (validates against /chat/completions and stores securely)

For the exact credential variables and their precedence, run jaato-scaffold explain provider nebius — the live registry is the source of truth, so it never drifts from the code. Then jaato-doctor confirms the credential resolves and the endpoint is reachable before your client calls connect().

Authenticate + verify

# Store credentials (validates against /chat/completions)
nebius-auth
nebius-auth status      # check
nebius-auth logout      # clear

# Confirm the daemon will resolve them
jaato-doctor --workspace . --env-file .env

Configuration

jaato doesn't duplicate the provider's environment variables here — they drift. The installed framework is the source of truth: run jaato-scaffold explain provider nebius for the full typed configuration (credential variables, base_url / context_length / modalities knobs and their defaults), and jaato-doctor to verify your environment resolves them.

Inspect config + verify

jaato-scaffold explain provider nebius        # typed knobs + env, from the live registry
jaato-doctor --workspace . --env-file .env

Popular model names

deepseek-ai/DeepSeek-R1
meta-llama/Llama-3.3-70B-Instruct
Qwen/Qwen2.5-72B-Instruct
mistralai/Mistral-Nemo-Instruct-2407
# your own registered fine-tune, addressed by name:
legislation-qa-private

Profile Configuration

Nebius has a small set of profile knobs under plugin_configs.nebius — the catalog auto-detects most settings, so overrides are rarely needed.

Key	Type	Description
`base_url`	str	Override the default base URL (e.g. a local proxy)
`context_length`	int	Manual context-window override, used when the catalog lacks the model
`modalities`	list[str]	Assert / correct input modalities (e.g. `["text","image"]`) for a model the catalog doesn't classify

Self-deployed & fine-tuned models

Fine-tunes run on the same serverless endpoint, addressed by name. After you register one with Token Factory (a custom name such as legislation-qa-private), point the profile at it. The provider's authenticated GET /v1/models is account-scoped, so your fine-tunes appear in the catalog and their context window (inherited from the base model) is detected automatically.

Profile config example (YAML)

name: nebius-reasoning
model: deepseek-ai/DeepSeek-R1
provider: nebius

plugins:
  - cli
  - file_edit

plugin_configs:
  nebius:
    # Usually unnecessary — the catalog auto-detects context + modality.
    context_length: 131072
    modalities: ["text"]

Point a profile at your fine-tune

provider: nebius
model: legislation-qa-private   # your deployed fine-tune's name

Model Catalog

list_models() queries the RichModel catalog at GET /v1/models. The fetch is authenticated (sends your key), so the listing is account-scoped — your deployed fine-tunes appear alongside the public catalog.

At connect() the provider bootstraps the active model's metadata from that catalog: the per-model context_length is the primary context-window tier, and architecture.modality (OpenRouter-style input->output) drives input-modality detection. There is no hardcoded fallback — if a model isn't listed, set context_length and the provider fails loud telling you so.

Browse the catalog

from shared.plugins.model_provider.nebius.provider import (
    NebiusProvider
)
from shared.plugins.model_provider.base import ProviderConfig

provider = NebiusProvider()
provider.initialize(ProviderConfig())

# List models (authenticated → includes your fine-tunes)
models = provider.list_models()

# Connect and check the catalog-detected context limit
provider.connect("deepseek-ai/DeepSeek-R1")
print(provider.get_context_limit())

Error Handling

Exception	Cause
`APIKeyNotFoundError`	No API key in env or stored credentials
`AuthenticationError`	API key rejected (401/403)
`RateLimitError`	Rate limit or credits exhausted (429)
`ModelNotFoundError`	Model not in the Nebius catalog
`ContextLimitError`	Prompt exceeds the model's context window
`InfrastructureError`	Upstream server error (5xx) or connection failure

Handle errors

from shared.plugins.model_provider.nebius.errors import (
    APIKeyNotFoundError,
    AuthenticationError,
    RateLimitError,
    ContextLimitError,
)

try:
    provider.initialize(config)
    provider.connect("deepseek-ai/DeepSeek-R1")
    result = provider.complete(messages, tools=tools)
except APIKeyNotFoundError:
    print("Run: nebius-auth  (jaato-doctor confirms it resolves)")
except AuthenticationError:
    print("Invalid API key")
except RateLimitError as e:
    if e.retry_after:
        print(f"Rate limited; retry in {e.retry_after}s")
except ContextLimitError:
    print("Prompt too long for this model")