LM Studio Provider

Run AI models locally through LM Studio's built-in server. No API costs, complete privacy, and access to thousands of models from the LM Studio catalog. Uses an OpenAI-compatible chat API plus LM Studio's native endpoints for model catalog and load control.

Provider Name	`lmstudio`
Module	`shared.plugins.model_provider.lmstudio`
SDK	`openai` (OpenAI-compatible API) + `httpx` (native endpoints)
Auth	None by default; optional bearer token when LM Studio's "Require API Token" is enabled

Highlights

Zero API costs — Run any model LM Studio supports locally
Privacy — Data never leaves your machine
Model catalog — Native /api/v0/models reports each model's real max_context_length
Load control — Optionally POST a load config to /api/v1/models/load at session start
Function calling — Full tool use via the OpenAI-compatible endpoint
Streaming — Real-time streaming with cancellation support

Two Operating Modes

Passive mode (default): the provider uses whatever model you have already loaded in the LM Studio UI or via lms load. Load-control mode: supply a load dict in your profile and the provider POSTs it to /api/v1/models/load before the first chat call, reconfiguring context length, GPU offload, and more.

Quick start

from jaato import JaatoClient

client = JaatoClient(provider_name="lmstudio")
client.connect(
    project=None,
    location=None,
    model="openai/gpt-oss-20b"
)
client.configure_tools(registry)

response = client.send_message(
    "Hello from LM Studio!",
    on_output=on_output
)

Prerequisites

# 1. Download LM Studio: https://lmstudio.ai
# 2. Open LM Studio and load a model
# 3. Start the local server (default port 1234):
#    Developer tab → Start Server

# Verify the server is running
curl http://localhost:1234/api/v0/models

Configuration

Environment Variables

Variable	Default	Description
`LMSTUDIO_HOST`	`http://localhost:1234`	LM Studio server URL
`LMSTUDIO_MODEL`	—	Default model identifier
`LMSTUDIO_CONTEXT_LENGTH`	Discovered from catalog	Override context window size (integer)
`LMSTUDIO_API_TOKEN`	—	Bearer token (only when LM Studio's "Require API Token" is enabled)

Context Length Auto-Detection

The provider queries /api/v0/models at connect time and reads each model's max_context_length automatically. LMSTUDIO_CONTEXT_LENGTH is only needed when the catalog does not report the field (unusual) or to pin a specific value. Fail-fast: if the context window cannot be resolved from either source, the provider raises rather than silently using a hardcoded fallback.

Profile Knobs (`plugin_configs.lmstudio`)

Key	Type	Description
`host`	str	Override `LMSTUDIO_HOST`
`context_length`	int	Override context window size (tier-2 after catalog auto-detect)
`api_token`	str	Bearer token override (when "Require API Token" is enabled)
`load`	dict	Passthrough body for `POST /api/v1/models/load`; absent = passive mode

Environment configuration

# .env file
JAATO_PROVIDER=lmstudio
LMSTUDIO_HOST=http://localhost:1234
LMSTUDIO_MODEL=openai/gpt-oss-20b

Remote LM Studio server

# Connect to LM Studio on another machine
LMSTUDIO_HOST=http://192.168.1.50:1234

With authentication enabled

# When "Require API Token" is enabled in LM Studio
LMSTUDIO_API_TOKEN=your-token-here

Load Control

When a session profile supplies a load dict under plugin_configs.lmstudio, the provider POSTs it to POST /api/v1/models/load before the first chat call. This lets you configure the in-memory model instance programmatically with context length, GPU offload, KV-cache placement, and more.

Load Parameters

All keys are passed through to LM Studio unchanged. LM Studio uses snake_case natively. Common knobs:

Key	Description
`context_length`	Context window size in tokens
`eval_batch_size`	Evaluation batch size
`flash_attention`	Enable Flash Attention (bool)
`offload_kv_cache_to_gpu`	Move KV cache to GPU VRAM (bool)
`num_experts`	Active experts for MoE models
`echo_load_config`	Log resolved load config (bool, for debugging)

Idempotent Load

The provider checks GET /api/v1/models before POSTing to /load. If a loaded instance already matches every requested config key, the POST is skipped — no spurious new instances are created.

Profile with load control (YAML)

name: local-gpt-oss
model: openai/gpt-oss-20b
provider: lmstudio

plugins:
  - cli
  - file_edit

plugin_configs:
  lmstudio:
    host: "http://localhost:1234"
    load:
      context_length: 16384
      flash_attention: true
      offload_kv_cache_to_gpu: true
      eval_batch_size: 512

Passive mode (no load dict)

name: local-passive
model: qwen/qwen2.5-coder-14b
provider: lmstudio

plugin_configs:
  lmstudio:
    host: "http://localhost:1234"
    # No 'load' key: uses whatever you loaded in the LM Studio UI

API Endpoints Used

The provider interacts with three distinct LM Studio surfaces, all on the same host:

Endpoint	Purpose
`POST /v1/chat/completions`	Chat (OpenAI-compatible, routed through `openai` SDK)
`GET /api/v0/models`	Model catalog: lists available models with `max_context_length`
`GET /api/v1/models`	Live instance catalog: loaded instances with their active config (used for load-reuse decisions)
`POST /api/v1/models/load`	Load/reconfigure model with specified parameters (load-control mode only)

Context Length Resolution

The provider reads max_context_length from /api/v0/models and, for loaded instances, the live config.context_length from /api/v1/models. The live instance config takes priority since it reflects what was actually configured (e.g., via a previous /load call or the LM Studio UI).

List available models

from shared.plugins.model_provider.lmstudio.provider import (
    LMStudioProvider
)
from shared.plugins.model_provider.base import ProviderConfig

provider = LMStudioProvider()
provider.initialize(ProviderConfig())

# List all models in catalog
models = provider.list_models()
print(models)
# ['lmstudio-community/Meta-Llama-3.1-8B-Instruct-GGUF',
#  'openai/gpt-oss-20b', ...]

# Connect and check context limit
provider.connect("openai/gpt-oss-20b")
print(provider.get_context_limit())
# 32768 (auto-detected from catalog)

Connectivity check

# The provider probes /api/v0/models at initialize() time.
# Verify manually:
curl http://localhost:1234/api/v0/models

Error Handling

Exception	Cause
`LMStudioConnectionError`	Server unreachable (not running, wrong port, firewall)
`LMStudioAuthenticationError`	Bearer token rejected (401); or "Require API Token" is on but no token set
`LMStudioModelNotFoundError`	Requested model is not in LM Studio's catalog
`LMStudioLoadError`	`/api/v1/models/load` returned a non-2xx status

Handle errors

from shared.plugins.model_provider.lmstudio.errors import (
    LMStudioConnectionError,
    LMStudioModelNotFoundError,
    LMStudioLoadError,
)

try:
    provider.initialize(config)
    provider.connect("openai/gpt-oss-20b")
except LMStudioConnectionError as e:
    print(f"LM Studio server not reachable: {e}")
    print("Is the server running? Developer tab → Start Server")
except LMStudioModelNotFoundError as e:
    print(f"Model not in catalog: {e}")
    print("Load the model in LM Studio first.")
except LMStudioLoadError as e:
    print(f"Load config rejected: {e}")

LM Studio vs Ollama

Feature	LM Studio	Ollama
GUI	Yes (desktop app)	No (CLI only)
Model catalog	In-app download + HuggingFace	Ollama library (`ollama pull`)
Load control	Yes (`/api/v1/models/load`)	No (passive only)
Context auto-detect	Yes (`/api/v0/models`)	No (manual override)
Default port	1234	11434
Auth support	Optional bearer token	None

Choose based on needs

# LM Studio: GUI-first workflow, load control needed
client = JaatoClient(provider_name="lmstudio")
client.connect(None, None, "openai/gpt-oss-20b")

# Ollama: headless/server, simpler setup
client = JaatoClient(provider_name="ollama")
client.connect(None, None, "qwen3:32b")