Connection Recovery
Build resilient clients that automatically reconnect to the jaato server after interruptions. Handle server restarts, crashes, and network issues without losing conversation state.
Why Connection Recovery?
When your client connects to the jaato server via IPC, the connection can be interrupted by:
- Server restarts (updates, configuration changes)
- Server crashes (out-of-memory, unhandled errors)
- Network interrupts (for WebSocket clients)
- System hibernation or sleep
Without recovery, your client would need manual reconnection and would
lose track of the active session. IPCRecoveryClient handles
all of this automatically.
Quick Start
Replace IPCClient with IPCRecoveryClient to get
automatic reconnection. The API is the same—you just get resilience for free.
Key Differences from IPCClient
events()automatically resumes after reconnection- Operations raise
ReconnectingErrorduring recovery instead of crashing - Session reattachment happens automatically after reconnect
reattach_session: true
(the default). After reconnecting, the client sends a session.attach
command with the stored session ID, and the server restores the conversation state.
import asyncio
from jaato_sdk.client import IPCRecoveryClient
async def main():
client = IPCRecoveryClient(
socket_path="/tmp/jaato.sock",
on_status_change=on_status
)
await client.connect()
# Create a session and track it for recovery
# Optionally pass profile= to use an agent profile
session_id = await client.create_session(
"my-session", profile="researcher-claude"
)
client.set_session_id(session_id)
# Use normally — recovery is automatic
await client.send_message("Hello!")
async for event in client.events():
print(event)
await client.close()
def on_status(status):
print(f"Connection: {status.state.value}")
asyncio.run(main())
Connection States
The recovery client tracks its connection through six states. Understanding these helps you build appropriate UI feedback.
| State | Meaning |
|---|---|
DISCONNECTED |
Initial state, or recovery gave up after max attempts |
CONNECTING |
Attempting initial connection or a retry attempt |
CONNECTED |
Active connection, events flowing normally |
RECONNECTING |
Connection lost, waiting for next retry (backoff) |
DISCONNECTING |
Graceful disconnect initiated by client |
CLOSED |
Terminal state — no more connection attempts |
What Your Client Should Do
- CONNECTING — Show "Connecting..." indicator
- CONNECTED — Normal operation, enable input
- RECONNECTING — Show retry status, disable send
- CLOSED — Show "Disconnected", offer manual reconnect
from jaato_sdk.client import (
IPCRecoveryClient,
ConnectionState,
)
client = IPCRecoveryClient("/tmp/jaato.sock")
# Check state directly
if client.state == ConnectionState.CONNECTED:
await client.send_message("Hello!")
# Convenience properties
client.is_connected # True when CONNECTED
client.is_reconnecting # True when RECONNECTING
client.is_closed # True when CLOSED
from jaato_sdk.client import ConnectionState
ConnectionState.DISCONNECTED # "disconnected"
ConnectionState.CONNECTING # "connecting"
ConnectionState.CONNECTED # "connected"
ConnectionState.RECONNECTING # "reconnecting"
ConnectionState.DISCONNECTING # "disconnecting"
ConnectionState.CLOSED # "closed"
Status Callback
The on_status_change callback fires on every state transition.
It receives a ConnectionStatus object with the current state
and recovery progress.
ConnectionStatus Fields
| Field | Type | Description |
|---|---|---|
state |
ConnectionState |
Current connection state |
attempt |
int |
Current reconnection attempt (0 when connected) |
max_attempts |
int |
Maximum attempts before giving up |
next_retry_in |
float | None |
Seconds until next retry (during RECONNECTING) |
last_error |
str | None |
Description of the last connection error |
session_id |
str | None |
Active session ID (if any) |
client_id |
str | None |
Client identifier assigned by server |
from jaato_sdk.client import (
IPCRecoveryClient,
ConnectionState,
)
from jaato_sdk.client.recovery import ConnectionStatus
def on_status(status: ConnectionStatus):
"""Update UI based on connection state."""
if status.state == ConnectionState.CONNECTED:
show_status("Connected")
enable_input()
elif status.state == ConnectionState.RECONNECTING:
msg = (
f"Reconnecting (attempt {status.attempt}"
f"/{status.max_attempts})"
)
if status.next_retry_in is not None:
msg += f" — retry in {status.next_retry_in:.1f}s"
show_status(msg)
disable_input()
elif status.state == ConnectionState.CONNECTING:
show_status("Connecting...")
elif status.state == ConnectionState.CLOSED:
if status.last_error:
show_status(f"Disconnected: {status.last_error}")
else:
show_status("Disconnected")
disable_input()
client = IPCRecoveryClient(
socket_path="/tmp/jaato.sock",
on_status_change=on_status,
)
Handling Operations During Recovery
When the connection is down, operations like send_message()
raise specific exceptions. Your client should handle these to provide
a good user experience.
Exception Types
| Exception | When | Action |
|---|---|---|
ReconnectingError |
Client is reconnecting | Queue and retry after reconnect |
ConnectionClosedError |
Connection permanently closed | Inform user, offer manual reconnect |
IncompatibleServerError |
Server version below client's minimum | Display upgrade message; not retried by recovery client |
close() transitions to the CLOSED state permanently —
no more reconnection attempts. Use disconnect() for temporary
disconnection.
from jaato_sdk.client.recovery import (
ReconnectingError,
ConnectionClosedError,
)
async def send_with_retry(client, message):
"""Send a message, queuing if reconnecting."""
while True:
try:
await client.send_message(message)
return
except ReconnectingError:
# Wait for reconnection, then retry
print("Reconnecting — message queued...")
await wait_for_connected(client)
except ConnectionClosedError:
print("Connection closed permanently.")
raise
async def wait_for_connected(client):
"""Wait until client reconnects."""
while client.is_reconnecting:
await asyncio.sleep(0.5)
if client.is_closed:
raise ConnectionClosedError()
# The events() iterator automatically reconnects.
# It yields events from the new connection
# seamlessly — no special handling needed.
async for event in client.events():
# This keeps working across reconnections.
# You don't need try/except here.
handle_event(event)
Session Reattachment
After reconnecting, the client can reattach to its previous session.
The server loads the session from disk (if evicted from memory) and
sends a SessionInfoEvent with the full session state.
What's Preserved
- Session ID
- Conversation history (persisted on server disk)
- Tool states (managed by server)
What's Lost
- Active IPC connection (replaced by new one)
- In-flight requests (pending permission responses)
- Real-time event stream (restarted after reconnect)
set_session_id() after creating a session —
without it, the recovery client can't reattach after reconnection.
client = IPCRecoveryClient("/tmp/jaato.sock")
await client.connect()
# Create session and track it
# (optionally with an agent profile)
session_id = await client.create_session(
"work", profile="researcher-claude"
)
client.set_session_id(session_id)
# After reconnection, the client automatically
# sends session.attach with this session_id.
# The server restores conversation history.
# If you have a session ID from a previous run
# (e.g., stored in a config file), you can
# reattach manually:
client = IPCRecoveryClient("/tmp/jaato.sock")
await client.connect()
# Attach to existing session
success = await client.attach_session(
"previous-session-id"
)
if success:
# Track it for future reconnections
client.set_session_id("previous-session-id")
Configuration
Recovery behavior is configured via RecoveryConfig.
Configuration is loaded and merged in precedence order:
- Built-in defaults (lowest)
- User config (
~/.jaato/client.json) - Project config (
.jaato/client.json) - Environment variables (highest)
RecoveryConfig Fields
| Field | Default | Description |
|---|---|---|
enabled |
true |
Enable automatic reconnection |
max_attempts |
10 |
Max reconnection attempts |
base_delay |
1.0 |
Initial backoff delay (seconds) |
max_delay |
60.0 |
Maximum backoff delay cap |
jitter_factor |
0.3 |
Random jitter range (0.3 = ±30%) |
connection_timeout |
5.0 |
Timeout per connection attempt |
reattach_session |
true |
Auto-reattach to previous session |
Environment Variables
| Variable | Config Field |
|---|---|
JAATO_IPC_AUTO_RECONNECT |
enabled |
JAATO_IPC_RETRY_MAX_ATTEMPTS |
max_attempts |
JAATO_IPC_RETRY_BASE_DELAY |
base_delay |
JAATO_IPC_RETRY_MAX_DELAY |
max_delay |
JAATO_IPC_RETRY_JITTER |
jitter_factor |
JAATO_IPC_CONNECTION_TIMEOUT |
connection_timeout |
JAATO_IPC_REATTACH_SESSION |
reattach_session |
{
"recovery": {
"enabled": true,
"max_attempts": 10,
"base_delay": 1.0,
"max_delay": 60.0,
"jitter_factor": 0.3,
"connection_timeout": 5.0,
"reattach_session": true
}
}
from jaato_sdk.client import (
IPCRecoveryClient,
RecoveryConfig,
)
# Custom config
config = RecoveryConfig(
max_attempts=5,
base_delay=2.0,
max_delay=30.0,
)
client = IPCRecoveryClient(
socket_path="/tmp/jaato.sock",
config=config,
)
# Or load from config files automatically
from jaato_sdk.client import get_recovery_config
config = get_recovery_config(
workspace_path="/path/to/project"
)
client = IPCRecoveryClient(
socket_path="/tmp/jaato.sock",
config=config,
)
Web Clients (WebSocket)
For WebSocket clients (web UIs, remote connections), the same recovery concepts apply but you'll implement them in your client language. The server-side WebSocket transport supports the same session reattachment protocol.
Key Patterns
- Use a reconnecting WebSocket library
- Track the session ID client-side
- Send
session.attachafter reconnecting - Handle the ping/pong keepalive mechanism
// TypeScript example for web clients
class JaatoWebClient {
private ws: WebSocket | null = null;
private sessionId: string | null = null;
private reconnectAttempt = 0;
private maxAttempts = 10;
connect(url: string) {
this.ws = new WebSocket(url);
this.ws.onopen = () => {
this.reconnectAttempt = 0;
// Reattach session if we had one
if (this.sessionId) {
this.send({
type: "session.attach",
session_id: this.sessionId,
});
}
};
this.ws.onclose = () => {
this.scheduleReconnect(url);
};
}
private scheduleReconnect(url: string) {
if (this.reconnectAttempt >= this.maxAttempts) {
return; // Give up
}
const delay = Math.min(
60_000,
1000 * Math.pow(2, this.reconnectAttempt)
);
this.reconnectAttempt++;
setTimeout(() => this.connect(url), delay);
}
}
Next Steps
- IPCRecoveryClient API Reference — Full method signatures, parameters, and return types
- IPC Recovery Architecture — Deep dive into backoff algorithms, error classification, and state machine design
- Client — Core client concepts and the
JaatoClientAPI - JaatoClient API Reference — Full API documentation
# Imports
from jaato_sdk.client import (
IPCRecoveryClient,
ConnectionState,
RecoveryConfig,
get_recovery_config,
)
from jaato_sdk.client.recovery import (
ConnectionStatus,
ReconnectingError,
ConnectionClosedError,
)
# States
ConnectionState.DISCONNECTED
ConnectionState.CONNECTING
ConnectionState.CONNECTED
ConnectionState.RECONNECTING
ConnectionState.DISCONNECTING
ConnectionState.CLOSED
# Key methods
client.connect() # Start connection
client.disconnect() # Temporary disconnect
client.close() # Permanent close
client.set_session_id(id) # Track for recovery
client.send_message(text) # Send (raises on down)
client.events() # Auto-reconnecting stream
# Properties
client.state # ConnectionState
client.is_connected # bool
client.is_reconnecting # bool
client.is_closed # bool
client.session_id # str | None