Connection Recovery

Build resilient clients that automatically reconnect to the jaato server after interruptions. Handle server restarts, crashes, and network issues without losing conversation state.

Why Connection Recovery?

When your client connects to the jaato server via IPC, the connection can be interrupted by:

  • Server restarts (updates, configuration changes)
  • Server crashes (out-of-memory, unhandled errors)
  • Network interrupts (for WebSocket clients)
  • System hibernation or sleep

Without recovery, your client would need manual reconnection and would lose track of the active session. IPCRecoveryClient handles all of this automatically.

State machine overview
DISCONNECTED CONNECTING CONNECTED lost RECONNECTING retry CLOSED max DISCON- NECTING

Quick Start

Replace IPCClient with IPCRecoveryClient to get automatic reconnection. The API is the same—you just get resilience for free.

Key Differences from IPCClient

  • events() automatically resumes after reconnection
  • Operations raise ReconnectingError during recovery instead of crashing
  • Session reattachment happens automatically after reconnect
Session Reattachment
Session reattachment is automatic when reattach_session: true (the default). After reconnecting, the client sends a session.attach command with the stored session ID, and the server restores the conversation state.
Minimal recovery client
import asyncio
from jaato_sdk.client import IPCRecoveryClient

async def main():
    client = IPCRecoveryClient(
        socket_path="/tmp/jaato.sock",
        on_status_change=on_status
    )

    await client.connect()

    # Create a session and track it for recovery
    # Optionally pass profile= to use an agent profile
    session_id = await client.create_session(
        "my-session", profile="researcher-claude"
    )
    client.set_session_id(session_id)

    # Use normally — recovery is automatic
    await client.send_message("Hello!")

    async for event in client.events():
        print(event)

    await client.close()

def on_status(status):
    print(f"Connection: {status.state.value}")

asyncio.run(main())

Connection States

The recovery client tracks its connection through six states. Understanding these helps you build appropriate UI feedback.

State Meaning
DISCONNECTED Initial state, or recovery gave up after max attempts
CONNECTING Attempting initial connection or a retry attempt
CONNECTED Active connection, events flowing normally
RECONNECTING Connection lost, waiting for next retry (backoff)
DISCONNECTING Graceful disconnect initiated by client
CLOSED Terminal state — no more connection attempts

What Your Client Should Do

  • CONNECTING — Show "Connecting..." indicator
  • CONNECTED — Normal operation, enable input
  • RECONNECTING — Show retry status, disable send
  • CLOSED — Show "Disconnected", offer manual reconnect
Checking state
from jaato_sdk.client import (
    IPCRecoveryClient,
    ConnectionState,
)

client = IPCRecoveryClient("/tmp/jaato.sock")

# Check state directly
if client.state == ConnectionState.CONNECTED:
    await client.send_message("Hello!")

# Convenience properties
client.is_connected      # True when CONNECTED
client.is_reconnecting   # True when RECONNECTING
client.is_closed         # True when CLOSED
ConnectionState enum values
from jaato_sdk.client import ConnectionState

ConnectionState.DISCONNECTED   # "disconnected"
ConnectionState.CONNECTING     # "connecting"
ConnectionState.CONNECTED      # "connected"
ConnectionState.RECONNECTING   # "reconnecting"
ConnectionState.DISCONNECTING  # "disconnecting"
ConnectionState.CLOSED         # "closed"

Status Callback

The on_status_change callback fires on every state transition. It receives a ConnectionStatus object with the current state and recovery progress.

ConnectionStatus Fields

Field Type Description
state ConnectionState Current connection state
attempt int Current reconnection attempt (0 when connected)
max_attempts int Maximum attempts before giving up
next_retry_in float | None Seconds until next retry (during RECONNECTING)
last_error str | None Description of the last connection error
session_id str | None Active session ID (if any)
client_id str | None Client identifier assigned by server
Status callback implementation
from jaato_sdk.client import (
    IPCRecoveryClient,
    ConnectionState,
)
from jaato_sdk.client.recovery import ConnectionStatus

def on_status(status: ConnectionStatus):
    """Update UI based on connection state."""
    if status.state == ConnectionState.CONNECTED:
        show_status("Connected")
        enable_input()

    elif status.state == ConnectionState.RECONNECTING:
        msg = (
            f"Reconnecting (attempt {status.attempt}"
            f"/{status.max_attempts})"
        )
        if status.next_retry_in is not None:
            msg += f" — retry in {status.next_retry_in:.1f}s"
        show_status(msg)
        disable_input()

    elif status.state == ConnectionState.CONNECTING:
        show_status("Connecting...")

    elif status.state == ConnectionState.CLOSED:
        if status.last_error:
            show_status(f"Disconnected: {status.last_error}")
        else:
            show_status("Disconnected")
        disable_input()

client = IPCRecoveryClient(
    socket_path="/tmp/jaato.sock",
    on_status_change=on_status,
)

Handling Operations During Recovery

When the connection is down, operations like send_message() raise specific exceptions. Your client should handle these to provide a good user experience.

Exception Types

Exception When Action
ReconnectingError Client is reconnecting Queue and retry after reconnect
ConnectionClosedError Connection permanently closed Inform user, offer manual reconnect
IncompatibleServerError Server version below client's minimum Display upgrade message; not retried by recovery client
close() Is Permanent
close() transitions to the CLOSED state permanently — no more reconnection attempts. Use disconnect() for temporary disconnection.
Queue-and-retry pattern
from jaato_sdk.client.recovery import (
    ReconnectingError,
    ConnectionClosedError,
)

async def send_with_retry(client, message):
    """Send a message, queuing if reconnecting."""
    while True:
        try:
            await client.send_message(message)
            return
        except ReconnectingError:
            # Wait for reconnection, then retry
            print("Reconnecting — message queued...")
            await wait_for_connected(client)
        except ConnectionClosedError:
            print("Connection closed permanently.")
            raise

async def wait_for_connected(client):
    """Wait until client reconnects."""
    while client.is_reconnecting:
        await asyncio.sleep(0.5)
    if client.is_closed:
        raise ConnectionClosedError()
events() handles reconnection automatically
# The events() iterator automatically reconnects.
# It yields events from the new connection
# seamlessly — no special handling needed.

async for event in client.events():
    # This keeps working across reconnections.
    # You don't need try/except here.
    handle_event(event)

Session Reattachment

After reconnecting, the client can reattach to its previous session. The server loads the session from disk (if evicted from memory) and sends a SessionInfoEvent with the full session state.

What's Preserved

  • Session ID
  • Conversation history (persisted on server disk)
  • Tool states (managed by server)

What's Lost

  • Active IPC connection (replaced by new one)
  • In-flight requests (pending permission responses)
  • Real-time event stream (restarted after reconnect)
Always Call set_session_id()
Always call set_session_id() after creating a session — without it, the recovery client can't reattach after reconnection.
Session tracking
client = IPCRecoveryClient("/tmp/jaato.sock")
await client.connect()

# Create session and track it
# (optionally with an agent profile)
session_id = await client.create_session(
    "work", profile="researcher-claude"
)
client.set_session_id(session_id)

# After reconnection, the client automatically
# sends session.attach with this session_id.
# The server restores conversation history.
Manual reattachment
# If you have a session ID from a previous run
# (e.g., stored in a config file), you can
# reattach manually:

client = IPCRecoveryClient("/tmp/jaato.sock")
await client.connect()

# Attach to existing session
success = await client.attach_session(
    "previous-session-id"
)
if success:
    # Track it for future reconnections
    client.set_session_id("previous-session-id")

Configuration

Recovery behavior is configured via RecoveryConfig. Configuration is loaded and merged in precedence order:

  1. Built-in defaults (lowest)
  2. User config (~/.jaato/client.json)
  3. Project config (.jaato/client.json)
  4. Environment variables (highest)

RecoveryConfig Fields

Field Default Description
enabled true Enable automatic reconnection
max_attempts 10 Max reconnection attempts
base_delay 1.0 Initial backoff delay (seconds)
max_delay 60.0 Maximum backoff delay cap
jitter_factor 0.3 Random jitter range (0.3 = ±30%)
connection_timeout 5.0 Timeout per connection attempt
reattach_session true Auto-reattach to previous session

Environment Variables

Variable Config Field
JAATO_IPC_AUTO_RECONNECT enabled
JAATO_IPC_RETRY_MAX_ATTEMPTS max_attempts
JAATO_IPC_RETRY_BASE_DELAY base_delay
JAATO_IPC_RETRY_MAX_DELAY max_delay
JAATO_IPC_RETRY_JITTER jitter_factor
JAATO_IPC_CONNECTION_TIMEOUT connection_timeout
JAATO_IPC_REATTACH_SESSION reattach_session
.jaato/client.json
{
  "recovery": {
    "enabled": true,
    "max_attempts": 10,
    "base_delay": 1.0,
    "max_delay": 60.0,
    "jitter_factor": 0.3,
    "connection_timeout": 5.0,
    "reattach_session": true
  }
}
Config in code
from jaato_sdk.client import (
    IPCRecoveryClient,
    RecoveryConfig,
)

# Custom config
config = RecoveryConfig(
    max_attempts=5,
    base_delay=2.0,
    max_delay=30.0,
)

client = IPCRecoveryClient(
    socket_path="/tmp/jaato.sock",
    config=config,
)

# Or load from config files automatically
from jaato_sdk.client import get_recovery_config

config = get_recovery_config(
    workspace_path="/path/to/project"
)
client = IPCRecoveryClient(
    socket_path="/tmp/jaato.sock",
    config=config,
)

Web Clients (WebSocket)

For WebSocket clients (web UIs, remote connections), the same recovery concepts apply but you'll implement them in your client language. The server-side WebSocket transport supports the same session reattachment protocol.

Key Patterns

  • Use a reconnecting WebSocket library
  • Track the session ID client-side
  • Send session.attach after reconnecting
  • Handle the ping/pong keepalive mechanism
WebSocket reconnection (TypeScript)
// TypeScript example for web clients

class JaatoWebClient {
  private ws: WebSocket | null = null;
  private sessionId: string | null = null;
  private reconnectAttempt = 0;
  private maxAttempts = 10;

  connect(url: string) {
    this.ws = new WebSocket(url);

    this.ws.onopen = () => {
      this.reconnectAttempt = 0;
      // Reattach session if we had one
      if (this.sessionId) {
        this.send({
          type: "session.attach",
          session_id: this.sessionId,
        });
      }
    };

    this.ws.onclose = () => {
      this.scheduleReconnect(url);
    };
  }

  private scheduleReconnect(url: string) {
    if (this.reconnectAttempt >= this.maxAttempts) {
      return; // Give up
    }

    const delay = Math.min(
      60_000,
      1000 * Math.pow(2, this.reconnectAttempt)
    );
    this.reconnectAttempt++;

    setTimeout(() => this.connect(url), delay);
  }
}

Next Steps

Quick reference
# Imports
from jaato_sdk.client import (
    IPCRecoveryClient,
    ConnectionState,
    RecoveryConfig,
    get_recovery_config,
)
from jaato_sdk.client.recovery import (
    ConnectionStatus,
    ReconnectingError,
    ConnectionClosedError,
)

# States
ConnectionState.DISCONNECTED
ConnectionState.CONNECTING
ConnectionState.CONNECTED
ConnectionState.RECONNECTING
ConnectionState.DISCONNECTING
ConnectionState.CLOSED

# Key methods
client.connect()           # Start connection
client.disconnect()        # Temporary disconnect
client.close()             # Permanent close
client.set_session_id(id)  # Track for recovery
client.send_message(text)  # Send (raises on down)
client.events()            # Auto-reconnecting stream

# Properties
client.state               # ConnectionState
client.is_connected        # bool
client.is_reconnecting     # bool
client.is_closed           # bool
client.session_id          # str | None