> ## Documentation Index
> Fetch the complete documentation index at: https://docs.r5d.chat/llms.txt
> Use this file to discover all available pages before exploring further.

# Workspace lifecycle

> How per-user workspaces start, hold leases, scale to zero, and retain their data.

Each user gets one workspace: a Deployment that scales between zero and one,
plus permanent supporting objects. This page covers how it behaves in
production and how to operate it.

## Resource names

Names derive from a SHA-256 hash of the user ID, so they are stable and reveal
nothing about the user:

```
r5d-<first 18 hex chars of sha256(userId)>
```

| Object                   | Name                  |
| ------------------------ | --------------------- |
| Deployment, Service, PVC | `r5d-<hash>`          |
| Identity Secret          | `r5d-<hash>-identity` |

All carry the labels `app.kubernetes.io/managed-by: r5d-chat`,
`app.kubernetes.io/name: r5d-chat-workspace`, `r5d.chat/workspace`, and
`r5d.chat/user-id`.

## Starting a workspace

<Steps>
  <Step title="A request needs the workspace">
    A chat run, file browse, upload, preview, or command triggers it.
  </Step>

  <Step title="Objects are reconciled">
    The PVC, identity Secret, and Service are created if missing, then the
    Deployment is set to one replica. Concurrent requests for the same user
    share one in-flight reconciliation.
  </Step>

  <Step title="Readiness is awaited">
    The control plane polls until the daemon's readiness probe passes, up to 90
    seconds, then returns a 503 with `workspace_start_timeout`.
  </Step>

  <Step title="Work proceeds">
    Requests are proxied to the daemon over the Service with the workspace's
    bearer token.
  </Step>
</Steps>

A cold start typically takes a few seconds, dominated by image pull on first use
and volume attach. Kata adds VM boot time.

## Activity and leases

Two mechanisms keep a busy workspace alive:

* **Activity timestamps** update on every workspace operation.
* **Active leases** are held by long-running work. Starting a shell command
  increments the lease count; polling to completion or cancelling decrements it.

A reconciler runs every 60 seconds and scales to zero only when the desired
state is running, the lease count is zero, and the last activity is older than
`WORKSPACE_IDLE_SECONDS`.

<Note>
  Because a running shell command holds a lease, a long build is never reaped
  mid-task even if it exceeds the idle timeout.
</Note>

## Scaling to zero

Scale-down sets the Deployment to zero replicas. Everything else stays: the
Service, identity Secret, database records, and the PVC with all files. The next
request scales the same Deployment back to one and remounts the same home
directory.

Set `workspace.idleSeconds` according to your tradeoff between cost and cold
starts. 1800 seconds is a reasonable default; interactive-heavy installations
often prefer 3600.

## Inspecting workspaces

```bash theme={null}
# All workspace deployments and their replica counts
kubectl -n r5d-chat get deployment \
  -l app.kubernetes.io/name=r5d-chat-workspace \
  -o custom-columns='NAME:.metadata.name,DESIRED:.spec.replicas,READY:.status.readyReplicas,IMAGE:.spec.template.spec.containers[*].image'

# Home volumes and their sizes
kubectl -n r5d-chat get pvc \
  -l app.kubernetes.io/name=r5d-chat-workspace \
  -o custom-columns='NAME:.metadata.name,STATUS:.status.phase,CAPACITY:.status.capacity.storage,CLASS:.spec.storageClassName'

# Find the workspace for a specific user
kubectl -n r5d-chat get deployment -l "r5d.chat/user-id=<user-id>"
```

Exec into a running workspace to verify its toolchain:

```bash theme={null}
POD="$(kubectl -n r5d-chat get pod \
  -l app.kubernetes.io/name=r5d-chat-workspace \
  -o jsonpath='{.items[0].metadata.name}')"

kubectl -n r5d-chat exec "$POD" -- sh -c \
  'bun --version && python3 --version && ffmpeg -version | head -1'
```

## Storage

Home volumes are `ReadWriteOnce` and mounted whole at `/home/r5d`. The chart
never uses `subPath`, which keeps it compatible with Kata.

`workspace.storageSize` applies when a PVC is created. Changing it affects only
new workspaces. To grow an existing volume, your storage class must allow
expansion:

```bash theme={null}
kubectl -n r5d-chat patch pvc r5d-<hash> \
  -p '{"spec":{"resources":{"requests":{"storage":"20Gi"}}}}'
```

Some drivers require the workload to be scaled to zero first, and some require a
restart to grow the filesystem.

<Warning>
  Workspace PVCs are annotated with `helm.sh/resource-policy: keep`. They
  survive `helm uninstall` and must be deleted explicitly. Deleting one destroys
  that user's files permanently.
</Warning>

## Removing a user's workspace

Deleting workspace data should be a deliberate, audited action:

```bash theme={null}
# 1. Confirm which user you are about to affect
kubectl -n r5d-chat get pvc r5d-<hash> \
  -o jsonpath='{.metadata.labels.r5d\.chat/user-id}'; echo

# 2. Back up first if there is any doubt
# 3. Scale down, then delete the objects
kubectl -n r5d-chat scale deployment r5d-<hash> --replicas=0
kubectl -n r5d-chat delete deployment r5d-<hash>
kubectl -n r5d-chat delete service r5d-<hash>
kubectl -n r5d-chat delete secret r5d-<hash>-identity
kubectl -n r5d-chat delete pvc r5d-<hash>
```

The control plane recreates a clean workspace on that user's next request.

## Troubleshooting

<AccordionGroup>
  <Accordion title="Workspace did not become ready in time">
    The pod failed to become ready within 90 seconds. Check pod events for image
    pull failures, volume attach problems, or insufficient quota:

    ```bash theme={null}
    kubectl -n r5d-chat describe pod -l r5d.chat/workspace=r5d-<hash>
    ```
  </Accordion>

  <Accordion title="ImagePullBackOff on workspace pods only">
    `workspace.imagePullSecret` is unset while the image is private. The control
    plane's own `imagePullSecrets` does not apply to workspace pods.
  </Accordion>

  <Accordion title="Pod stays Pending">
    Usually no node can satisfy the volume or the resource requests. Check that
    the storage class exists and provisions `ReadWriteOnce` volumes, and confirm
    namespace quota headroom.
  </Accordion>

  <Accordion title="Workspaces never scale down">
    Leases may be stuck above zero from processes that never completed, or the
    control plane may be failing its reconcile loop. Check control-plane logs and
    confirm `WORKSPACE_IDLE_SECONDS` is what you expect.
  </Accordion>

  <Accordion title="Agent commands are killed unexpectedly">
    The container exceeded its memory limit. Raise
    `workspace.resources.limits.memory` and confirm with:

    ```bash theme={null}
    kubectl -n r5d-chat get pod <pod> -o jsonpath='{.status.containerStatuses[0].lastState.terminated.reason}'
    ```
  </Accordion>

  <Accordion title="Files disappeared after a restart">
    Files outside `/home/r5d` are not persisted. Only the home directory is on
    the volume; everything else resets when the pod restarts.
  </Accordion>
</AccordionGroup>