> ## Documentation Index
> Fetch the complete documentation index at: https://docs.r5d.chat/llms.txt
> Use this file to discover all available pages before exploring further.

# Backup and restore

> What to back up, how the pieces depend on each other, and how to validate a restore.

r5d.chat state spans a database, two secrets, and one volume per user. They must
be treated as a **single recovery set**, because restoring one without the
others produces a broken installation.

## The recovery set

<ResponseField name="1. PostgreSQL database" required>
  Users, sessions, conversations, messages, encrypted provider and MCP
  configuration, workspace records, branding, and audit history.
</ResponseField>

<ResponseField name="2. SETTINGS_ENCRYPTION_KEY" required>
  Decrypts provider API keys, custom headers, MCP headers, OIDC client secrets,
  and workspace identity tokens. Without it, those rows are unreadable
  ciphertext.
</ResponseField>

<ResponseField name="3. BETTER_AUTH_SECRET" required>
  Signs sessions. Restoring without it forces every user to sign in again, which
  is survivable but disruptive.
</ResponseField>

<ResponseField name="4. Workspace PVC snapshots">
  Each user's home directory. Not recoverable from the database.
</ResponseField>

<ResponseField name="5. Registry and integration credentials">
  The GHCR pull secret and any `AWR_*` values needed to bring the installation
  back up.
</ResponseField>

<Warning>
  A database backup on its own is not sufficient. Restoring it with a different
  `SETTINGS_ENCRYPTION_KEY` leaves every user's provider configuration
  permanently unusable, and they must re-enter their credentials.
</Warning>

## Back up the database

For the bundled PostgreSQL, take a logical dump:

```bash theme={null}
POD="$(kubectl -n r5d-chat get pod \
  -l app.kubernetes.io/name=r5d-chat-postgresql \
  -o jsonpath='{.items[0].metadata.name}')"

kubectl -n r5d-chat exec "$POD" -- \
  pg_dump -U r5d_chat -d r5d_chat --format=custom \
  > "r5d-chat-$(date -u +%Y%m%d%H%M%S).dump"
```

Verify the dump is non-trivial and readable before trusting it:

```bash theme={null}
pg_restore --list r5d-chat-*.dump | head
```

For an external managed database, use the provider's snapshot mechanism and its
documented restore procedure.

## Back up the secrets

Export the Secret and store it in a secrets manager, not in Git:

```bash theme={null}
kubectl -n r5d-chat get secret r5d-chat-secrets -o yaml \
  > r5d-chat-secrets.yaml
```

<Note>
  This file contains base64-encoded plaintext credentials. It is as sensitive as
  the database itself. Encrypt it at rest.
</Note>

## Back up workspace volumes

Use your CSI driver's snapshot support. With a `VolumeSnapshotClass` available:

```bash theme={null}
kubectl -n r5d-chat get pvc \
  -l app.kubernetes.io/name=r5d-chat-workspace \
  -o jsonpath='{range .items[*]}{.metadata.name}{"\n"}{end}' |
while read -r pvc; do
  cat <<YAML | kubectl apply -f -
apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeSnapshot
metadata:
  name: ${pvc}-$(date -u +%Y%m%d)
  namespace: r5d-chat
spec:
  volumeSnapshotClassName: csi-rbdplugin-snapclass
  source:
    persistentVolumeClaimName: ${pvc}
YAML
done
```

Snapshots of a live `ReadWriteOnce` volume are crash-consistent. For a
consistent point-in-time copy, scale the workspace to zero first.

Alternatively, copy files out of a running workspace:

```bash theme={null}
kubectl -n r5d-chat exec "$POD" -- tar czf - -C /home/r5d . > home-backup.tgz
```

## Restore

<Steps>
  <Step title="Prepare an empty namespace">
    Restore into a clean namespace rather than on top of a damaged one, so you
    can compare before cutting over.

    ```bash theme={null}
    kubectl create namespace r5d-chat-restore
    ```
  </Step>

  <Step title="Restore the original secrets first">
    The encryption key must be in place before the application reads any
    encrypted row.

    ```bash theme={null}
    sed 's/namespace: r5d-chat/namespace: r5d-chat-restore/' \
      r5d-chat-secrets.yaml | kubectl apply -f -
    ```
  </Step>

  <Step title="Restore the database">
    Install the chart so the database exists, then load the dump:

    ```bash theme={null}
    kubectl -n r5d-chat-restore exec -i "$PG_POD" -- \
      pg_restore -U r5d_chat -d r5d_chat --clean --if-exists \
      < r5d-chat-backup.dump
    ```
  </Step>

  <Step title="Restore workspace volumes">
    Recreate each PVC from its snapshot using the **same name** as the original.
    Names are derived from the user ID hash, so a renamed PVC will not be found.

    ```yaml theme={null}
    apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
      name: r5d-<hash>
      namespace: r5d-chat-restore
    spec:
      accessModes: [ReadWriteOnce]
      storageClassName: rook-ceph-block
      resources:
        requests:
          storage: 10Gi
      dataSource:
        name: r5d-<hash>-20250219
        kind: VolumeSnapshot
        apiGroup: snapshot.storage.k8s.io
    ```
  </Step>

  <Step title="Deploy the matching application version">
    Use the chart version that matches the backup. A newer version may expect a
    schema the restored database does not have; run the upgrade afterwards so
    migrations apply in order.
  </Step>
</Steps>

## Validate the restore

Do not declare success on a green rollout alone.

<AccordionGroup>
  <Accordion title="Authentication" icon="user-check">
    Sign in with a local account and, if configured, through SSO.
  </Accordion>

  <Accordion title="Decryption" icon="lock-open">
    Open **Settings → Providers**. Providers should list correctly and **Test
    connection** should succeed. Failure here means the encryption key does not
    match the database.
  </Accordion>

  <Accordion title="Conversation history" icon="comments">
    Open an older conversation and confirm messages and tool results render.
  </Accordion>

  <Accordion title="Workspace identity and files" icon="folder-open">
    Start a workspace and browse the home directory. Files should match the
    snapshot. Recoll indexes can be rebuilt from the files if search is stale.
  </Accordion>

  <Accordion title="A real agent task" icon="robot">
    Run a task that writes a file and returns a download link, end to end.
  </Accordion>
</AccordionGroup>

## Retention and deletion

`helm uninstall` is not a data purge. It removes chart-managed compute and
routing, while these persist:

* The bundled database's StatefulSet PVC
* Every workspace PVC, which carries keep semantics
* Database rows, encrypted credentials, and conversations if the database is
  external

Before uninstalling, take an inventory and a final backup:

```bash theme={null}
kubectl -n r5d-chat get pvc
helm uninstall r5d-chat --namespace r5d-chat
kubectl -n r5d-chat get all,pvc,secret
```

<Warning>
  Deleting a namespace destroys everything still namespaced in it, including
  retained PVCs. Never use namespace deletion as a shortcut, and never as an
  accidental purge.
</Warning>

Test your restore procedure on a schedule. An untested backup is a hypothesis.