> ## Documentation Index
> Fetch the complete documentation index at: https://docs.r5d.chat/llms.txt
> Use this file to discover all available pages before exploring further.

# Troubleshooting

> Diagnose the most common installation, authentication, workspace, and model failures.

## Start here

These four commands identify most problems:

```bash theme={null}
# 1. Is the control plane healthy?
curl -fsS https://chat.example.com/api/health

# 2. Are the pods running?
kubectl -n r5d-chat get pods

# 3. What do the logs say?
kubectl -n r5d-chat logs deployment/r5d-chat-r5d-chat --tail=100

# 4. What has the cluster been doing?
kubectl -n r5d-chat get events --sort-by=.lastTimestamp | tail -30
```

## Installation

<AccordionGroup>
  <Accordion title="Control-plane pod is in CrashLoopBackOff">
    The environment schema is validated at startup, so a missing or invalid
    value stops the process immediately.

    ```bash theme={null}
    kubectl -n r5d-chat logs deployment/r5d-chat-r5d-chat --previous
    ```

    Common causes:

    * `BETTER_AUTH_SECRET` or `SETTINGS_ENCRYPTION_KEY` shorter than 32 characters
    * `BASE_URL` not a valid absolute URL
    * `DATABASE_URL` missing or malformed
  </Accordion>

  <Accordion title="Init container 'migrate' never completes">
    Migrations cannot reach the database. Confirm the database pod is ready and
    the connection string resolves:

    ```bash theme={null}
    kubectl -n r5d-chat get pods -l app.kubernetes.io/name=r5d-chat-postgresql
    kubectl -n r5d-chat logs deployment/r5d-chat-r5d-chat -c migrate
    ```

    With the bundled database, the password inside `DATABASE_URL` must match
    `POSTGRES_PASSWORD` in the same Secret.
  </Accordion>

  <Accordion title="/api/health returns 503">
    The application is up but the database query failed. Check credentials,
    connection limits, and whether the database pod restarted.
  </Accordion>

  <Accordion title="PVC stays Pending">
    The storage class is missing or cannot provision `ReadWriteOnce` volumes.

    ```bash theme={null}
    kubectl get storageclass
    kubectl -n r5d-chat describe pvc <pvc-name>
    ```
  </Accordion>

  <Accordion title="Ingress returns 404 or 502">
    Confirm the ingress class matches your controller, DNS points at it, and the
    Service has endpoints:

    ```bash theme={null}
    kubectl -n r5d-chat get ingress
    kubectl -n r5d-chat get endpoints r5d-chat-r5d-chat
    ```
  </Accordion>
</AccordionGroup>

## Authentication

<AccordionGroup>
  <Accordion title="Login succeeds but immediately returns to the login page">
    The most common misconfiguration. `publicUrl` does not match the URL in the
    browser, so the session cookie is scoped to the wrong host and rejected.
    Align `publicUrl`, `ingress.host`, and DNS, including `https://` versus
    `http://`.
  </Accordion>

  <Accordion title="Setup page still appears after creating an account">
    The setup transaction did not complete. Check logs for a database error, and
    query the state:

    ```bash theme={null}
    curl -fsS https://chat.example.com/api/setup
    ```
  </Accordion>

  <Accordion title="Setup rejects the claim token">
    The token must match `SETUP_CLAIM_TOKEN` in the running pod's environment
    exactly. If you changed the Secret, restart the Deployment so it is re-read.
  </Accordion>

  <Accordion title="OIDC fails with redirect_uri_mismatch">
    The provider's registered redirect URI must be exactly
    `https://<publicUrl-host>/api/auth/oauth2/callback/<provider-key>`.
  </Accordion>

  <Accordion title="All users were signed out unexpectedly">
    `BETTER_AUTH_SECRET` changed. Restore the original value to preserve
    sessions; otherwise users simply sign in again.
  </Accordion>
</AccordionGroup>

## Workspaces

<AccordionGroup>
  <Accordion title="Workspace did not become ready in time">
    The pod missed the 90-second readiness window.

    ```bash theme={null}
    kubectl -n r5d-chat get pods -l app.kubernetes.io/name=r5d-chat-workspace
    kubectl -n r5d-chat describe pod <workspace-pod>
    ```

    Usual causes are a slow first image pull, volume attach delays, exhausted
    quota, or a missing RuntimeClass when `workspace.runtimeClassName` is set.
  </Accordion>

  <Accordion title="ImagePullBackOff on workspace pods only">
    `workspace.imagePullSecret` is not set. The control plane's `imagePullSecrets`
    does not apply to workspace pods, so both must be configured for private
    images.
  </Accordion>

  <Accordion title="Workspaces never scale to zero">
    Either activity is genuinely ongoing, a lease is stuck above zero from a
    process that never finished, or the reconciler is erroring. Check
    control-plane logs and confirm `WORKSPACE_IDLE_SECONDS`.
  </Accordion>

  <Accordion title="Agent commands die without output">
    The container hit its memory limit and was OOM-killed:

    ```bash theme={null}
    kubectl -n r5d-chat get pod <pod> \
      -o jsonpath='{.status.containerStatuses[0].lastState.terminated.reason}'
    ```

    Raise `workspace.resources.limits.memory`.
  </Accordion>

  <Accordion title="Installed tools vanish between sessions">
    Only `/home/r5d` persists. Anything installed into system paths is lost when
    the pod scales to zero. Add durable tooling to the workspace image instead.
  </Accordion>

  <Accordion title="Document search returns nothing">
    Recoll indexes incrementally and may not have indexed recent files yet. Very
    new or unsupported formats also return no matches. Try `grep` for exact
    text.
  </Accordion>
</AccordionGroup>

## Models and providers

<AccordionGroup>
  <Accordion title="Test connection fails immediately">
    The outbound guard blocked the URL. It must be HTTPS and publicly
    resolvable, unless `ALLOW_PRIVATE_EGRESS` or `ALLOW_INSECURE_HTTP` is
    enabled.
  </Accordion>

  <Accordion title="Chat returns a provider authentication error">
    The stored API key is invalid or lacks model access. Keys are write-only;
    re-enter rather than trying to read it back.
  </Accordion>

  <Accordion title="Model not found">
    The provider model ID must match the provider's exact identifier, and the base
    URL should end at `/v1` without repeating the version segment.
  </Accordion>

  <Accordion title="Long conversations start failing">
    The configured context window exceeds the model's real limit. Lower it to the
    documented value.
  </Accordion>

  <Accordion title="Provider settings became unreadable">
    `SETTINGS_ENCRYPTION_KEY` no longer matches the one used to encrypt the data,
    usually after a restore. Restore the original key; otherwise credentials must
    be re-entered.
  </Accordion>

  <Accordion title="No default model is set">
    The application promotes the oldest enabled model automatically when a
    default is removed. If no enabled model exists, add or enable one.
  </Accordion>
</AccordionGroup>

## Helm

<AccordionGroup>
  <Accordion title="helm repo add fails with 404">
    GitHub Pages must be enabled for the repository and at least one release
    must have published an `index.yaml`. Verify directly:

    ```bash theme={null}
    curl -fsSL https://charts.r5d.dev/index.yaml | head
    ```
  </Accordion>

  <Accordion title="helm search finds no versions">
    The local cache is stale. Run `helm repo update r5d-chat`, then `helm search
          repo r5d-chat --versions`.
  </Accordion>

  <Accordion title="OCI pull is unauthorized">
    The package is private. Authenticate with a token that has `read:packages`:

    ```bash theme={null}
    helm registry login ghcr.io -u <username>
    ```
  </Accordion>

  <Accordion title="Upgrade fails and rolls back">
    With `--atomic`, Helm reverts automatically. Inspect why before retrying:

    ```bash theme={null}
    kubectl -n r5d-chat get events --sort-by=.lastTimestamp | tail -30
    kubectl -n r5d-chat logs deployment/r5d-chat-r5d-chat --previous
    ```
  </Accordion>
</AccordionGroup>

## Collecting diagnostics

When reporting an issue, gather non-sensitive context:

```bash theme={null}
{
  echo "== Helm =="
  helm list -n r5d-chat
  helm get values r5d-chat -n r5d-chat

  echo "== Workloads =="
  kubectl -n r5d-chat get pods,svc,ingress,pvc

  echo "== Events =="
  kubectl -n r5d-chat get events --sort-by=.lastTimestamp | tail -50

  echo "== Logs =="
  kubectl -n r5d-chat logs deployment/r5d-chat-r5d-chat --tail=200
} > r5d-chat-diagnostics.txt
```

<Warning>
  Review the output before sharing it. `helm get values` can include credentials
  if you passed them through values instead of `existingSecret`.
</Warning>