Scope and release status
Private automations are available to every account by default. The chart enables durable runs, schedules, webhooks, bounded inference, and the executor. Each automation still requires its owner’s approval. Private services remain a separate opt-in facility. The platform accepts a bounded, authenticated webhook and returns a fixed asynchronous acknowledgement. It never returns tenant code, proxies a workspace port, serves tenant HTML, or offers a public result/inference API. Workspaces retain only the private authenticated daemon transport; do not add tenant Ingress, NodePort, LoadBalancer, port-forwarding or tunnel product features. Ingress policy does not prevent arbitrary same-user shell code from making outbound relays.Packaging and enablement
The control-plane image also includes.output/automations/executor.mjs and migrate.mjs. The executor is an independent Deployment, not a timer started by an HTTP import. It has no Service or public health listener. It reuses the control plane’s existing Secret and namespace-scoped workspace permissions. Do not inject provider or infrastructure credentials into automation command environments.
The following values default to true: automations.durableRuns, enabled, schedules, webhooks, inference, and allOwners. Every current and future account has access without maintaining an allowlist. This does not bypass authentication, enabled-account checks, grants, quotas, ownership, or version-specific approval. automations.ownerIds defaults to []; to deliberately restrict access, set allOwners: false and list explicit owner IDs. privateServices.enabled defaults to false and additionally requires privateServices.capacityApproved (runtime R5DCHAT_SERVICE_CAPACITY=1).
Outside Helm, the control plane uses the same enabled defaults when the corresponding R5DCHAT_* variables are absent. Explicit values must be exactly true to enable a capability; false disables it. A nonempty R5DCHAT_AUTOMATION_OWNER_IDS restricts access unless R5DCHAT_AUTOMATION_ALL_OWNERS=true is also set. Existing explicit overrides survive upgrades: remove old opt-out settings or set the values above to true when enabling an older installation.
executor.enabled defaults to true and creates the dedicated process; executor.roles defaults to executor,triggers,retention; add services explicitly for private-service authorization renewal. There is no fixed shared executor run cap or per-owner/per-automation active or queued run cap. Ordinary chat is a conversation-owned agent loop, not an automation run per message. Each conversation has one fenced loop owner; independent conversations run concurrently without numeric caps. Inputs arriving while its loop is active remain in the agent inbox or steer it at a tool/model boundary. Tools have no read/exclusive or owner-wide queue; only same-owner writes/edits to the same normalized lexical path are ordered within a server process. This is not a cross-replica filesystem lock. Daily quotas, per-run bounds, authorization and resource limits still apply. Default poll interval is 1000ms, drain deadline 30000ms, termination grace 45s. These are process knobs, not throughput promises. Use resource requests/limits independent of the web deployment. SIGTERM stops claims and drains; exceeding the deadline must surface unknown dispatched effects rather than replay them. Deployment readiness is only process health, not evidence that an external action succeeded.
automations.policy is a fixed ceiling declaration checked against the protocol, not editable working quota configuration. Changed values fail chart rendering. A reviewed code/protocol change and tests are required to change these ceilings:
Provider cost may be unavailable; token ceilings are not exact monetary billing. Verify owner/IP/endpoint throttling and fair multi-owner progress under your workload, not just a single-owner benchmark. No per-service cgroup sandbox or exact CPU isolation is promised; same-user root can inspect other same-user files and processes.
Daily checks and retention
Track oldest queued run/trigger age, queue depth per owner, webhook accepted/rejected/429/503 counts, attempts, stale leases, unknown outcomes, cancellation latency, executor RSS/CPU, workspace wake latency, PG connections/locks, PVC free bytes, and retention sweep errors. Never record webhook bodies, secrets, prompts or browser profiles as metric labels. Alert on sustained growth, capacity approaching a ceiling, or a missed cleanup window; numeric SLO thresholds require measured staging evidence. Events expire before results. An expired event cursor returnscursor_expired; fetch retained status rather than restarting the action. Removing visible history must retain request digests, idempotency keys and provider occurrence/event tombstones through their retry horizon. A replay after that horizon requires a separately reviewed decision. Do not use TRUNCATE, cascading history deletion, or manual tombstone removal to fix disk pressure.
Keep logs and payloads bounded, private, redacted and outside artifact publication or broad search/indexing. Secret values are encrypted with the settings encryption key; rotate/revoke through owner controls, not log export. Browser state and mutable SQLite belong in the private workspace home, separate from immutable release files. Audit access/export/deletion; access to a project’s shared chat does not confer ownership of private credentials or automation state. Legal erasure must explicitly reconcile retained dedupe metadata and backups instead of silently allowing immediate replay.
Recovery decision table
Killing a process cannot undo external effects. Duplicate notifications after a provider accepted a send but lost its response are possible independently of booking deduplication. A successful process exit is not proof of a booking: require remote confirmation evidence.
Backup and paused restore
Back up PostgreSQL, workspace homes (including SQLite WAL/checkpoints and daemon journals), artifact storage, and the encryption key as a coordinated recovery set. Quiesce admission/dispatch and application writers or use an application-consistent SQLite backup; a raw copy of a live DB file alone is insufficient. Encrypt backups, restrict access, test integrity and document timestamps, image versions and RPO/RTO measurements. Never publish secret-bearing diagnostics as artifacts. Restore into an isolated namespace with all feature flags off, empty owner allowlist and no executor or private services. Before starting any restored workspace, persist automations paused, schedules not enabled, webhook endpoints non-executing, services desired stopped with expired authorization, and queued/running work quarantined for reconciliation. Invalidate restored delegated tokens, preserve tombstones, and verify no old service authorization is accepted. An environment kill switch alone is not a durable paused restore. Compare external reservation/action state with restored intent/results; a backup predating remote success cannot determine that success did not occur. Never automatically replay queued events, missed schedules or unknown commands. Have the owner review fresh triggers and new approvals after reconciliation. Do not restore old dependency releases into an already-running workspace.Emergency disable and rollback
- Persist owner pause/disable and stop desired services; use the operator kill switch for new admission/dispatch. It is not an instant remote process kill.
- Persist cancellation for active runs, observe process-group shutdown, and quarantine unknown outcomes. Observe service authorization expiry; inspect actual processes.
- Drain and scale down the executor; disable triggers/inference/services, set
automations.allOwners: false, clear the owner allowlist, then disable automation/durable flags. Render the exact rollback manifests and validate them before applying. - Roll back to a recorded compatible image/chart only after checking schema/protocol compatibility. Keep additive migrations, run/operation history and dedupe state; never apply destructive down-migrations.
- Verify no new dispatch, no public workspace route, retained owner read/export/disable access, and no restored service restart. Re-enable only with an audited reconciliation and separate capacity decision.
bun scripts/test-automation-chart.ts and the explicit bun --no-env-file scripts/test-automations.ts integration harness before approving an upgrade. The latter creates its own disposable PostgreSQL and rejects missing or skipped required suites; it is distinct from legacy empty integration commands.