1 - deploy
Install, upgrade or reconfigure the cluster, and take backups
Synopsis
deploy init [prod|lab]
deploy [run] [-i <inventory>] [<ansible-playbook option>...]
deploy backup [-i <inventory>] [<ansible-playbook option>...]
deploy -h | --help
Description
deploy is on the installation ISO, at /mnt/esb3027/deploy; it is not on the PATH. Run it as root on the first server in the inventory, with the ISO mounted at /mnt/esb3027. It reaches every machine in the inventory, the first server included, over SSH.
deploy reads its configuration from /etc/acd/deploy, applies it to every machine, and installs or upgrades the cdn-platform Helm release. The first run installs; every later run re-applies the configuration and changes only what differs. Running it again is safe, including after a failed run. It is the only supported way to change what is installed.
When deploy stops on a problem it recognizes, its message starts with a code such as [ACD-TLS-HOSTNAME]. Each code is described in Installation Troubleshooting.
With -h or --help, deploy prints its usage and exits.
Commands
init
Prepares the first server. Run it once before the first deploy, and again after mounting a new ISO elsewhere. It:
- Installs
ansible-core from the operating system’s repositories with dnf, unless it is already installed. - Installs the Ansible collections bundled on the ISO, unless they are already installed.
- Creates
/etc/acd/deploy and every file in it that is missing: see Files. It never overwrites a file that exists. - Points
preset.yaml at the chosen preset on the ISO.
| Argument | Effect |
|---|
prod | Uses the production preset. A new configuration gets prod when no preset is given. |
lab | Uses the lab preset. |
| (none) | On an existing configuration, keeps the current preset and re-points preset.yaml at the mounted ISO. |
deploy init takes no other arguments.
run
deploy [run] [-i <inventory>] [<ansible-playbook option>...]
Runs the deployment. run is the default and can be left out. Every run takes a backup before it changes anything, even with the preflight checks skipped, and stops if the backup fails. The stages of a run are described in Running deploy.
backup
deploy backup [-i <inventory>] [<ansible-playbook option>...]
Takes a backup and changes nothing. It runs only the gather-inputs and backup stages, so it also skips the preflight checks. See Files for what a backup holds.
Options
| Option | Commands | Effect |
|---|
-i <inventory> | run, backup | Uses this inventory file instead of /etc/acd/deploy/inventory.ini. The settings still come from /etc/acd/deploy/group_vars/all.yaml, which must exist. |
-h, --help | all | Prints the usage and exits with status 0. |
--skip-tags preflight | run | Skips the preflight checks. The run still works out every setting and takes the backup. The check of each machine’s network after it is changed cannot be skipped. |
-v, -vv, -vvv | run, backup | More detailed output, for support. |
Every other option is passed to ansible-playbook. These are rejected, because every run must start with gathering inputs and the backup:
| Option | Rejected with |
|---|
--start-at-task, or any prefix of it from --sta | every command |
--skip-tags naming always | every command |
-t, --tags, or any prefix of --tags from --ta | deploy backup |
--limit | run: the run stops with [ACD-PARTIAL-RUN], because what each machine’s firewall trusts depends on all of them |
Environment
| Variable | Effect |
|---|
ACD_CONFIG_DIR | The configuration folder. Default: /etc/acd/deploy. Every path on this page under /etc/acd/deploy moves with it. |
Files
Configuration
In /etc/acd/deploy, created by deploy init and edited by you:
| File | Contents |
|---|
inventory.ini | The machines and their roles. See Inventory. |
group_vars/all.yaml | The site settings. See Site Settings. |
values.override.yaml | Your application settings. See Application Settings. |
preset.yaml | A link to the chosen preset on the ISO. Change it with deploy init prod or deploy init lab. |
certs/ | The TLS certificate and key. See TLS Certificates. |
maxmind/ | The MaxMind databases. See GeoIP Databases. |
README.md | A copy of the configuration reference shipped on the ISO. |
Written by a Run
In /etc/acd/deploy:
| File | Contents |
|---|
deploy.log | Every run’s output, newest last, each under a header with its start time in UTC |
values.yaml | The Helm values the cluster received. Generated: do not edit. See Application Settings. |
decision-record.json | What the run worked out about the cluster and each machine |
probe/<machine>.json | Everything the run read from each machine |
support-bundle-acd.tar.gz | Written only when the final health check fails |
Backups
Each run, and each deploy backup, writes a folder on the first server, <backup_dir>/<UTC time>-cdn-platform/, readable only by root. backup_dir is a site setting and defaults to /var/lib/acd/backups. deploy never deletes a backup.
| File | Contents |
|---|
<machine>.tar.gz | That machine’s files as they were before the run: k3s configuration, unit files and token, network connection profiles, firewall, sysctl and kernel module settings, SELinux configuration, /etc/hosts and routing table names. The first server’s archive also holds /etc/acd/deploy, except certs/, maxmind/ and deploy.log. On a running cluster, one server’s archive also holds an etcd snapshot of the cluster. |
<machine>.files.sha256 | A checksum for every file in that archive |
SHA256SUMS | A checksum for each archive |
manifest.yaml | The release, the machines, the run time and the etcd snapshot’s path |
A backup holds configuration only, never the data stored in the cluster. See Running deploy.
Exit Status
| Status | Meaning |
|---|
0 | The run, backup or initialization finished, or -h printed the usage |
1 | deploy refused to start: an unknown command or bad arguments, a rejected option, no configuration in /etc/acd/deploy (run deploy init), ansible-core missing, a preset.yaml that points at an ISO no longer mounted there, or a failed deploy init |
| other | The exit status of ansible-playbook: the run started and stopped on a failure. The last message names the cause, usually with an [ACD-…] code. |
Examples
Prepare a new lab installation:
/mnt/esb3027/deploy init lab
Install, or apply a changed configuration:
Apply the configuration without the preflight checks:
/mnt/esb3027/deploy --skip-tags preflight
Take a backup and change nothing:
/mnt/esb3027/deploy backup
See Also
2 - acd-admin
Day-to-day cluster operations by admin target name
Synopsis
acd-admin list
acd-admin status [<target>]
acd-admin logs <target> [-f] [-n <lines>] [-p] [-c <container>] [--since <time>] [--until <time>]
acd-admin restart <target> [--timeout <duration> | --no-wait | --node <node>]
acd-admin shell <target> [--pod <pod> | --node <node>] [-c <container>]
acd-admin exec <target> [--pod <pod> | --node <node> | --all] [-c <container>] -- <command> [<argument>...]
acd-admin cordon <node>
acd-admin uncordon <node>
acd-admin drain <node> [--force]
acd-admin [<command>] -h | --help
acd-admin -V | --version
Description
acd-admin manages the running cluster by admin target: a short name such as manager, kafka or postgresql. It works out which Kubernetes objects, labels and namespaces a target means, and handles each kind the safe way for it. acd-admin list prints the valid names. A name that is not a target is an error that lists them.
It is installed on every machine at /usr/local/bin/acd-admin. Run it as root on a server, after deploy has installed the cluster. Anywhere else it stops at once with:
Error: run acd-admin on a server node as root, or set KUBECONFIG.
Before deploy has installed the cdn-platform release, it stops with:
Error: the cdn-platform chart is not installed; run deploy first.
Each command runs in a short-lived pod in the acd-system namespace, started from the release that is installed, so the tool always matches it. The pod is removed when the command ends, Ctrl-C included. A pod left behind by a dropped session can be removed with kubectl delete pod -n acd-system <pod>.
restart, cordon, uncordon, drain, and opening a shell or exec session each record a Kubernetes Event on the affected object, with the command line, the user who logged in to the server and its hostname. acd-admin status <target> shows them. Because the command line is recorded, never pass a password or other secret as an argument.
Bash completion for commands, options, targets and nodes is installed on every machine.
Commands
list
Prints every admin target and its kind, including targets with no pods.
status
acd-admin status [<target>]
With no target, prints one line per target, then one line per node.
| Column | Contents |
|---|
TARGET | The admin target |
READY | Ready pods out of desired pods |
RESTARTS/1H | Containers whose last restart was in the past hour. A container that restarted several times counts once, so this is a lower bound. |
HEALTH | The first that applies: down (pods are desired but none is Ready), rolling (a new version is rolling out), degraded (some desired pods are not Ready), ok |
CHANGED | Time since the target last changed |
NODE | The node |
READY | Whether the node is Ready |
ROLES | The node’s roles |
SCHEDULING | Whether the node is cordoned or drained |
DIRECTOR | Whether the node runs the director. The director in Kubernetes is not supported in this release. |
With a target, prints one line per pod (POD, NODE, READY, RESTARTS, AGE, LAST TERMINATION). For kafka it adds the Strimzi conditions, and for postgresql the CloudNativePG phase and the current primary. It ends with the target’s recent Warning events and the audit Events of acd-admin itself (WHEN, KIND, OBJECT, REASON, MESSAGE).
The output is for reading; there is no machine-readable format.
logs
acd-admin logs <target> [-f] [-n <lines>] [-p] [-c <container>] [--since <time>] [--until <time>]
Shows the logs of every pod and container of the target, init containers excluded. Each line starts with its pod, and its container when the pod has more than one: <pod>[/<container>] <line>. A bounded window is merged in time order across pods.
| Option | Effect |
|---|
-f, --follow | Keeps printing new lines, including from pods created meanwhile, such as during a restart. |
-n, --lines <lines> | Shows only the last <lines> lines of each pod. Default: all. Not with --until. |
-p, --previous | Shows the previous instance of each container, such as one that crashed. |
-c, --container <container> | Shows only this container. |
--since <time> | Shows lines from this time on. |
--until <time> | Shows lines up to this time. Not with --follow. |
<time> is a duration before now, such as 2m, 1h 30m or '10m ago', or an RFC 3339 timestamp in UTC, such as 2026-10-16T12:00:00Z. Only the logs the cluster still holds are available: those of the current and the previous instance of each container.
restart
acd-admin restart <target> [--timeout <duration> | --no-wait | --node <node>]
Restarts every pod of the target, the safe way for its kind: a rolling restart for ordinary workloads, a broker-by-broker rolling update through Strimzi for kafka, and for postgresql a CloudNativePG restart of the replicas first, then a switchover of the primary.
By default it waits, showing progress such as 2/3 pods updated, until the restart is complete. Ctrl-C stops the waiting only: the restart carries on, and acd-admin says so. A target with nothing running is left alone, and acd-admin says so.
| Option | Effect |
|---|
--timeout <duration> | How long to wait, such as 90s or 10m, before exiting non-zero and reporting what is stuck. The restart carries on, and nothing is rolled back. Default: 5m for a Deployment, 10m for any other kind. |
--no-wait | Starts the restart and returns at once. Not with --timeout. |
--node <node> | Restarts only the target’s pod on this node, by deleting it for its controller to recreate. Refused for kafka and postgresql. Not with --timeout or --no-wait. |
shell
acd-admin shell <target> [--pod <pod> | --node <node>] [-c <container>]
Opens an interactive session in one Ready pod of the target:
| Target | Session |
|---|
postgresql | psql on the current primary |
kafka | A shell on a broker, with the Kafka command line tools on the PATH and the bootstrap address set |
| any other | /bin/sh |
| Option | Effect |
|---|
--pod <pod> | Uses this pod. |
--node <node> | Uses the target’s pod on this node. |
-c, --container <container> | Uses this container. |
Without --pod or --node, it uses the first Ready pod, or the primary for postgresql. If no pod of the target is Ready, it refuses and lists the pods.
exec
acd-admin exec <target> [--pod <pod> | --node <node> | --all] [-c <container>] -- <command> [<argument>...]
Runs a command, exactly as given after --, in one pod of the target, chosen as for shell. It gets a terminal only when standard input is one, so output can be redirected or piped.
| Option | Effect |
|---|
--pod <pod> | Runs in this pod. |
--node <node> | Runs in the target’s pod on this node. |
--all | Runs in every pod of the target, one after another, with no terminal and no standard input. Each line starts with its pod. Ends with a list of the pods where the command failed, and exits non-zero if any did. |
-c, --container <container> | Runs in this container. |
cordon
Stops new pods from being scheduled on the node. Pods already there keep running.
uncordon
acd-admin uncordon <node>
Returns a cordoned or drained node to service.
drain
acd-admin drain <node> [--force]
Takes the node out of service for maintenance. It cordons the node, and moves its workloads to other nodes respecting their disruption budgets. Metrics, log shipping, storage and ingress forwarding keep running on the node, so it can still be observed. While it waits, it reports what it is waiting for, such as a storage volume’s last healthy replica. It never moves the acd-admin pod doing the drain.
A drained node stays drained through later deploy runs, until acd-admin uncordon.
| Option | Effect |
|---|
--force | Drains even when it would remove the last Ready director. The director in Kubernetes is not supported in this release. |
Options
Every command accepts:
| Option | Effect |
|---|
-h, --help | Prints the help for acd-admin, or for the command, and exits. |
acd-admin alone also accepts:
| Option | Effect |
|---|
-V, --version | Prints the version and exits. |
Environment
| Variable | Effect |
|---|
KUBECONFIG | The cluster credentials to use. Default: /etc/rancher/k3s/k3s.yaml. |
Exit Status
| Status | Meaning |
|---|
0 | The command succeeded |
1 | The command failed, including a restart that timed out, an exec --all that failed in any pod, and a refused drain |
2 | Invalid command line, such as an unknown option or two options that cannot be combined |
130 | Interrupted with Ctrl-C. An interrupted restart carries on in the cluster. |
Examples
See which targets exist and how they are doing:
acd-admin list
acd-admin status
acd-admin status postgresql
Follow the manager’s logs from ten minutes ago:
acd-admin logs manager -f --since '10m ago'
See why a container crashed:
acd-admin logs manager -p
Restart Kafka, allowing up to 20 minutes:
acd-admin restart kafka --timeout 20m
Restart only the manager’s pod on one node:
acd-admin restart manager --node manager-3
Open psql on the PostgreSQL primary:
acd-admin shell postgresql
Run a command in every Redis pod:
acd-admin exec redis --all -- redis-cli ping
Take a node out of service and return it:
acd-admin drain manager-3
acd-admin uncordon manager-3
See Also
3 - acd-support-dump
Write a redacted diagnostic tarball for AgileTV support
Synopsis
acd-support-dump [<output path>]
Description
acd-support-dump captures a point-in-time diagnostic snapshot of the cluster into one compressed tarball, for attaching to a support ticket. It is installed on every machine at /usr/local/bin/acd-support-dump. Run it as root on a server. It needs no network access beyond the cluster, so it works air-gapped.
The tarball holds pod logs, cluster resources and events, node system logs, database and Longhorn health, metric history and automated triage findings. Secret values are never collected, and the collected data is redacted before the tarball is written. It does not collect database contents. See Support Dumps for the full list.
Progress goes to standard error. The only line written to standard output is the tarball’s path, so a script can capture it.
Arguments
| Argument | Effect |
|---|
<output path> | Where to write the tarball. Default: support-dump-<hostname>-<UTC time>.tar.gz in the current folder, where <UTC time> looks like 20261016T120000Z. |
Environment
| Variable | Effect |
|---|
KUBECONFIG | The cluster credentials to use. Default: /etc/rancher/k3s/k3s.yaml. |
Files
| File | Contents |
|---|
/usr/share/acd/acd-support-dump.yaml | What to collect and how to redact it. Shipped with the release: do not edit. |
Exit Status
| Status | Meaning |
|---|
0 | The tarball was written |
| non-zero | The collection failed, or the collection engine or its specification is missing |
Examples
Write the tarball to the current folder:
Write it to a chosen path:
acd-support-dump /tmp/support-dump.tar.gz
See Also