Site Settings

Overview

Your site settings live in /etc/acd/deploy/group_vars/all.yaml. They are the only deployment settings you set: everything that depends on them, such as the ingress hostnames, the TLS secret, Zitadel’s external domain and the list of CDN Directors, is worked out from them, so related values cannot fall out of step.

Every setting appears once in the file, with comments, and already holds a working default. Change a setting by editing its value in place, then run deploy.

A list can be written on one line or one entry per line. Both mean the same:

manager_hostnames: [manager.example.com, cdn.example.com]

manager_hostnames:
  - manager.example.com
  - cdn.example.com

Settings

manager_hostnames

Required. Every hostname that users and CDN Directors use to reach the manager.

manager_hostnames: [manager.example.com]

• The first entry is the main hostname. Your TLS certificate must cover it, and it is the only name users can sign in under: Zitadel rejects sign-ins made under any other name, including a machine’s IP address.
• Each name needs a DNS record pointing at the public address of one or more of the machines.
• Leave the list empty, [], only for a quick lab try-out. A temporary hostname and a self-signed certificate are then used.

external_directors

The CDN Directors that use this manager. They run on their own machines, outside the cluster: deploy installs nothing on them, and the manager routes to them.

external_directors:
  - {name: director-1, address: 192.0.2.201}
  - {name: director-2, address: 192.0.2.202}

• name is your own name for the Director. The manager publishes its configuration under it, for example at https://<manager hostname>/confd/director-1/config/__active/.
• address must be the Director’s IPv4 address, not a hostname. deploy stops with [ACD-DIRECTOR-ADDRESS] otherwise.
• Leave the list empty, [], if you have none yet, then add your Directors later and run deploy again. Until you do, the preset’s placeholder Directors stay in the configuration: director-1 and director-2 at 192.0.2.1 and 192.0.2.2 with the production preset, router-1 at 192.0.2.1 with the lab preset. Those addresses are reserved for documentation and reach nothing.

The Directors reach Kafka on 9092/tcp and 30000-30003/tcp on every machine, with one more port per broker beyond the default three. The host firewall does not filter these ports: restrict them on your network’s own firewall. See Kafka Ports.

Important: Each CDN Director must also be able to reach the manager’s ingress (HTTPS on 443/tcp) under one of the names in manager_hostnames. The ingress answers on every address of every cluster machine, public and backplane alike. So Directors on the backplane network can reach the manager there, instead of over the public network. In that case:

• Give the Directors a hostname that resolves, for them, to the machines’ backplane addresses, and add it to manager_hostnames, after the main hostname.
• Make sure your TLS certificate covers that name too, as a Subject Alternative Name (SAN). Otherwise the Directors’ certificate checks fail. See TLS Certificates.

manage_firewall

Whether deploy configures firewalld for you. The default is true.

• true: deploy starts and enables firewalld (it must be installed) and configures it on every machine. The machines trust each other for all traffic, the public interface is placed in firewalld’s public zone, and the ports clients need are opened. Before the first run, make sure the public zone allows everything else you need on the public interface, SSH included: firewall-cmd --zone=public --list-all should list ssh under services. See Firewall for exactly what is set up.
• false: you configure the firewall yourself. deploy only checks that the required ports are open and the pod and service networks are trusted, and stops before changing anything if not. Only firewalld is checked: if it is not running, the check is skipped, and your own firewall must allow the same traffic.

To switch a running cluster from true to false, follow Managing the Firewall Yourself first.

manage_routes

Whether deploy keeps cluster traffic on the main routing table. The default is true; leave it on.

With true, deploy adds lookup main routing rules at priority 50 for the pod network, the service network, and the network the machines reach each other on. This matters on machines that use policy routing, where traffic from an address is sent through its own routing table: without these rules, cluster traffic can leave through the wrong interface. On machines without policy routing, the rules are harmless.

• deploy owns priority 50 on the NetworkManager profiles of the backplane and public interfaces, and replaces any rule there at that priority. Keep your own rules at other priorities.
• Your own rules that cluster traffic can match, such as from <address> lookup <table>, must have a priority number of 51 or more. deploy stops with [ACD-ROUTING-CONFLICT], naming the rule, otherwise.
• If your rules must stay at 50 or below, set manage_routes: false and add your own lookup main rules for those three networks, at a priority other than 50 and ahead of your source rules. See Managing Routing Yourself for the rules and the commands.

Either way, deploy checks that cluster traffic uses the main routing table before it changes anything.

Locked at First Install

The settings below are marked as a separate section in the file. Set them before the first deploy and never change them afterwards: the cluster is built around them. Changing one later means uninstalling k3s from every machine and installing again. A reinstall starts with an empty cluster: the PostgreSQL databases, Kafka’s messages and everything else stored in the cluster are lost unless you have backed them up.

The same applies to each machine’s node_ip and node_external_ip in the inventory, and to the machines’ hostnames. On a running cluster, deploy compares these with what the cluster uses, and stops before changing anything if they differ.

backplane_network

Optional, but recommended. The address range of the backplane network, the network the cluster machines use to talk to each other, such as "10.10.0.0/24". Leave it empty, "", if you have no backplane: the machines then talk to each other over the public network.

When it is set, deploy finds each machine’s interface in that range and configures the cluster to use it: all node-to-node (east-west) traffic runs over it, including Kubernetes, its pod network and the cluster datastore.

• The backplane should be a separate network from the public network, the one holding the default route. Setting it to the public network’s range works, but then every host on the public network is trusted for all traffic.
• Every machine needs exactly one interface in the range, and must be reachable from the others on it. deploy stops with [ACD-BACKPLANE-NO-MATCH] or [ACD-BACKPLANE-AMBIGUOUS] otherwise.
• The backplane does not have to be reserved for the cluster: other hosts, such as your CDN Directors, may also be on it. With manage_firewall: true, every host on the backplane can reach every port on every cluster machine, including the cluster datastore (etcd, 2379-2380/tcp), the Kubernetes API (6443/tcp) and the kubelet (10250/tcp), so connect only hosts you trust to it.

See Configuring Segregated Networks.

cluster_pod_cidr and cluster_svc_cidr

The address ranges Kubernetes gives its pods and its services. The defaults are 10.42.0.0/16 and 10.43.0.0/16.

Change them only if your network already uses addresses in these ranges. The two must not overlap each other, or any address or route on any machine. deploy stops with [ACD-CIDR-OVERLAP] or [ACD-CIDR-IN-USE] before changing anything if they do.

firewall_cluster_zone and firewall_public_policy

The names of the firewalld zone and policy that deploy keeps as its own. The defaults are acd-cluster and acd-public. The zone trusts the cluster, and the policy opens the ports reached from outside.

deploy removes anything in them that it did not put there, so pick names nothing else on the machines uses. Never use one of firewalld’s own names, such as public or trusted. The zone name can be up to 17 characters and the policy name up to 18, using letters, digits, - and _. The defaults suit most sites.

File Locations

Most sites leave these empty: the files are picked up from the configuration folder.