Security & threat model
This page describes how hostcd secures communication between the control plane and agents, what an attacker can and cannot do, and how secrets, enrollment and TLS actually work. It states the guarantees — it does not disclose source. Everything here matches the behaviour shipped in the signed releases; it is condensed from the threat model that ships with the product.
Architecture and trust boundaries
Git (GitLab/GitHub) ──signed/reviewed pushes──► hostcd-server
(webhook, outbound) renders per-host state,
holds sealed credentials,
operators ──login (HTTPS)───────────────────────► serves the operator UI
│ outbound only
hosts dial OUT, long-poll HTTPS │ (no inbound to hosts)
hostcd-agent hostcd-agent hostcd-agent (each root, dial-out only)
- Git → server is a trust relationship, not a boundary hostcd can police. hostcd verifies commit signatures when you configure
allowed_signers, but it cannot un-apply a malicious commit a valid signer pushed. Write access to a tracked folder is, in effect, root on every host that folder targets. Protecting the repo is your job — branch protection, required review, signed-commit verification. - Server → host is one-directional at runtime. Hosts never accept connections; the agent only dials out and polls. The single exception is the one-time bootstrap (SSH install / cluster provisioning), after which the link is pull-only again.
- Inside the server host is inside the boundary. Root on the server can read
bootstrap.keyand therefore every sealed credential, and can read rendered manifests. Treat the server host as a crown-jewel box. - An agent trusts the server's manifests. A compromised server is therefore a fleet-wide compromise — the same concentration of trust Argo CD has over a cluster, and the reason the server host and the git repo are the two things to guard hardest.
The questions a security engineer asks
Enrollment — how a new host joins
A host is enrolled by an admin, over the one-time bootstrap SSH channel, which mints a fresh per-host bearer token (stored server-side only as a SHA-256 digest) and installs the agent. SSH host keys are pinned on first use (TOFU); a changed key is refused. A host with no SSH path (a Windows agent) is enrolled with a token minted by the admin API and set on the host. A per-host token authorizes only that host's poll and is revocable on its own — revoking one does not affect the others. There is no static fleet-wide enrollment secret required in steady state.
Long-poll mechanics
The agent opens an outbound HTTPS long-poll to the server and holds it until the server has a change for that host or a server-side timeout elapses, then reconnects — so a change reaches a host in about a second, not a fixed interval, with no inbound port. Bodies are capped (32 MB agent-poll, 1 MB API) and headers have a 10 s read timeout; long-polls and SSE are intentionally long-lived, so rate-limiting belongs in your proxy. It works through a load balancer or proxy because the agent is an ordinary HTTPS client.
Agent permissions
The agent runs as root, because manifest operations (package, service and file management on system paths) require it — exactly as ansible-pull from cron would. It executes only what the server renders from git for its own host. The agent does not accept inbound connections, does not run arbitrary shell outside manifest operations, and exposes no management API on the host. The systemd unit it installs is hardened (ProtectKernelModules, ProtectClock, LockPersonality, RestrictRealtime, PrivateTmp, a 0700 state dir).
Compromised control plane — blast radius
This is the highest-impact scenario, stated plainly: a compromised server can render arbitrary manifests to every connected agent, so it is a fleet-wide compromise. There is no second signer between the server and a host for host manifests. hostcd does not pretend otherwise — the mitigations are operational: treat the server host as crown-jewels (minimal access, patched, monitored), ship the actor-attributed audit trail somewhere durable, and wrap bootstrap.key with Vault Transit to protect data at rest (a live compromised server still decrypts — inherent to any KMS envelope). A compromised agent, by contrast, is one host only: it cannot spread to others through hostcd.
Secrets lifecycle
Sealed credentials at rest on the server — SSH keys used for the one-time install, admin kubeconfigs, webhook signing secrets, the session key, the SAML SP key — are encrypted with AES-256-GCM under bootstrap.key (optionally wrapped by Vault Transit). SSH keys and kubeconfigs are never served to agents. Secrets referenced by a manifest are resolved at the server and delivered inside that host's rendered manifest over HTTPS; a file marked sensitive: true never leaves the host it belongs to. Secrets are not written to git in plaintext.
TLS — default and options
The server should be reached only over HTTPS; install-over-SSH refuses a non-https server (except loopback), and the agent verifies the server certificate. Plaintext HTTP on a non-loopback listener without -tls-cert is warned about, not blocked — terminate TLS directly or at a proxy. Mutual TLS is opt-in (-agent-mtls): with it on, a leaked token alone no longer authorizes a poll (the host also needs its client certificate); with it off, a valid per-host token over verified server TLS is sufficient. We state this honestly rather than claim mTLS-by-default.
Adversaries and how hostcd answers them
| Adversary | What hostcd does |
|---|---|
| Network attacker between agent and server | Install refuses non-https (except loopback); the agent verifies the server cert; optional -agent-mtls means a leaked token alone no longer authorizes a poll. |
| Someone who steals one host's agent token | Per-host tokens are pinned to the host name and cannot pull another host's manifest. App destination matching uses admin-assigned labels, frozen at enrolment — a stolen token cannot relabel its host into another app's (and its secrets') scope. Revoke drops the digest and refuses every token for that host at once. |
| Machine-in-the-middle on the bootstrap SSH | Host key pinned on first use (TOFU); a changed key is refused. |
| Brute-force / credential spray on operator login | ~700 ms delay per failure, per-account lockout (5/15 min) and per-client-address lockout, dummy-hash compare so timing does not reveal account existence. |
| Stolen or stale operator session | Stateless signed cookies with a per-user version; bump it (sign-out, password reset) to kill all of a user's sessions. OIDC/SAML single logout ties session life to the IdP. |
| Low-privilege operator overreach | admin/viewer enforced server-side; viewers scoped to clusters, folders and host-label selectors; the last admin cannot be removed. |
| Malicious repo content | Trees extracted without symlinks; source: and app paths cannot leave their folder; git args cannot start with -; sensitive: true files never leave the host. |
Known limitations and residual risk
Stated so you can decide with eyes open — these are real and intentional-for-now:
- Root on the server reads all sealed secrets. Vault Transit protects data at rest, not a live compromised server. Lock the server host down.
- A compromised server is a fleet compromise. There is no second signer between server and host for host manifests.
- The git push token is powerful — effectively root on every targeted host. Signed-commit verification reduces, but does not eliminate, the value of a stolen push token; branch protection and review are the real controls.
/metricsis unauthenticated by default. SetHOSTCD_METRICS_TOKENif it is reachable by anyone who should not see fleet topology.- Auth can be turned off (
-insecure-no-auth/ empty admin password) for demos and loopback use — never expose such an instance to a network. - Pre-1.0: no per-release security back-ports yet; upgrade to the latest to get a fix.
Operator hardening checklist
- Server reachable only over HTTPS (
-tls-cert/-tls-keyor a TLS proxy). - Repo has branch protection + required review;
allowed_signersset for signed-commit verification. bootstrap.keyis0600, backed up alongsidestate.json, and wrapped with Vault Transit beyond a single box.- The server host is treated as crown-jewels: minimal access, patched, monitored.
- Per-host agent tokens in use; the shared enrollment token left unset once hand-installed agents are gone.
-agent-mtlsenabled if a leaked token must not be enough on its own.- Operators use SSO (OIDC/SAML) and least-privilege viewer scopes; admin password strong and rotated.
- Behind a proxy, set
-trust-proxyand a request-rate limit; setHOSTCD_METRICS_TOKENif/metricsis exposed; ship the audit trail somewhere durable.
Report a vulnerability per the source repository's SECURITY.md. Related: operator security summary · security & fix history.