aman
← all writeups

Traefik ignores your certificateRefs

Traefik picks SNI certificates from its own TLSStore, not from the Gateway's certificateRefs, so new domains got the default cert. A helm upgrade could also wipe every cert from the Gateway.

Date
2026-06-02
Type
note
Impact
New domains served the cluster's default certificate instead of their own. The helm upgrade problem could have broken HTTPS for every hostname on the cluster at once.

A new domain had a valid Let’s Encrypt certificate: issued, Ready, and listed in the Gateway’s websecure listener certificateRefs. Browsers still got the cluster’s default cert, argocd-tls, and a name-mismatch error. By the Gateway API spec, this config was correct.

What actually happens

Traefik implements Gateway API, but SNI certificate selection still goes through its own TLSStore CRD. A cert that is only in the Gateway’s certificateRefs falls through to the default store cert. Every Gateway resource shows Ready, and the wrong cert gets served anyway.

The second problem was worse. The Gateway’s websecure listener and all its certificateRefs are managed by a plain kubectl apply of a separate gateway.yaml. The Traefik chart values only define the web listener. A routine helm upgrade traefik can regenerate the Gateway from the chart template and drop every certificateRef: ArgoCD, SSO, test, beta and Grafana, all at once.

Why

Two different things each own part of the same config, and neither knows about the other.

  • certificateRefs are there for Gateway API compliance. The TLSStore is what Traefik actually uses to choose a cert per hostname.
  • The Gateway object has two writers: helm, which renders it from the chart, and kubectl apply, which adds the TLS listener. Whichever runs last wins. Helm doesn’t know the listener exists.

What to do instead

Each new TLS domain needs three changes:

  1. Add the secret to certificateRefs on the Gateway’s websecure listener.
  2. Add the secret to the TLSStore under extraObjects in the Traefik helm values.
  3. Apply the TLSStore change to the live cluster. Patching is enough:
kubectl patch tlsstore default -n traefik --type='json' \
  -p='[{"op":"add","path":"/spec/certificates/-","value":{"secretName":"<new-secret>"}}]'

And for the helm problem:

  • Routine cert additions use kubectl patch on the TLSStore, not a full helm upgrade.
  • Any Traefik helm upgrade is followed by kubectl apply -f gateway.yaml to restore the listener and its refs.

With Gateway API, check what your implementation actually reads, not just what the spec says it should.

← all writeups