Dependencies¶
Dependency versions live in the files that consume them: mise configuration,
bootstrap.toml, Cargo manifests and locks, the toolbox Dockerfile,
Kubernetes and Flux manifests, GitHub Actions workflows, and the air-gap image
inventory. There is no central version catalog.
Renovate discovers the pins through renovate.json5 and
opens weekly update PRs through the hosted Renovate GitHub App. Pending and
proposed updates appear in the Renovate dependency dashboard issue.
Managed surfaces¶
Renovate discovers and updates versions in:
mise.toml,mise.aws.toml,mise.azure.toml,mise.gcp.toml,mise.local-host.toml, andmise.local-talos.toml: tool pins and the Zarf CLI pin, plus the azure-cli pipx pin (mise.azure.toml) and the gcloud pin (mise.gcp.toml, resolved via the gcr.io/google-cloud-cli docker datasource). Explicit per-tool custom managers replace the native mise manager so each pin resolves against the intended upstream project.pyproject.tomlanduv.lock: documentation site dependencies (mkdocs-material, pytest) through the native Python managers.bootstrap-rs/Cargo.tomlandbootstrap-rs/Cargo.lock: Rust crate dependencies through Renovate's Cargo manager.bootstrap.toml: the Flux Operator, cert-manager, and CAPI Operator chart pins consumed bykrops-bootstrap. One annotation-driven custom manager reads the adjacent# renovate:metadata.mise run validatecross-checks these pins against their declarative Helm releases and proxies. cert-manager's pin, itspivot.sh/HelmRelease counterparts, itsairgap/zarf.yamlchart pin, and its fourairgap/images.txt/airgap/zarf.yamlimage tags all share theplatform-chartsgroup so they can't drift apart (issue #322). The Helm index publishes cert-manager versions asv1.21.x. The pins inbootstrap.toml,pivot.sh, and the HelmReleases are spelled without the leadingv;airgap/zarf.yamlkeeps thevas a literal in its annotation-driven line (the custom manager regex consumes it and theautoReplaceStringTemplateadds it back). The packageRule that resolvescharts.jetstack.iocarriesextractVersionto strip thevfrom the datasource before Renovate matches and writes back versions; without it, bare pins would receive a spurious leadingv, and the zarf chart pin would becomevv1.21.x.bootstrap-rs/Dockerfile: digest-pinned build and runtime base images, and the mise CLI and Podman remote-client build arguments used by the toolbox. Themise installlayer names tools without versions (python,uv, etc.), so every pin resolves from the copiedmise.tomlat build time; there is no inline version to keep in lockstep.mgmt/**andworkload/**YAML: Flux, Helm, Kubernetes manifests, chart values, and clusterctl provider CRs undercapi-providers/.kindest/nodeimage tags wherever they are referenced in management manifests and air-gap scripts, and thekind-node-imagepin inbootstrap.toml(the kind bootstrap node, #647).airgap/images.txtandairgap/zarf.yaml: container image references, pinned by digest.airgap/zarf.yamlalso embeds its own Helm chart-version pins (e.g. cert-manager's); the cert-manager one is annotation-driven likebootstrap.toml's, so it isn't just carried along by the image-ref manager.airgap/zarf.yamlandairgap/files/clusterctl-providers.yaml: the CAPI core, kubeadm bootstrap, kubeadm control-plane, and CAPD provider release files, rendered versions, and staged config paths. One custom manager per pattern covers all four via adepNamealternation (e.g.kubernetes-sigs/cluster-api-(?:core|bootstrap-kubeadm|...)) rather than one manager per provider, since the four differed only in that name;depNameTemplateresolves the match back to the realkubernetes-sigs/cluster-apirepo for version lookup..github/workflows/: GitHub Actions references and the Renovate CLI pin used by the digest and managed-pin coverage tests.pivot.sh: imperative cert-manager and CAPI Operator chart pins, retained and grouped withbootstrap.tomland the Git manifests until the native shell path retires.virtualized-e2e/**/*.yaml.tmpl: annotated docker image pins in the WireMock harness templates (issue #355), digest-pinned like the air-gap inventory while retaining readable tags.
Grouping rules keep GitHub Actions updates together (excluding workflow container
images and runners), Flux updates together, CAPI updates together, imperative
chart pins with their declarative counterparts, and every pin whose value is a
literal Kubernetes release version together. Renovate proposes one PR at the
newest available version for each dependency, rather than parallel major and
non-major update PRs. Base images in bootstrap-rs/Dockerfile and air-gap
images are digest-pinned while retaining readable tags. Nothing automerges
except patch and minor bumps of the renovate CLI pin in
.github/workflows/validate.yml: that pin only selects the CLI the CI Renovate
tests run, and the renovate-digest-pinning job exercises the new version
before the merge. A minimumReleaseAge of 3 days (npm's unpublish window)
keeps a freshly published, possibly compromised release from landing on
main unreviewed.
The kubernetes-version group (#142) covers kindest/node (docker datasource:
the local-host node image, the kind-node-image pin in bootstrap.toml, and
both Cluster topology.version pins),
kubernetes/kubernetes (github-releases datasource: the kubectl pin in
mise.toml and the local-talos TalosControlPlane.spec.version annotation),
and the airgap/images.txt images kubeadm itself deploys for that release
(kube-apiserver, kube-controller-manager, kube-proxy, kube-scheduler,
coredns/coredns, etcd, pause, all under the registry.k8s.io docker
datasource). All of these carry a literal Kubernetes release version rather
than an independent versioning scheme, so a Kubernetes bump opens a single PR
across every file that tracks it. This gives same-PR visibility only:
Renovate still resolves each images.txt component to its latest upstream tag,
not the tag kubeadm deploys for the tracked release (etcd and coredns can
drift, and pause differs between the two platforms), so a reviewer must
reconcile them (see the update procedure). Other registry.k8s.io
images (the CAPI/kubeadm provider controllers) are unaffected: they're
matched by exact depName, not by registry host, and stay in the separate
cluster-api group. The kind CLI and Talos's own talosVersion
machine-config contract version each follow their own release cadence and
are intentionally excluded from this group.
cert-manager's Helm chart version and its container image tags used to be
tracked as separate, ungrouped dependencies. Four separate Renovate PRs
(#281-#284) bumped only the image tags in airgap/images.txt and
airgap/zarf.yaml to v1.21.2, while nothing bumped any chart-version pin,
still at 1.21.1. Since values/cert-manager.yaml has no image-tag override,
the chart deployed pods expecting v1.21.1 images by default, but Zarf had
only mirrored v1.21.2, so every cert-manager pod sat in ImagePullBackOff
and Helm's install wait ran out its 15-minute timeout (issue #322) --
initially misdiagnosed as a Kubernetes-version incompatibility and reported
as such upstream before being retracted
(cert-manager/cert-manager#9123).
The platform-charts group now covers cert-manager's chart pin and its
image tags together, same as the kubernetes-version group (#142) does for
Kubernetes; airgap/zarf.yaml's own chart-version line was not tracked by
any manager at all before this (confirmed with a real renovate
--dry-run=full run, which returned zero dependencies for that line), so an
annotation-driven custom manager was added for it, mirroring
bootstrap.toml's pattern.
cert-manager 1.21 officially supports Kubernetes 1.33-1.36 according to its
published support window. The repo currently pins Kubernetes v1.36.4 in
mgmt/local-host/clusters/docker/cluster.yaml, which is within that window.
The test tests/test-cert-manager-kubernetes-support.py
validates the pairing against a hardcoded list of empirically tested
combinations. It warns with a GitHub annotation when the pairing is not yet
validated, but never fails the build; it only fails if it cannot parse the
version pins themselves. When a Renovate PR changes either the cert-manager
version in bootstrap.toml or the Kubernetes version in
mgmt/local-host/clusters/docker/cluster.yaml, trigger or wait for a green
air-gapped nightly deploy on that new pairing, then add the row to the
VALIDATED_PAIRINGS list in that test file.
The CAPI group spans both the github-releases/github-release-attachments
release lookups and the docker-datasource digest-pinned images those same
providers deploy (registry.k8s.io/cluster-api*,
registry.k8s.io/cluster-api-helm/*, gcr.io/k8s-staging-cluster-api/*), so
a CAPI version bump lands its release assets and images in one PR instead of
two. CAPZ's one-minor-at-a-time override (issue #71) only matches the
github-releases datasource, so it is unaffected by the image grouping.
Toolbox release version¶
The krops-bootstrap package version lives in bootstrap-rs/Cargo.toml. It is a
release version, not a dependency pin, so Renovate does not increment it. When
a v* tag is pushed, .github/workflows/toolbox-release.yml fails unless the
tag matches that package version, then publishes the multi-architecture
toolbox image, signs it, and attaches an SPDX SBOM attestation.
Lifecycle tool versions installed in the image come from mise.toml and
mise.aws.toml. The Dockerfile separately pins its Rust builder and Debian
runtime base images, plus the mise installer and Podman remote-client build
arguments. Renovate manages those base references and build arguments.
Update procedure¶
- Wait for a Renovate PR. For configuration troubleshooting, use the pinned local dry-run procedure in AGENTS.md under "Editing renovate.json5".
- Review the raw and rendered diffs. The
validateworkflow checks kustomize builds, air-gap digest pinning, managed-pin extraction coverage,bootstrap.tomlconsistency, YAML, and (#142) thatairgap/images.txt'skube-apiserver/kube-controller-manager/kube-proxy/kube-scheduler,coredns,etcd, andpausepins match what the pinnedkindest/nodeversion's realkubeadmbinary actually deploys (airgap/tests/test-airgap-kubeadm-images.py), rather than drifting to whatever the latest upstream tag happens to be. The test verifies the kubeadm binary's SHA-256 and each pinned digest against the registry. Renovate bumps these pins like any other; when a bump disagrees with kubeadm the check fails, andpython3 airgap/tests/test-airgap-kubeadm-images.py --fixregenerates every tracked pin from kubeadm and the registry so the fix can be committed by hand. The workload cluster, the CAPD management cluster, and the kind bootstrap mgmt node all run the samekindest/nodeversion. The kind-bootstrap leg of that invariant is enforced bytests/test-kind-node-image-consistency.py, which requires thekind-node-imagepin inbootstrap.tomlto equal the ClusterClasscustomImagepins, the Clustertopology.versionpins, and the airgapkindest/noderefs (#647). The check compares againstkubeadm config images listfor that version, not a livecrictlharvest against a running node -- worth re-confirming there once an operator has one. Also (#322) that cert-manager's chart version (bootstrap.toml) matchespivot.sh,airgap/zarf.yaml's embedded chart version, and the cert-manager image tags inairgap/images.txt/airgap/zarf.yaml(tests/test-cert-manager-version-consistency.py) -- catches the same drift theplatform-chartsRenovate group prevents, regardless of how it happens. - For toolbox inputs, also require the
bootstrap-rsworkflow's Rust checks and container build/smoke job. - For a
kubernetes-versionPR, re-harvest the images kubeadm deploys for the new release (crictl imageson a node) and reconcile theairgap/images.txtcomponent tags (etcd,coredns/coredns,pause) with them before merging. - Merge manually.
The best fix for #142 items 3-4 would be Renovate itself running a script
that regenerates airgap/images.txt's k8s component pins from
kubeadm config images list right after a kindest/node bump, via
postUpgradeTasks
with executionMode: "branch" -- the PR would open already correct, with
nothing left to verify. That isn't available here: postUpgradeTasks is not
enabled by default on the hosted Mend/Renovate GitHub App this repo runs,
only on a self-hosted Renovate, and this repo deliberately moved off
self-hosting (issue #90, 6ad1db6) to avoid maintaining a runner. Mend has
allowlisted postUpgradeTasks for individual hosted-App repos on request
before (after reviewing the script), so this is a request to make, not a
hard platform limit -- tracked in
renovatebot/renovate#46270,
with the tech debt tracked in #326. The validate.yml
test-airgap-kubeadm-images.py check is the fallback in the meantime: it
fails the PR before merge instead of letting the drift ship silently, and
--fix is the manual stand-in for the script. If Mend allowlists
postUpgradeTasks for this repo, or self-hosting is revisited, replacing
this check with that script would close the gap properly and make the CI
test redundant.
If an image appears in both a manifest and the air-gap inventory
(airgap/images.txt or airgap/zarf.yaml), update both in the same PR. There
is no automated completeness check between manifests and the inventory, so
verify the pairing during review.
Intentional differences¶
- EKS cluster versions are not Renovate-managed pins and upgrade independently
of
kindest/node. - EKS addon versions (
*-eksbuild.*) have no public registry datasource and are updated manually. - Unversioned local tags such as
localhost:5001/krops-airgap:latesthave no comparable release version and remain untracked. scripts/toolbox-run.shdefaultsTOOLBOX_IMAGEto the mutableghcr.io/polarsquad/krops-toolbox:latest; Renovate does not manage this runtime default. The wrapper revalidates the mutable tag against the registry on every run (best-effort pull with a local-cache fallback), so a fresh release reaches hosts that already pulled:latest(issue #645).- The toolbox Dockerfile installs
docker-ce-clifrom Docker's apt repository without a package-version pin. Its client version intentionally follows that repository, while the base image, mise installer, and Podman client remain Renovate-managed. *.sops.yamlversion:fields, KubernetesapiVersionstrings, Helm chartappVersionvalues,bootstrap-rs/Cargo.toml's package version, and the Zarf packagemetadata.versionare not dependency pins.- The Zarf CLI pin in
mise.tomlkeepsversionunprefixed and addsvliterally inasset_pattern(issue #324), so the rendered asset name does not depend on how mise normalizes av-prefixed version.asset_patternalso remapsarch()toamd64/arm64, since Zarf's release assets don't use mise's defaultx64/arm64naming.tests/test-mise-zarf-pin.pyinstalls the pinned release via mise and checks the reported version, since a template mismatch otherwise fails silently until the pin is exercised. The Renovate rule for this pin anchors on the[tools."github:zarf-dev/zarf"]header; an unanchoredversion = "..."pattern also matches insidemin_version.