enbi

Comprised of two parts:

• NixBuild CRD. An instance of one looks like this:

  Spec:
    Flake URL:     git+https://nossa.ee/~talya/chog?rev=f5d531177a7330e9b270c15cca5df0eb72b8684c
    Image Tag:     nossa.ee/talya/chog:f5d531177a7330e9b270c15cca5df0eb72b8684c
    Package Name:  chog-docker.stream-layered
    System:        aarch64-linux
  

  When created, this will build packages.aarch64-linux.chog-docker.stream-layered from the given flake, on a node in the cluster which has the target architecture. (Nodes can be excluded from builds with -node-repel.)

  It’s expected that the output will be a Docker/OCI tarball image itself, or (preferably) a script generated by streamLayeredImage. The resulting tarball will be loaded into the containerd of the build host; use something like Spegel to make it automatically available to any host that might need it.

  You can see what’s happening in kubectl get:

  NAME                    IMAGE TAG                                                      SYSTEM          STATUS      BUILD NODE   AGE
  podwatcher-5d74cd57bb   nossa.ee/talya/chog:f5d531177a7330e9b270c15cca5df0eb72b8684c   aarch64-linux   Succeeded   cass         71m
  

  And more details in kubectl describe:

  Status:
    Build Job Name:  podwatcher-5d74cd57bb-5d74cd57bb
    Build Node:      cass
    Conditions:
      Last Transition Time:  2025-08-31T08:34:01Z
      Message:               Build completed successfully and result was fetched from endpoint
      Reason:                ResultSuccess
      Status:                True
      Type:                  Success
      Last Transition Time:  2025-08-31T08:34:01Z
      Message:               Image loaded
      Reason:                BuildFinished
      Status:                True
      Type:                  Complete
    Image Digest:            sha256:707b2b90f3f21c5cccad07429f51e1a37b9833d4ff87b323786d7cb246b8360c
    Image Size:              112199680
    Status:                  Succeeded
  

• Annotations to be added to your PodTemplateSpec. Like this:

  Pod Template:
    Labels:       app.kubernetes.io/name=chog
    Annotations:  enbi.hrzn.ee/nixbuild-flakeUrl: git+https://nossa.ee/~talya/chog?rev=f5d531177a7330e9b270c15cca5df0eb72b8684c
                  enbi.hrzn.ee/nixbuild-imageTag: nossa.ee/talya/chog:f5d531177a7330e9b270c15cca5df0eb72b8684c
                  enbi.hrzn.ee/nixbuild-packageName: chog-docker.stream-layered
  

  If a Pod with these annotations is found to be failing to start up due to a missing image, and the tag of that image matches the one in the annotation, a corresponding NixBuild will be created; after the image is built and loaded, your containers will come up at next retry!

Left to do:

• flesh out the tests; we’ll need to manipulate the state surrounding the controller (create Nodes, PVs, etc.).
• check our behaviour when a Pod fails to start because it’s trying to run a container with the wrong arch; we probably will ignore it (search codebase for RestartCount).
• running in Docker for Mac’s kind doesn’t work well wrt. building across Nodes; there’s no Spegel-like auto-pull across Nodes, only from the host cache (which we don’t (can’t?) touch).
• consider limiting the number of leftover successful NixBuilds.
• make it easier to deploy for others (and to update our own deployment, given the situation with config/ and how we use Timoni).
• document assumptions, limitations.
• local-overlay-store support?
• system (Node) Nix daemon support?

Built while following the Kubebuilder book.

License

Copyright © 2025 Asherah Connor <ashe@kivikakk.ee>

Licensed under the Apache License, Version 2.0 (the “License”); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0; it is included in this distribution at LICENSE.

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an “AS IS” BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.