Differences

This shows you the differences between two versions of the page.

--- projects:cluster [2025/07/14 16:04] – Add Caddy samp20
+++ projects:cluster [2026/04/24 14:25] (current) – samp20
@@ Line 6: / Line 6: @@
   * Be simpler to setup and use than Kubernetes.
   * Work across networks. For example between a cloud provider and self-hosted.
-  * Not require a separate management service.
 ===== Parts list =====
@@ Line 13: / Line 12: @@
   * Python based core.
-  * JSON for cluster configuration.
+  * PostgreSQL database for cluster configuration.
-    * Will eventually be signed (possibly JWT).
   * runc (or other OCI compliant runtime) for container management.
-  * Wireguard for the node mesh network.
   * NFTables for firewall configuration.
-  * Python Flask for config distribution.
+  * Python Flask for web UI/API.
-  * Caddy for HTTPS ingress proxy
+  * Caddy for HTTPS ingress proxy.
 ===== General architecture =====
-The core of the system will be a python service that will receive the configuration (probably via UNIX socket so the Flask http server can be kept in a separate container), validate it and update the various components through python plugins using ''importlib.metadata.entry_points''.
+{{drawio>projects:cluster-architecture.png}}
-The configuration will consist of objects representing different parts of the system that need to be configured. An example is below:
+===== RunC integration =====
-<code>
+RunC containers are created with a ''config.json'' and a ''rootfs''. RunC can generate a base configuration. While we could technically include that in the cluster config directly, it probably makes more sense to use the RunC generated config and merge the cluster config into it. A Docker config (Docker uses RunC under the hood) is even more restricted, specifying what syscalls are allowed for example. We may want to add this in the future, but not initially.
-{
-    "hosts": {
-        "cloud01": {
-            "wg_network": {
-                "type": "wireguard_network",
-                "link_name": "wg_cluster",
-                "address": "10.69.0.1/16"
-            },
-            "peer_local01": {
-                "type": "wireguard_peer",
-                "network": "wg_network",
-                "allowed_ips": ["10.69.1.0/24"]
-            }
-        }
-    }
-}
-</code>
-This configuration, while technically able to be created by a human, will likely be created/updated by a separate **offline** tool that consumes a more human friendly layout. This is in contrast to Kubernetes which relies on an **online** service to manage these updates.
+==== Pulling the root filesystem ====
-===== RunC integration =====
+There exists a few tools that can help with this. [[https://github.com/containers/skopeo|Skopeo]] and [[https://umo.ci/|Umoci]]. An example is below:
-RunC containers are created with a ''config.json'' and a ''rootfs''. RunC can generate a base configuration. While we could technically include that in the cluster config directly, it probably makes more sense to use the RunC generated config and merge the cluster config into it. A Docker config (Docker uses RunC under the hood) is even more restricted, specifying what syscalls are allowed for example. We may want to add this in the future, but not initially.
+  - ''%%skopeo copy docker://busybox:latest oci:busybox:latest%%''. Will create a ''busybox'' folder.
+  - ''%%umoci unpack --image busybox:latest bundle%%''. Unpacks the OCI image into an OCI runtime bundle in the ''bundle'' folder. This bundle also has a ''config.json''. It'll be interesting to compare that with the RunC generated one.
-We will need a mechanism for pulling the Root filesystem. There is probably a standardized way of downloading these from a container registry and unpacking them. For now we won't worry about private registries, but something to consider if this project becomes more widespread.
+It may be preferred to follow the [[https://github.com/opencontainers/image-spec/blob/main/conversion.md|OCI conversion spec]] directly instead of relying on a 3rd party tool such as Umoci.
 ===== Network and Wireguard =====
@@ Line 66: / Line 47: @@
 ===== Caddy =====
-Caddy can be configured directly through JSON. We can merge together multiple container configs, along with any required global configuration, and pass this directly to Caddy. There are probably some gotchas to be aware of when merging. A good starting point will be to take a existing ''Caddyfile'' and convert it to JSON to see what the structure is like and how it can be split.
+Caddy can be configured directly through JSON.