≪ View Page

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
infrastructure:nixos-boxes [2025/12/21 07:46] – diamondinfrastructure:nixos-boxes [2026/01/08 09:12] (current) – move bitwarden-sops out into a page diamond
Line 4: Line 4:
  
 As the name describes, these boxes run NixOS, but they're also managed using GitOps. Their code lives in [dma/infra/nixos](https://codeberg.org/dma/infra/tree/main/nixos). Any commits pushed to `main` will automatically get deployed within 10 seconds to all machines. As the name describes, these boxes run NixOS, but they're also managed using GitOps. Their code lives in [dma/infra/nixos](https://codeberg.org/dma/infra/tree/main/nixos). Any commits pushed to `main` will automatically get deployed within 10 seconds to all machines.
 +
 +## Pages
 +
 +```dokuwiki
 +<catlist infrastructure:nixos-boxes: -noHead>
 +```
 +
 +## Structure
 +
 +- `nixos/`
 +  - `machines/` - NixOS machine configurations.
 +  - `modules/` - Reusable NixOS modules.
 +    - `sanity.nix` - Sane defaults for all NixOS machines. For now, these are
 +      server-oriented, so if you're using a NixOS box for IoT-adjacent purposes,
 +      you won't need to import this module.
 +  - `Justfile` - recipe file for common operations (see
 +    [Common Operations](#common-operations)).
 +
 +## Onboarding Yourself
 +
 +Currently, these are the major steps to onboarding yourself to manage these
 +machines:
 +
 +1. Gain access to these machines over the network, ideally over Tailscale.
 +   Currently, the point of contact for Tailscale is @diamond, but this may
 +   change once we have our own Headscale instance.
 +2. Be added into `./modules/ssh/public_keys.txt`. This also requires a
 +   re-deployment from an existing administrator.
 +3. Set up your local environment. See
 +   [Setting Up Your Local Environment](#setting-up-your-local-environment).
 +
 +### Setting Up Your Local Environment
 +
 +You will need at least [Nix][nix] installed on your local machine. Optionally,
 +install [Direnv][direnv] as well for easier environment variable management.
 +
 +If you're just using Nix, you can enter a shell with all dependencies by
 +running:
 +
 +```sh
 +nix develop
 +```
 +
 +If you're using Direnv, create a `.envrc` file under the `nixos` directory (this
 +directory) with the following content:
 +
 +```sh
 +use flake
 +```
 +
 +> **Tip:**
 +>
 +> You may have a need to save secret environment variables such as Bitwarden
 +> session keys in your `.envrc`. For this reason, consider adding it to your
 +> local gitignore by running:
 +>
 +> `echo ".envrc" >> "$(git rev-parse --show-toplevel)/.git/info/exclude"`
 +
 +## Deployment
 +
 +All NixOS boxes are automatically deployed with GitOps practices. The following diagram briefly explains its architecture:
 +
 +```dokuwiki
 +<div centeralign>
 +<mermaid>
 +    flowchart TD
 +        codeberg[dma/infra @ main]
 +        codeberg --[pull every 15s]--> comin_a
 +        codeberg --[pull every 15s]--> comin_b
 +        subgraph nixos_machine_a[NixOS Machine A]
 +            nixos_rebuild_a[nixos-rebuild]
 +            comin_a[comin.service] --> nixos_rebuild_a
 +        end
 +        subgraph nixos_machine_b[NixOS Machine B]
 +            nixos_rebuild_b[nixos-rebuild]
 +            comin_b[comin.service] --> nixos_rebuild_b
 +        end
 +</mermaid>
 +</div>
 +```
 +
 +Whenever a new commit is pushed to the `main` branch, all machines race to deploy this new commit. This way, we achieve GitOps without requiring an actual CI/CD pipeline, simplifying the deployment pipeline by a lot.
 +
 +To see how caught up the deployments are on the NixOS fleet, run `just watch`. This spawns a dashboard of machines and their commits which refreshes every 5 seconds. Use this immediately after pushing to comprehensively view how the machines are going along.
 +
 +### Testing Branches
 +
 +[comin](https://github.com/nlewo/comin), the GitOps tool we use, supports using testing branches to try out configurations without persisting them to the next reboot.
 +
 +To use this, simply push a new commit to a new `testing-{machine}` branch, where `{machine}` is the name of the machine. For example, a commit that updates `home-assistant-one`'s configuration would be pushed to `testing-home-assistant-one`.
 +
 +Once everything is tested to be working, you can simply `git checkout main` then `git rebase testing-{machine}` to add your new commits on top. Then, push to main as usual.
 +
 +
 +## Common Operations
 +
 +Most, if not all, common operations can be performed via the `just` command. To
 +see a list of available recipes, run:
 +
 +```sh
 +just
 +```
 +
 +## Playbooks
 +
 +This section contains useful playbooks for managing these machines.
 +
 +### Setting Up a New Machine
 +
 +This section will not go into the detailed steps of installing NixOS on a new
 +machine, but rather the recommendations for setting it up:
 +
 +#### Creating a configuration.nix
 +
 +Create a `machines/<machine>/configuration.nix` manually. For most machines
 +(especially common Thinkcentre models), it should be enough to just import
 +`nixos-hardware` directly.
 +
 +After creating this file, add it to the local `flake.nix`, under
 +`.nixosConfigurations.<machine>`. **Use `self.lib.nixosSystem`** unless you
 +have a good reason not to.
 +
 +#### Disk Partitioning
 +
 +Create a `machines/<machine>/disko.nix` containing the disk layout using
 +[disko][disko].
 +
 +**It is strongly recommended to use full disk encryption via LUKS + Btrfs**,
 +unless you are absolutely sure the machine will not store any secrets or run
 +any secure workloads. You may refer to existing NixOS machines for disko
 +examples.
 +
 +Don't forget to add the file to the local `flake.nix`, under
 +`.diskoConfigurations.<machine>` as well. Prefer `self.lib.diskoConfiguration`
 +for this.
 +
 +#### Installing NixOS Quickly
 +
 +Install NixOS directly by flashing it onto the hard drive using a SATA-to-USB or NVME-to-USB adapter. This will save you the trouble of having to transfer this repository over to the machine's live environment.
 +
 +If you're inside the Nix develop environment, you can run:
 +
 +```sh
 +sudo disko-install \
 +  --mode format \
 +  --disk '<machine-disk>' '/dev/sdXY' \
 +  --flake "path://$PWD#<machine>"
 +```
 +
 +> **Note:** Use `path://$PWD` rather than `.` since `disko` has buggy Flake subdirectory support.
 +
 +> **Note:** This doesn't set the root password. If you're one of the infra admins, you should have your SSH keys inside `./modules/ssh/public_keys.txt` and SSH directly into the box's LAN IP to continue provisioning.
 +
 +If you're doing this from the NixOS live environment, and you want to install directly into the same machine, you can run:
 +
 +```sh
 +nix run 'nixpkgs#disko' -- --mode destroy,format,mount --flake '.#<machine>'
 +nixos-install --no-channel-copy --flake '.#<machine>'
 +```
 +
 +#### Re-using Modules
 +
 +For convenience, import the few core modules: `network`, `sanity`, and `ssh`.
 +See existing `configuration.nix` files for reference.
 +
 +#### Secure Boot
 +
 +After installation, set up Secure Boot and TPM2 unlocking as soon as you can.
 +The `secureboot` module should aid you in this process, but refer to
 +[Lanzaboote][lanzaboote]'s existing documentation for more details:
 +
 +- [Prepare Your System](https://github.com/nix-community/lanzaboote/blob/master/docs/getting-started/prepare-your-system.md)
 +- [Enable Secure Boot](https://github.com/nix-community/lanzaboote/blob/master/docs/getting-started/enable-secure-boot.md)
 +
 +### Adding Bitwarden Secrets to a Machine
 +
 +Moved to [a separate playbook](/infrastructure/playbook/bitwarden-sops-nix).