Run the panel locally
The panel is a Django project meant to use by Nix: enter its dev shell with
nix-shell in panel/, then drive it with the manage wrapper (manage migrate, manage runserver, manage createsuperuser). Running it against a
live API and Centrifugo, and provisioning the deployer's incus credentials so
the "Deploy" button works end to end, are operational concerns maintained
alongside the code: the sections below give the current setup, dev-shell, and local spin-up
instructions.
Run these from the panel/ devShell, where .# resolves to the panel flake;
from the repo root write them as ./panel#manage /
./panel#checks.x86_64-linux.panel-basic (the thin root entrypoint re-exports
only the deploy family). Without flakes, run any app via its matching dev shell
-- e.g. nix-shell -A devShells.x86_64-linux.manage --run 'manage migrate' default.nix is the non-flake equivalent of nix run .#manage -- migrate.
Panel tests run with nix build .#checks.x86_64-linux.panel-basic.
Development
The nix run and nix build commands on this page need the Nix features nix-command and flakes.
See Prerequisites for how to enable them.
-
To obtain all tools related to this project, enter the development environment with
nix-shell, or withnix developif you use flakes. Enter it from inside the checkout of the repository: the shell links the generated static files intosrc/, writes a developmentSECRET_KEYto.credentials/, and sets the variables the panel needs. The environment reference documents each variable the panel reads.If you want to do that automatically on entering this directory:
-
Run
direnv allowin the directory where repository is stored on your machineNote
This is a security boundary, and allows automatically running code from this repository on your machine.
-
Run the Django unit tests, on sqlite and with no VM:
nix-build -A checks.panel-python-tests(or
nix build .#checks.x86_64-linux.panel-python-testsvia flakes.) -
Run the NixOS integration test, which boots the panel and the api on one container and probes them over HTTP:
nix-build -A checks.panel-basic -
List all available Django management commands with:
manage -
With a fresh database, run:
manage migrate -
Run the server locally:
manage runserver -
Create an admin user:
manage createsuperuserand log in at http://localhost:8000/admin.
-
After changing the database schema, add migrations and apply them:
manage makemigrations manage migrate -
Use the Django Debug Toolbar for troubleshooting in the browser
Local spin-up
There are two local setups. Pick Simple unless you specifically want live WebSocket progress updates.
Simple setup (synchronous, no Windmill/Centrifugo)
The panel talks only to the api/ HTTP server. Deployments run as background
subprocess threads in api/; with CENTRIFUGO_BROWSER_WS_URL unset, the
deploy/destroy loading screens poll deployments/<pk>/state/ until the
deployment reaches a terminal state, so they resolve instead of hanging.
# terminal 1 (api/ devShell)
nix run .#terraform-backend-dev
# terminal 2 (api/ devShell), see api/docs/reference/environment.md for what each variable does
export API_CORS_ORIGINS=http://127.0.0.1:8000,http://localhost:8000
export DEPLOYMENT_ROOT="$(git rev-parse --show-toplevel)"
export DEPLOYMENT_ATTRIBUTE=operator-tf-incus-operator-all
export DEPLOYMENT_STATE_ADDRESS_VARS='{"hp_state_address":"hosting-provider-addresses"}'
nix run .#api-http-stack
# terminal 3 (panel/ devShell), first run only
manage migrate
# terminal 3 (panel/ devShell)
export API_BROWSER_BASE_URL=http://127.0.0.1:8088
manage runserver
The deployment form fetches its schema and rename table from api/ directly in
the browser, so set API_BROWSER_BASE_URL before manage runserver to point
the browser at the api server (:8088). The api server must allow that origin
via API_CORS_ORIGINS, as terminal 2 does; see
Run the API locally.
Match the host to the one in the browser's address bar -- 127.0.0.1 and
localhost are distinct origins. Leaving API_BROWSER_BASE_URL unset makes the
form fetch same-origin (:8000), where the panel has no /schema route, so the
fetches 404.
api/ checks each call against the panel, so the panel must run; see Authorization.
Authorization
The browser exchanges its session for a short-lived bearer token at the panel's
/api/token/ and sends that token to api/. api/ sends the headers of each call
to the panel's /api/authz/, which verifies the token and tells api/ if the
user can make the call. Both setups use this callback.
api-http-stack sets AUTHZ_CALLBACK_URL to
http://127.0.0.1:8000/api/authz/. The panel must run, else api/ refuses all
calls that need authorization.
The callback accepts only calls that carry the token in
$XDG_RUNTIME_DIR/fediversity-authz/token. api-http-stack sends that token,
and the panel devShell links $CREDENTIALS_DIRECTORY/AUTHZ_CALLBACK_TOKEN to
it. The first of the two to start writes the token, so you can start them in
any order. The devShell keeps an AUTHZ_CALLBACK_TOKEN that is a regular file.
To use api/ without the panel, add this to terminal 2:
export API_AUTHZ_ALLOW_ALL=1
Then api/ does not call the panel and allows all calls, with or without a token.
Full setup (Windmill + Centrifugo)
Adds Windmill (deployment job tracking in api/) and Centrifugo (live
loading-screen updates over WebSocket). Run the simple-setup terminals plus
api/'s windmill-dev and the panel's centrifugo-dev:
# terminal 4 (api/ devShell)
nix run .#windmill-dev
# terminal 5 (panel/ devShell)
nix run .#centrifugo-dev
Set the api-side env vars (see
Run the API locally), and on the
panel side, before manage runserver:
export CENTRIFUGO_BROWSER_WS_URL=ws://127.0.0.1:8090/connection/websocket
With this set, the spinners get the outcome of a build from a Centrifugo
publication. While the subscription is live, the poll of the state slows down to
once every 30 seconds. When the subscription becomes live, the spinner reads the
state once, because Centrifugo does not replay a publication that it sent
before. While the subscription is not live, the poll runs every 2 seconds. The
slow poll stays, because api/ publishes only when it in turn has
CENTRIFUGO_API_URL set, and a publication is best effort.
Caveat (host consistency). Centrifugo forwards the session
Cookieto Django on the WS handshake. Use one host consistently -- both127.0.0.1or bothlocalhost-- for the panel URL andCENTRIFUGO_BROWSER_WS_URL, or the cookie is not shared across origins and the WS connect silently fails.
Set API_HTTP_URL in the panel environment to point at the api server: this is
the URL the panel process dials for its server-side calls (the state poll
behind the spinners, the config-diff baseline, the deployment-list proxy), as
opposed to API_BROWSER_BASE_URL, which the browser uses. It defaults to
http://127.0.0.1:8088, which is right only while api/ is colocated. Under
NixOS both variables come from one option, services.panel.apiBaseUrl: the two
ends could in principle reach api/ at different addresses, but no deployment of
ours needs them to. In production, the NixOS module of api/ runs Centrifugo
behind nginx on its own subdomain, and the panel sets
services.panel.centrifugo.wsDomain to that name.
For the panel's "Deploy" button to actually trigger a deployment (and not fail resolving the base image), the deployer host also needs incus trust tokens and an uploaded base image; see Provisioning the deployer's incus credentials in the api README.
Checks
./panel#checks.<system>.apps-panel is a full deployment check. It runs the
panel on the deployer machine, deploys some services through it, and checks that
they are on the target machines. It builds on the deployment checks of core
(see core/checks/README.md). The only difference
is that it deploys through the panel: a client node runs a browser that
Selenium scripts control, and starts the deployment as a human does.
-
Run Lighthouse on the pages of the panel.
result/lighthouseholds the reports:nix-build -A checks.panel-lighthouseTo audit one more page, add it to
PAGESinnix/lighthouse.mjs.