NightWatcher2

Tools to interface with Unihedron Sky Quality Meters (SQMs) in furtherance of participating in the Dark Sky Network (DSN), operated by the Southern Arizona Dark Sky Association.

SQMs measure sky brightness (magnitude per square arc-second). NightWatcher2 reads them, stores the readings in a time-series database, and exposes that data through an API and a web UI. It also polls co-located weather stations, and automatically exports data on a schedule — uploading to the DSN shared storage, and/or pushing to a companion WordPress site for public display.

It also doubles as an image-file ingestion point for an observatory or observing site. A generic extension registry lets companion tools plug into the same web UI while they run, without adding anything non-SQM to the core — the first being nightwatcher-ingest, a FITS ingest pipeline that files raw astrophotography frames and can stamp each with the sky-brightness reading nearest the moment it was captured.

Components

Component	Description	Status
SQM device library	Talk to SQM-LE (Ethernet/TCP) and SQM-LU (USB serial); parse the Unihedron protocol; network + USB discovery; calibration control	Done
sqmctl	CLI to discover and query a single SQM over TCP or serial	Done
Database	MariaDB (libmariadb) store — readings, sensors, calibration/config history, weather, exports, auth, settings, extensions (12 tables)	Done
nwdb	CLI to register sensors, poll an SQM into the DB, and query readings	Done
nightwatcherd	Daemon: polls active sensors + weather stations on their intervals, records readings, runs scheduled exports, serves the API/UI	Done
REST API	Embedded HTTP/JSON: auth, sensor/weather CRUD, readings/query, live poll/test/discover, calibration, exports, server settings; optional HTTPS	Done
Web UI	Login, dashboard, sensor/weather management, query + time-series graph (Sun/Moon/phase + ambient overlays), sensor test + calibration, DSN export, users, DB + server settings, plus dynamic tabs for registered companion tools	Done
Weather	Modular pull providers (Ambient Weather, Weather Underground) normalized to SI	Done
Data export	Modular scheduled exporters — DSN community .dat → Google Drive, or webhook push to an HTTP endpoint	Done
WordPress connector	Companion plugin (nightwatcher-wp) receives the webhook push; serves a read-only, public date-range graph	Done
Extension registry	Companion tools register + get a dynamic web-UI tab while running; first is nightwatcher-ingest (FITS ingest + SQM stamping)	Done
Security	Admin login (PBKDF2), off-localhost read-auth, optional self-signed HTTPS/TLS	Done
Packaging	debconf-driven .deb (amd64 + arm64 / Raspberry Pi) — prompts for DB/API setup; systemd unit; udev rule for USB access	Done

Building

Requirements: a C++17 compiler, CMake ≥ 3.16, and libmariadb-dev for the database layer (or configure with -DNW_WITH_DB=OFF to skip it).

cmake -B build -DCMAKE_BUILD_TYPE=RelWithDebInfo
cmake --build build --parallel
ctest --test-dir build --output-on-failure

Run the skeleton daemon:

./build/nightwatcherd --version
./build/nightwatcherd --config build/nightwatcher.conf.example

Cross-compiling for ARM (e.g. Raspberry Pi 64-bit)

sudo apt install g++-aarch64-linux-gnu
cmake -B build-arm64 -DCMAKE_TOOLCHAIN_FILE=cmake/toolchain-aarch64.cmake -DNW_BUILD_TESTS=OFF
cmake --build build-arm64 --parallel

Talking to an SQM (sqmctl)

sqmctl discovers and queries SQM-LE units over the network (TCP, default port 10001) and SQM-LU units over USB serial (115200 8N1).

Finding a unit

If an SQM-LE picked up an unknown IP via DHCP, scan its subnet — each host is probed on port 10001 and confirmed with a unit-info (ix) query, so only real SQMs are reported. discover-usb does the same for USB SQM-LUs by probing the local serial ports:

sqmctl discover 192.168.1.0/24
#   192.168.1.73:10001  serial=00000413 model=3 feature=1 protocol=2
sqmctl discover-usb
#   /dev/serial/by-id/usb-FTDI_FT232R_USB_UART_XXXX-if00-port0  serial=00007475 ...

Subnet options: --port N (default 10001), --timeout MS (default 700), --concurrency N (default 128). A /24 scans in roughly a second.

Querying a unit

sqmctl --tcp 192.168.1.50:10001 info               # unit info        (ix)
sqmctl --tcp 192.168.1.50:10001 read               # averaged reading (rx)
sqmctl --serial /dev/ttyUSB0 read                  # same, over USB   (SQM-LU)
sqmctl --serial /dev/ttyUSB0 cal                   # calibration info (cx)

Both transports accept info, read, unaveraged, and cal. USB devices need read/write on the port (the dialout group; the .deb handles this).

Testing without hardware

A software simulator emits canned Unihedron responses so the library and CLI can be exercised end-to-end without a physical meter:

./build/sqm-sim 10001 &                 # listen on 127.0.0.1:10001
./build/sqmctl --tcp 127.0.0.1:10001 read

Database

Readings and configuration are stored in MariaDB via MariaDB Connector/C (libmariadb).

The schema (sql/schema.sql) has 12 tables: sensors, readings (with a quality flag for saturated/suspect data), config_log (calibration/config history), an operational events log, weather_stations / weather_readings (a co-located weather station such as an Ambient Weather WS-2000, polled by the daemon), users / sessions (API/UI auth), export_targets / export_log (scheduled DSN uploads), settings (runtime server config), and extensions (the companion-tool registry). Stored units are metric/SI.

One-time setup

sudo apt install -y mariadb-server libmariadb-dev
sudo mariadb < sql/setup.sql     # create the database + application user
sudo mariadb < sql/schema.sql    # create the tables

setup.sql creates a development user nightwatcher with password nightwatcher — change it for any real deployment. The tools read the connection from the environment:

export NW_DB_HOST=127.0.0.1 NW_DB_PORT=3306
export NW_DB_USER=nightwatcher NW_DB_PASSWORD=nightwatcher NW_DB_NAME=nightwatcher

nwdb

Register a sensor with full site metadata (serial/protocol/feature are auto-filled from the device via ix unless --no-probe). New sensors are added disabled — verify them first, then enable database population:

nwdb add-sensor DSN003 --tcp 172.22.4.112:10001 \
     --name "CrestaLoma Observatory" --site "CrestaLoma Observatory" \
     --lat 31.9500 --lon -111.6000 --elev 1200 \
     --timezone America/Phoenix --installed 2026-07-18
nwdb add-sensor DSN006 --serial /dev/ttyUSB0 --name "USB unit"   # SQM-LU over USB
nwdb set-sensor DSN003 --status active   # enable polling (starts DB population)
nwdb set-sensor DSN003 --elev 1205       # partial edit — only elevation changes
nwdb show DSN003                         # full metadata for one sensor
nwdb sensors                             # list registered sensors
nwdb poll DSN003                         # read the SQM now and store the reading
nwdb readings DSN003                     # show recent stored readings
nwdb cal DSN003                          # read + store calibration

API

nightwatcherd serves a JSON API on [api] bind:port (default 127.0.0.1:8080), sharing the same nw_db layer as nwdb — so the web UI drives the same code. Writes require an admin login session (cookie) or the optional static NW_API_TOKEN; reads are open from localhost, but also require that login/token once the server is bound off localhost. On first start the daemon seeds an admin account (password admin, flagged must-change). The server can also present HTTPS with an auto-generated self-signed certificate ([api] tls = on, or the web UI's Server tab); bind address, port, and TLS are editable at runtime.

Log in (cookie-based sessions), then use the cookie — or the token the login returns — for writes:

curl -s -c jar -X POST localhost:8080/api/v1/login -d '{"username":"admin","password":"admin"}'
curl -s -b jar localhost:8080/api/v1/me
curl -s -b jar -X POST localhost:8080/api/v1/me/password -d '{"current_password":"admin","new_password":"s3cret"}'
curl -s -b jar -X POST localhost:8080/api/v1/users -d '{"username":"obs","password":"pw","role":"viewer"}'

Roles: admin (full access + user management) and viewer (read-only). Alternatively, a static token for scripts:

export NW_API_TOKEN=$(openssl rand -hex 16)
curl -s localhost:8080/api/v1/health
curl -s localhost:8080/api/v1/sensors
curl -s "localhost:8080/api/v1/sensors/DSN003/readings?limit=10"
curl -s "localhost:8080/api/v1/discover?cidr=172.22.4.0/24"

AUTH="Authorization: Bearer $NW_API_TOKEN"
curl -s -X POST  localhost:8080/api/v1/sensors -H "$AUTH" \
  -d '{"id":"DSN003","tcp":"172.22.4.112:10001","name":"CrestaLoma Observatory","lat":31.9500,"lon":-111.6000,"elev":1200}'
curl -s -X PATCH localhost:8080/api/v1/sensors/DSN003 -H "$AUTH" -d '{"elevation_m":1205}'
curl -s -X POST  localhost:8080/api/v1/sensors/DSN003/poll -H "$AUTH"
curl -s -X POST  localhost:8080/api/v1/db/init -H "$AUTH"     # create any missing tables

Beyond the basics above, the API also covers: sensor test (a non-persisting self-check) and enable/disable; calibration read/record/arm/disarm/set with history; weather-station CRUD + live poll; export targets (DSN .dat→Drive or webhook push), run-now, and logs; USB discovery (GET /discover/usb); server settings (bind/port/TLS + restart-to-apply); and DB maintenance (GET /db/status, POST /db/init, DELETE /sensors/{id}/readings?before=DATE).

Installing

NightWatcher installs as a self-contained bundle under a single prefix (default /usr/local/nightwatcher), which keeps the install tidy and trivial to remove:

cmake --build build --parallel
sudo cmake --install build          # or: sudo make -C build install

This lays down bin/, web/, sql/, share/, and the systemd unit under the prefix, then (reaching outside it) symlinks the CLIs onto PATH (/usr/local/bin/{sqmctl,nwdb,nwexport-auth}), symlinks the unit into /etc/systemd/system/, and seeds /etc/nightwatcher/nightwatcher.conf (never overwriting an existing one). Change the prefix with -DCMAKE_INSTALL_PREFIX=….

To finish: sudo systemctl daemon-reload && sudo systemctl enable --now nightwatcherd. To remove everything: sudo /usr/local/nightwatcher/uninstall.sh (or sudo make -C build uninstall) — this leaves /etc/nightwatcher (your config + secrets) and the database in place.

Debian package (.deb)

CI builds installable .debs for amd64 and arm64 (Raspberry Pi) on every push — download them from the run's Artifacts. To build one yourself:

cmake -B pkg -DNW_INSTALL_SYSTEM_LINKS=OFF -DCMAKE_BUILD_TYPE=RelWithDebInfo
cmake --build pkg --parallel
( cd pkg && cpack -G DEB )                 # -> pkg/nightwatcher_<ver>_<arch>.deb
sudo apt install ./pkg/nightwatcher_*.deb  # resolves libmariadb3/libssl3, runs the postinst

The install is debconf-driven: it prompts for the database password, host, the web-UI/API bind address, port, and HTTPS, whether to create the nightwatcher database + user now (via the local MariaDB/MySQL root socket — the daemon builds the tables and seeds an admin login on first start), and whether to start the service. Re-run those prompts any time with:

sudo dpkg-reconfigure nightwatcher

The package installs the bundle under /usr/local/nightwatcher; its postinst applies those answers to /etc/nightwatcher/nightwatcher.conf and writes the password to /etc/nightwatcher/nightwatcher.env (mode 0600), then creates the PATH/systemd/udev symlinks, seeds the config, installs the udev rule for USB SQM-LU access, adds the installing user to dialout, and enables (and optionally starts) the service. apt remove reverses it; apt purge also drops /etc/nightwatcher and the saved answers.

Configuration

make install seeds /etc/nightwatcher/nightwatcher.conf from config/nightwatcher.conf.example.in; edit it for your site. It configures only the daemon itself (database connection + API); sensors are registered in the database with nwdb add-sensor (or the API), and each sensor's cadence is its poll_interval_s.

Web UI

With web_root set (default /usr/local/nightwatcher/web), the daemon serves a browser UI at http://<host>:<api-port>/ (or https:// with TLS on) — static HTML/JS (dark theme, uPlot graph, no external CDNs) talking to the API. It provides a login page (default admin/admin, must-change on first login), a live status dashboard, sensor management (add over the network or via a USB scan, a non-persisting Test, Calibrate, and enable/disable), weather-station management, a readings query with a time-series graph (and DSN-format download), DSN export configuration, an events log, user management, database maintenance (schema status/init, pruning), and a server tab (listen address, port, HTTPS). Admin-only controls are hidden for viewer accounts.

The readings graph can overlay Sun/Moon altitude, a moon-phase glyph, and ambient temperature (from a co-located weather station, matched by shared site) — each toggleable — computed in-browser via a vendored SunCalc (BSD-2-Clause; no CDN). The SQM's own internal temperature is labelled sensor °C to distinguish it from ambient.

Running the daemon

nightwatcherd polls every status = 'active' sensor from the database on its interval, stores readings (with the quality flag), and logs connect/disconnect/errors to the events table.

export NW_DB_PASSWORD=nightwatcher
./build/nightwatcherd --config build/nightwatcher.conf.example

Signals: SIGTERM / SIGINT shut down gracefully; SIGHUP reloads active sensors, weather stations, and export targets (so anything added via nwdb or the API is picked up without a restart — the API triggers this automatically on such changes). Under systemd (the unit is installed and enabled as above, from config/systemd/nightwatcherd.service.in), put the database password in /etc/nightwatcher/nightwatcher.env as NW_DB_PASSWORD=... (mode 0600). The unit runs as a DynamicUser in the dialout group so it can reach a USB SQM-LU on /dev/ttyUSB*.

Extensions & companion tools

NightWatcher stays a focused SQM tool at its core, but it can host companion tools without absorbing their code. A generic extension registry (an extensions table plus /api/v1/extensions) lets a separate tool register itself and heartbeat while it runs; the web UI then shows a dynamic tab for it, and drops the tab when the tool stops. A plain SQM deployment never sees any of it — the core ships no astrophotography or tool-specific code.

• nightwatcher-ingest — a config-driven FITS-frame ingest pipeline: it watches an incoming directory, classifies each raw astrophotography frame from its header, renames it to a standard, and files it into an archive tree. Pointed at NightWatcher, it stamps each frame with the sky-brightness reading nearest the moment it was taken and registers an Ingest tab that shows its transfer history live.
• NightWatcher-AirWatcher — a C++ daemon that pulls raw FITS frames off ZWO ASIAir devices over their SMB share and drops them into the nightwatcher-ingest landing directory, replacing a generic SMB sync tool (e.g. GoodSync) with a purpose-built copier that adds copy scheduling, optional deletion, subnet discovery, and its own web UI. It registers a read-only AirWatcher tab showing each configured ASIAir, its address, total Autorun frame count, and how many remain to copy. The two companions chain into one path: ASIAir → AirWatcher → incoming/ → nightwatcher-ingest → archive.

The WordPress connector and the appliance platform helper below are further companions, tied in through the export system and the platform socket rather than the extension registry.

Appliance platform helper

An appliance image that ships NightWatcher (today NightWatcher-Pi) has to configure things the daemon deliberately cannot reach from inside its sandbox: Wi-Fi, removable media, the system clock. Such an image runs a small privileged helper on a unix socket at /run/nightwatcher-platform.sock (override with NW_PLATFORM_SOCKET). Where that socket exists, /api/v1/platform and /api/v1/platform/<path> proxy admin-authenticated requests to it, and the helper's probe response ({name, label, ui}) tells the web UI to add a tab whose ES module the helper itself serves at /api/v1/platform/<ui>. The panel rides the daemon's own origin, session, and certificate, so there is no second port and no second login.

Where no such socket exists, which is every ordinary install, those routes return 404 and no tab appears. The core carries no appliance-specific code; it only forwards.

Data export

Export targets live in the database (export_targets) and are run by the daemon on a schedule (nightly, weekly, or monthly at a local time — the monthly one can target the last day of the month for a contiguous file — or a fixed interval), each tracking a watermark so it only sends new readings; every run is recorded in export_log. Two target types ship today:

• dsn — builds the DSN Community Standard Skyglow .dat file for the month and uploads it to Google Drive (OAuth; authorize once with nwexport-auth). Configured from the web UI's export tab or the API.

• webhook — pushes readings (incrementally, since the watermark) to an HTTP endpoint as JSON with a bearer token, chunked for large backfills; each push also carries the co-located weather station's readings (matched by shared site) so the receiver can overlay ambient temperature. This feeds the WordPress connector below, and is configured via the export-targets API:

  curl -s -b jar -X POST localhost:8080/api/v1/export-targets \
    -d '{"id":"wp-036","sensor_id":"DSN003","target":"webhook","schedule":"interval",
         "interval_s":300,"status":"active",
         "config":{"url":"https://example.com/wp-json/nightwatcher/v1/ingest",
                   "token":"<bearer>","site_id":"DSN003-S"}}'
  curl -s -b jar -X POST localhost:8080/api/v1/export-targets/wp-036/run   # push now

  Secrets in a target's config (tokens, OAuth keys) are stored server-side and returned masked (***).

WordPress connector — nightwatcher-wp

A companion WordPress plugin (separate repository) that receives the webhook push and gives a site read-only, public access to the data. The daemon POSTs to the plugin's authenticated REST endpoint (/wp-json/nightwatcher/v1/ingest); the plugin stores the readings (de-duplicated on sensor_id + ts_utc) and renders a date-range sky-brightness graph via a shortcode:

[nightwatcher_graph sensor="DSN003" days="7"]

Because NightWatcher pushes, the daemon can stay on a private LAN — nothing needs to reach inward to it. Generate the bearer token in the plugin's Settings → NightWatcher SQM and use it as the target's config.token.

Repository layout

src/sqm/      SQM device library (TCP + serial transports, protocol codec, discovery)
src/db/       database access layer (libmariadb)
src/auth/     password hashing (PBKDF2) + session tokens
src/api/      embedded HTTP/HTTPS server + REST handlers
src/weather/  pull-based weather providers (Ambient, Weather Underground)
src/export/   exporters (DSN .dat -> Google Drive, webhook push) + scheduler
src/daemon/   nightwatcherd (scheduler, config, logging, main)
src/cli/      sqmctl, nwdb, nwexport-auth
include/      public headers
web/          static web UI (index.html, css, js, vendored uPlot)
sql/          database schema + one-time setup
config/       example config, systemd unit, udev rule (templated .in files)
cmake/        ARM toolchain + install/uninstall helpers
debian/       .deb maintainer scripts (postinst/prerm/postrm) + debconf config/templates
docs/         architecture, SQM protocol, DSN notes
tests/        unit + integration tests

Documentation

• docs/architecture.md — system architecture and roadmap
• docs/sqm-protocol.md — Unihedron SQM command/response reference
• docs/dsn.md — Dark Sky Network notes

License

GPL-3.0-or-later. The project may link GPL/LGPL libraries (e.g. MariaDB Connector/C); the database client is MariaDB Connector/C (LGPL-2.1) rather than Oracle's libmysqlclient for clean GPLv3 compatibility.