Skip to content

Latest commit

 

History

628 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

HiveHub

Project renamed: HiveScale → HiveHub. The project outgrew its original "dual scale" scope and now acts as a data collector / hub for many different types of beehive sensors and scales (up to 16 hives per ESP32), so the name was changed to match. A few internal identifiers — the database measurement columns, the OTA target value, the Docker image name (…/hivescale-api), the device's stored-config (NVS) namespace and MQTT topics — still use the old hivescale name on purpose: changing them would need data/firmware migrations and could strand existing measurements or deployed-device config. The third-party beehivemonitoring.com "HiveScale" wireless weight scale is an unrelated product and keeps its own name.

Hardware status: all published PCBs are tested and working. The recommended build is the XIAO ESP32-C6 on the Scale Module V0.4 with NAU7802, MAX17048, TPS63020, TP4056, RTC, SD card and SHT40 — optionally expanded to 16 scales with the NAU7802 breakout PCB (TCA9548A mux + up to 8× NAU7802). See pcb-design/README.md. The old ESP32 30-pin board is obsolete and no longer recommended.

HiveHub is an ESP32-based data collector for beehive sensors and scales. It gathers weight, temperature, humidity, sound, vibration, power state, and network state from one or more hives (up to 16 per ESP32) and sends the readings to a self-hosted FastAPI backend backed by PostgreSQL, where they can be displayed in HivePal.

Natively supported sensors

Every sensor is optional and compiled in per device — HiveHub reads the following directly on the ESP32:

  • Ambient sensor (SHT4x / SHT3x / BME280) — one device-level outside-hive sensor on the shared I2C bus, selectable at build time: SHT4x/SHT40 (default) or SHT3x for temperature + humidity, or a BME280 which additionally reports barometric pressure (ambient_pressure_hpa).
  • DS18B20 — per-hive in-hive temperature probes (1-Wire).
  • INMP441 — in-hive sound via I2S MEMS microphones with per-band FFT.
  • MAX17048 — LiPo battery voltage, state-of-charge, and low-battery alerts (I2C).
  • Wired load cells via HX711 or NAU7802 — the hive scales themselves; NAU7802 over I2C (optionally behind a TCA9548A mux) scales to many channels for multi-hive setups.

On top of these wired sensors, HiveHub also bridges a range of wireless BLE/GATT sensors and scales (HolyIot, RuuviTag, HiveInside, and beehivemonitoring.com HiveHeart / HiveScale devices) — see Features below.

🌐 Website & setup guide: a small static site lives in website/ — a feature overview, a step-by-step setup guide, a complete "Build your own" guide (BOM with price estimates, wiring, firmware, backend — downloadable as PDF), and an in-browser secrets.h configurator that builds your firmware config (sensors, BLE/GATT options, power modules) without hand-editing macros. It deploys to GitHub Pages via .github/workflows/pages.yml (enable Settings → Pages → Source: GitHub Actions), e.g. https://macnite.github.io/HiveHub/.


Features

Every sensor is optional and compiled in per device — start with weight and add the rest.

  • Dual load cells — two HX711 amplifiers drive two independent hive scales (the one always-on measurement).
  • Backend load-cell temperature compensation — corrects HX711 thermal drift on read from the stored raw values; see docs/temperature-compensation.md.
  • Per-hive temperature — optional DS18B20 probes on a shared 1-Wire bus (off by default), or an in-hive BLE sensor as the source.
  • Per-hive in-hive sound — optional INMP441 stereo I2S microphones with per-band FFT (off by default).
  • Per-hive vibration — from a paired in-hive BLE sensor (a HiveInside nRF54LM20A gives full FFT bands; a HolyIot/RuuviTag beacon gives a low-rate magnitude), capturing the ~20 Hz pre-swarm signal microphones miss. (The old wired LIS3DH/LIS2DH12 driver has been removed — in-hive vibration is BLE-only.)
  • In-hive BLE sensors — pair up to two battery beacons (HolyIot 25015, RuuviTag, HiveInside nRF54LM20A, or beehivemonitoring.com HiveHeart) for temperature/humidity/pressure/vibration with no wiring. Note that a HiveHeart is read over GATT and accepts only one connection at a time, so HiveHub, the beehivemonitoring Hub and the beehivemonitoring app cannot share one — see beehivemonitoring-gatt.md.
  • Ambient temperature & humidity — one selectable device-level I2C sensor: SHT4x/SHT40 (default), SHT3x, or a BME280 that also reports barometric pressure (ambient_pressure_hpa). Picked at build time (ENABLE_SHT4X_AMBIENT / ENABLE_SHT3X_AMBIENT / ENABLE_BME280_AMBIENT).
  • RTC timekeeping — a DS3231 timestamps measurements without depending on NTP.
  • SD card cache & backup — local buffering when uploads fail, plus an append-only persistent backup that can be downloaded in AP mode and re-imported via HivePal.
  • Claim-code pairing — claim devices from HivePal without manual database setup.
  • Remote configuration & commands — sampling interval, scale offsets/factors, calibration, OTA checks, provisioning, reboot, Wi-Fi reset, and factory reset.
  • OTA firmware updates — owner-scoped releases with an accept-to-apply gate; the device also relays firmware over BLE GATT to a HiveInside sensor or a HiveTraffic counter. Both relays are version-gated and explicit: publishing a release never pushes it to a sub-device on its own.
  • Wi-Fi provisioning portal — opened by the setup button (or remotely with a start_provisioning command) for field configuration, including pairing wireless sensors.
  • Inspection mode — one press of the external button, a dashboard switch or a HivePal API call marks the window while a hive is open, so the 40 kg step from lifting two supers stays out of the charts, insights and alert rules. Nothing is deleted: the readings remain in the database and in every export.
  • Multi-network Wi-Fi — up to three saved networks.
  • Hive audio (optional) — ask a paired HiveInside node (firmware 0.6.2+) to record 10–60 seconds from inside the hive, then play it back in the dashboard. The hub collects the request on its next wake, so a recording arrives within one reporting interval rather than instantly. Authenticated with a shared key and stored as WAV on your own server. Useful for checking what an insight flagged — or just for listening to the bees. See Listening to a hive.
  • Insights — backend auto-evaluation of weight, temperature, sound, vibration, and entrance traffic per hive, based on these publications.
  • Insight alert notifications (optional) — get swarm / robbing / winter-risk alerts by e-mail (SMTP) and/or Web Push to your phone or an installable dashboard PWA (Android, iOS 16.4+, desktop) when an insight first fires or escalates. Off by default; see Insight alert notifications.
  • Optional off-grid mode — solar/LiPo charging (CN3791 MPPT + TPS63020) with MAX17048 LiPo telemetry.
  • Optional entrance bee counters — wireless HiveTraffic counters over BLE/GATT only (wired I2C BeeCounters are no longer supported).
  • Built-in web dashboard (optional) — a dependency-free dashboard served from the backend at /dashboard for single-owner self-hosts, protected by username + password login (first visit runs a setup wizard; admin/viewer roles): cross-device hive comparison, charts for every data group, plus device-config editing, hive renaming, firmware/OTA and calibration controls, SD-backup import, and user management. Off by default (ENABLE_LOCAL_DASHBOARD); see server/dashboard/README.md and the live demo.
  • Publish data for your website — from Device & admin → Publish data, publish one chart (a metric, the hives you pick, a rolling period) and embed it in a club page, blog or shop with a one-line <iframe> — or fetch the same slice as JSON/CSV and draw it yourself. Everything else stays behind the login: a published link exposes only the labels you typed, no device IDs or other readings, and can be revoked at any time. See Publish data.
  • HivePal integration — dedicated /api/v1/app/... endpoints using a HivePal service key, per-user JWTs, and per-user access roles.
  • Optional MQTT bridge — mirror every reading to an MQTT broker (Home Assistant, Node-RED, openHAB…) with Home Assistant auto-discovery, alongside the built-in PostgreSQL store. Off by default; see MQTT / Home Assistant integration.
  • PCB designs (tested and working) — KiCad ESP32-C6 Scale Module (recommended) and NAU7802 breakout board with fabrication outputs, plus the Power Module and legacy 30-pin Scale Module.
  • Docker Compose deployment — the API and PostgreSQL database.

Repository structure

HiveHub/
├── firmware/                   # ESP32 PlatformIO project (src/, include/)
├── server/                     # Python FastAPI backend, insights, migrations
│   ├── dashboard/              # Built-in login-protected web dashboard (served at /dashboard)
│   └── embed/                  # Public "Publish data" chart pages (served at /embed/chart/<token>)
├── docker/                     # Docker Compose deployment
├── website/                    # Static site + secrets.h configurator (GitHub Pages)
│   └── dashboard-demo/         # Backend-free dashboard demo (sample data)
├── docs/                       # Hardware, API, deployment, and feature docs
├── pcb-design/                 # KiCad breakout PCB design and fabrication files
├── test-data/                  # Mock server and decoder/insight unit tests
└── .github/workflows/          # CI: backend image build + website Pages deploy

Hardware

Core components

All links are affiliate links and support this project directly.

The recommended build is the XIAO ESP32-C6 on the Scale Module V0.4 — the complete bill of materials with price estimates and shop links lives on the Build your own page (also downloadable as CSV).

Component Role
Seeed Studio XIAO ESP32-C6 Main controller (recommended — plugs into the Scale Module V0.4 PCB)
NAU7802 (I2C, on the Scale Module / breakout PCB) + load cells Weight measurement (recommended — 2 scales on the Scale Module, up to 16 via the NAU7802 breakout PCB with its TCA9548A mux + up to 8× NAU7802)
SHT4x / SHT40 (or SHT3x / BME280) Ambient temperature and humidity (BME280 also adds barometric pressure)
DS3231 RTC Offline timekeeping (remove its 4.7 kΩ I2C pull-ups — see the I2C pull-up note in docs/wiring.md)
MicroSD card module + card Local cache and backup storage
MAX17048 LiPo battery gauge (on the Scale Module)
TPS63020 buck-boost Battery/5 V input to the regulated 3.3 V rail
TP4056 USB-C charger LiPo charging
Momentary pushbutton Provisioning and factory reset
IP-rated enclosure, glands, frame hardware Outdoor installation
ESP32 Dev Board + 2x HX711 Legacy 30-pin build (obsolete — no longer recommended)

Optional sensors (enable per device in secrets.h)

Component Firmware flag Role
2x DS18B20 waterproof probes ENABLE_DS18B20_HIVE_TEMP Internal hive temperature (or use an in-hive BLE sensor)
2x INMP441 sound sensors ENABLE_INMP441_MICS Internal hive sound with per-band FFT
MAX17048 ENABLE_MAX17048_BATTERY LiPo voltage, state-of-charge, and low-battery alert
In-hive BLE sensor (HolyIot 25015 / RuuviTag / HiveInside / HiveHeart) ENABLE_BLE_SCAN, ENABLE_BEEHIVE_GATT Temp / humidity / pressure / vibration, no wiring — paired by MAC
CN3791 solar charger · 10 Ah LiPo · 6 V 4.5 W solar panel Hardware only Off-grid charging path feeding the Scale Module's TPS63020

Optional bee counters

Entrance traffic counting (in/out bees) is BLE/GATT-only:

Firmware pin mapping

Pins are defined in firmware/include/config.h (with optional per-device overrides in secrets.h). The firmware source is split into focused units under firmware/src/ (main.cpp, hivehub_network.cpp, portal.cpp, sensors.cpp, scale_bus.cpp, i2c_bus.cpp, hive_config.cpp, mics.cpp, ble_sensor.cpp, beehive_gatt.cpp, bee_counter_client.cpp, storage_power.cpp, device_prefs.cpp, status_led.cpp, globals.cpp).

The table below is the legacy 30-pin ESP32 map. The recommended XIAO ESP32-C6 build (pio run -e xiao_esp32c6) has its own pin map — NAU7802 scales over I2C on D4/D5, DS18B20 on D1, SD on D3/D8–D10, button on D2, no HX711/INMP441 — see docs/wiring.md and pcb-design/README.md.

Signal GPIO Notes
HX711 #1 DOUT / SCK 16 / 17 Scale 1; SCK held high during deep sleep to power down the HX711
HX711 #2 DOUT / SCK 32 / 33 Scale 2; SCK held high during deep sleep to power down the HX711
DS18B20 1-Wire data 4 Shared bus for both probes; 4.7 kΩ pull-up to 3.3 V (ENABLE_DS18B20_HIVE_TEMP)
I2C SDA / SCL 21 / 22 RTC, SHT4x, NAU7802/TCA9548A, optional MAX17048 — shared bus at an explicit 100 kHz
SD CS / SCK / MISO / MOSI 5 / 18 / 23 / 19 MicroSD over SPI
Setup button 27 INPUT_PULLUP; short press opens provisioning AP, long press factory resets

On the XIAO ESP32-C6 (firmware 0.25.0+) the setup button is the board's on-board USER/BOOT button, and the external D2 button is the inspection button. The 30-pin board above is unchanged. Note that GPIO9 cannot wake the C6 from deep sleep and holding it at power-up enters the serial bootloader, so on that board press and hold the USER button and the AP opens a second into the next wake (up to one send interval away) — or press RESET, release it, then hold BOOT to get there in under a minute (never hold BOOT across the RESET: that is the flashing gesture). For a sealed hub use Start AP mode in the dashboard — see docs/ap-mode-sd-download.md. | INMP441 BCLK / WS / SD | 14 / 13 / 34 | I2S, shared by both mics; GPIO34 is input-only (ENABLE_INMP441_MICS) |

See docs/wiring.md for detailed wiring and pcb-design/README.md for the KiCad breakout PCB pinout.


Firmware setup

Prerequisites

  • PlatformIO (VS Code extension or CLI).

Configuration

cp firmware/include/secrets.example.h firmware/include/secrets.h

Edit firmware/include/secrets.h:

#define DEVICE_ID           "hive-001"
#define API_KEY             "your-api-key-here"   // unique per device — see note below
#define CLAIM_CODE          "ABCD-1234"
#define CLAIM_CODE_REVISION 1
#define API_BASE_URL        "https://your-backend-domain.com"   // HTTPS (TLS is verified)

#define WIFI1_SSID          "your-wifi-ssid-1"
#define WIFI1_PASS          "your-wifi-password-1"
// WIFI2_* and WIFI3_* are optional fallbacks

Values in secrets.h seed the device's persistent Preferences on first boot. Later changes are usually made through the backend or provisioning portal. Set FORCE_RESEED true only when you intentionally want to overwrite stored preferences from the build file.

Per-device API key: give each device its own unique API_KEY (generate one with openssl rand -hex 32). The backend registers the key against the device's device_id on first contact and rejects mismatches afterwards, so a leaked key only affects that one device. It no longer has to match the server's API_KEY environment variable — that value is now only the master/admin key for tooling.

TLS / certificate verification

The firmware verifies the backend's TLS certificate. It ships the ISRG Root X1 (Let's Encrypt) root CA in firmware/include/ca_cert.h and syncs the clock over NTP after connecting so validity can be checked. Therefore:

  • By default, the backend must be reachable over HTTPS with a valid certificate (a reverse proxy with Let's Encrypt is the simplest setup).
  • For a CA other than Let's Encrypt, replace the certificate in firmware/include/ca_cert.h (instructions are in that file).
  • NTP (UDP port 123) must be reachable from the device's network.

For a trusted LAN deployment that cannot use TLS, add the following firmware build setting to secrets.h and use an http:// API_BASE_URL (for example, http://192.168.1.10:31115). This is intentionally not a server environment variable: the ESP32 selects its transport before it can contact the server.

#define ALLOW_INSECURE_HTTP 1

The opt-in keeps HTTPS working and certificate-verified, but also permits HTTP for API requests and OTA downloads. HTTP exposes the per-device API key, measurements, commands, and firmware image to anyone able to observe or alter traffic, so use it only on an isolated, trusted network.

Optional modules

Optional sensors are enabled per build in secrets.h. The defaults shipped in secrets.example.h:

#define ENABLE_DS18B20_HIVE_TEMP 0   // wired in-hive temperature probes (default off)
#define ENABLE_INMP441_MICS      0   // stereo I2S mics + per-band FFT (default off)
#define ENABLE_BLE_SCAN          1   // in-hive BLE sensor bridge — HolyIot/RuuviTag/HiveInside (default on)
#define ENABLE_BEEHIVE_GATT      0   // beehivemonitoring.com HiveHeart / HiveScale over GATT (default off)
#define ENABLE_MAX17048_BATTERY  0   // LiPo fuel-gauge telemetry (default off)

The secrets.h configurator writes these (and the wireless-sensor macros) for you.

Cellular (SIM7080G) transport is no longer part of this firmware. LTE, solar, and battery handling now live on a separate Power Module that connects to the Scale Module over I2C/ESP-NOW. The ESP32 firmware itself is Wi-Fi only.

Download a prebuilt image (no toolchain)

Every CI run builds both targets and attaches the images to the run, and every tagged release carries them permanently:

Where What you get Keeps
Releases Images for the tagged version forever
Actions → CI → a run → Artifacts Images for that exact commit, incl. branches and PRs 90 days

Two files per board — esp32-c6 (XIAO, recommended) and esp32 (legacy 30-pin):

  • hivehub_<board>_<version>_full.bin — bootloader + partition table + otadata + application, written as a single file at offset 0x0. Use this for a blank board.
  • hivehub_<board>_<version>.bin — application image only. This is the file to upload to your backend for OTA, or to the dashboard's firmware uploader.
# XIAO ESP32-C6, blank board
esptool.py --chip esp32c6 --port /dev/ttyACM0 write_flash 0x0 hivehub_esp32-c6_0.24.7_full.bin

# Legacy 30-pin ESP32
esptool.py --chip esp32 --port /dev/ttyUSB0 write_flash 0x0 hivehub_esp32_0.24.7_full.bin

Prebuilt images contain no secrets.h: Wi-Fi, server URL and API key are set on the device through the setup portal, and the compile-time feature flags stay at their defaults. To change a flag (DS18B20, INMP441 mics, GATT sensors …) you still need a local build.

firmware/ci/package_build.sh <env> produces the same files locally after a pio run.

Flash from source

cd firmware
pio run --target upload
pio device monitor   # 115200 baud

PlatformIO dependencies

platformio.ini installs the required libraries automatically:

  • bogde/HX711, paulstoffregen/OneWire, milesburton/DallasTemperature
  • adafruit/Adafruit SHT4x Library, adafruit/RTClib, bblanchon/ArduinoJson
  • kosme/arduinoFFT — per-band FFT for the INMP441 mics and the vibration bands
  • h2zero/NimBLE-Arduino (2.x) — the in-hive BLE sensor bridge and GATT clients

Optional libraries:

  • sparkfun/SparkFun MAX1704x Fuel Gauge Arduino LibraryENABLE_MAX17048_BATTERY

Wi-Fi provisioning portal

Press the setup button (the on-board USER button on the XIAO ESP32-C6, GPIO27 on the legacy 30-pin board) to manage field configuration without reflashing. A hub already sealed in an enclosure can open the same portal remotely with a start_provisioning command from the dashboard or API.

Action Result
Short press Starts HiveHub-Setup-XXXX AP; open http://192.168.4.1
Long press, 10 seconds Clears stored Preferences and reboots

The portal is organised by hive: it edits Wi-Fi networks, backend URL, device ID, claim code and API settings, and manages the hive registry — add a hive, pick its scale source (auto-detected NAU7802 channels or a wireless HiveScale) and one in-hive sensor (a DS18B20 probe by ROM or a BLE/GATT sensor by MAC, with a built-in BLE scan). It also offers live scale calibration (tare + known weight, fully offline) and an SD-card download (TAR) of the on-device backup. It closes automatically after 10 minutes. See docs/multi-hive.md and docs/ap-mode-sd-download.md.


Server setup

Docker Compose

cd docker
cp .env.example .env
# edit API_KEY, HIVEPAL_SERVICE_API_KEY, HIVEPAL_JWT_SECRET, POSTGRES_PASSWORD,
# PUBLIC_BASE_URL, and the database volume path in docker-compose.yml
docker compose up -d

The API listens on port 31115 by default. API_KEY and POSTGRES_PASSWORD are required — compose refuses to start without them rather than falling back to insecure defaults. Uploaded OTA firmware binaries persist in the firmware-data volume.

Setting Default
API image ghcr.io/macnite/hivehub:latest
API port 31115
Database image postgres:16-alpine
Database name/user hivescale

See docs/docker-install.md and docs/truenas-install.md for full guides.

Manual / local

cd server
pip install -r requirements.txt
DATABASE_URL="postgresql://hivescale:password@localhost:5432/hivescale" \
API_KEY="your-master-admin-key" \
HIVEPAL_SERVICE_API_KEY="your-hivepal-service-key" \
HIVEPAL_JWT_SECRET="must-match-hivepal-jwt-secret" \
PUBLIC_BASE_URL="https://your-domain.example.com" \
uvicorn main:app --host 0.0.0.0 --port 8000

Environment variables

Variable Required Description
DATABASE_URL Yes PostgreSQL connection string
API_KEY Yes Master/admin key in X-API-Key for server-to-server tooling (firmware-release registration, command queueing, latest-measurements, time). Devices use their own per-device keys, registered on first contact.
HIVEPAL_SERVICE_API_KEY Yes, for HivePal Service key sent by HivePal in X-HivePal-Service-Key
HIVEPAL_JWT_SECRET Yes, for HivePal Shared HS256 secret used to verify the per-user Authorization: Bearer tokens HivePal sends. Must match HivePal's JWT_SECRET.
PUBLIC_BASE_URL Recommended Public base URL used for OTA firmware download links
FIRMWARE_DIR Optional Firmware binary directory, default /app/firmware
DB_POOL_MIN_SIZE / DB_POOL_MAX_SIZE Optional DB connection pool bounds (default 1 / 10)
RATE_LIMIT_ENABLED / RATE_LIMIT_DEFAULT Optional Per-client-IP rate limit (default on, 120/minute)
MAX_BODY_BYTES / MAX_FIRMWARE_BYTES Optional Request-body and firmware-upload size caps (default 256 KiB / 16 MiB)
ENABLE_LOCAL_DASHBOARD Optional Serve the built-in login-protected dashboard at /dashboard and its /api/v1/local/* API (default off). It serves every device on the server behind one login, so keep it off on multi-tenant deployments.
DASHBOARD_SESSION_SECRET / DASHBOARD_SESSION_TTL_HOURS / DASHBOARD_COOKIE_SECURE Optional Dashboard login-session settings: signing secret (auto-generated and persisted when blank), session lifetime (default 168 h), HTTPS-only cookie flag
ENABLE_PUBLIC_EMBEDS Optional Allow the dashboard's Device & admin → Publish data panel to serve individual published charts without a login, for embedding in a website (default on; nothing is public until an admin publishes it, and the panel is hidden when this is off). See Publish data.
TRUST_PROXY_HEADERS Optional Trust CF-Connecting-IP / X-Forwarded-For for rate-limit client IPs (default false; set true only behind a reverse proxy that overwrites these headers, otherwise a direct client can spoof them to dodge the limiter)
INSIGHTS_RECONCILE_* Optional Background insight-history reconciliation (see server/.env.example)
SMTP_* / NOTIFY_MIN_SEVERITY Optional E-mail channel for insight alert notifications (off by default — see Insight alert notifications)
WEB_PUSH_ENABLED / VAPID_* Optional Web Push channel for insight alert notifications (off by default — see Insight alert notifications)
MQTT_* Optional MQTT bridge to Home Assistant / Node-RED / openHAB (off by default — see MQTT / Home Assistant integration)
TZ Optional Server timezone, for example Europe/Berlin

The backend auto-creates tables and runs idempotent ALTER TABLE statements on startup; the SQL files in server/migrations/ can also be applied manually.


MQTT / Home Assistant integration

The backend can optionally mirror every measurement to an MQTT broker in addition to storing it in PostgreSQL — so HiveHub data flows into Home Assistant, Node-RED, openHAB or any MQTT consumer. It is off by default and purely additive: the bridge runs in a background thread and is fail-soft, so a broker outage never affects ingestion or the API.

Enable it by setting MQTT_ENABLED=true and pointing MQTT_HOST at your broker (see server/.env.example / docker/.env.example for the full list):

MQTT_ENABLED=true
MQTT_HOST=192.168.1.10        # your broker (e.g. the Mosquitto add-on)
MQTT_PORT=1883
MQTT_USERNAME=hivescale       # optional
MQTT_PASSWORD=...             # optional
MQTT_HA_DISCOVERY=true        # auto-create Home Assistant entities

Topics

With MQTT_BASE_TOPIC=hivescale (the default):

Topic Payload
hivescale/bridge/availability online / offline (bridge last-will)
hivescale/<device_id>/availability online / offline (per device)
hivescale/<device_id>/state Retained JSON of the latest reading (every non-null measurement field)

Home Assistant

When MQTT_HA_DISCOVERY=true, the bridge publishes MQTT-discovery configs the first time it sees each device. Home Assistant then automatically creates one device per scale with a curated set of sensors — scale weights, hive/ambient temperature & humidity, battery voltage/charge, solar power, Wi-Fi signal, and bee-counter totals. Any in-hive wireless modules (a beehivemonitoring.com HiveHeart/HiveScale or a HolyIOT BLE sensor) are exposed as their own Home Assistant devices, nested under the hub, each carrying its own full telemetry — temperature, humidity, battery and link signal (plus weight and pressure for the scale, and sound frequency/energy/peak for the HiveHeart). Those per-device temperature/humidity entities are published even though the hive's resolved temperature/humidity also appear on the hub device, so a hive fitted with several modules surfaces each one's readings separately. All other fields remain available in the raw state JSON for custom templated sensors. Just make sure the MQTT integration is set up and pointed at the same broker.

See docs/mqtt.md for the full topic layout, every configuration variable, and the per-hive / per-module Home Assistant discovery details.


API overview

Interactive Swagger docs are at http://<host>:31115/docs. See docs/api.md for the full reference and schemas.

Device-facing endpoints

Require the X-API-Key header (per-device key, except where noted as master/admin).

Method Endpoint Description
GET /health Health check (no auth)
GET /api/v1/time UTC server time for RTC sync (master/admin)
POST /api/v1/measurements Submit a measurement (incl. optional telemetry)
GET /api/v1/measurements/latest Latest measurements for dashboards (master/admin)
GET/PATCH /api/v1/devices/{id}/config Get / update device configuration
GET /api/v1/devices/{id}/firmware Check for a firmware update (?version=, ?target=hivescale, ?board=)
POST /api/v1/firmware/releases Register a firmware release (master/admin)
GET /firmware/{filename} Download a firmware binary
POST /api/v1/devices/{id}/commands Queue a remote command (master/admin)
POST /api/v1/devices/{id}/commands/update-hiveinside Queue a HiveInside OTA relay over BLE (?slot= hive index, ?force=)
GET /api/v1/devices/{id}/commands/next Claim next pending command
POST /api/v1/devices/{id}/commands/{cmd_id}/result Report command result

App endpoints for HivePal

Require both X-HivePal-Service-Key and a per-user Authorization: Bearer <hivepal-jwt> token (verified with HIVEPAL_JWT_SECRET).

Method Endpoint Description
POST /api/v1/app/devices/claim Claim a device by claim code
GET/DELETE /api/v1/app/devices · /{id} List devices · remove own membership (releases the device when it was the last one)
DELETE /api/v1/app/devices/{id}/claim Release the device outright: drop every member and unclaim it (owner)
GET/PATCH /api/v1/app/devices/{id}/config Get / update config (incl. tempco_*)
GET/PATCH /api/v1/app/devices/{id}/channels List / update scale display names
GET /api/v1/app/devices/{id}/measurements[/latest] Measurements (with date filter) / latest
POST /api/v1/app/devices/{id}/measurements/import Bulk-import SD-card backup rows (idempotent)
POST /api/v1/app/devices/{id}/temp-compensation/fit Fit a load-cell temperature coefficient
GET/POST/DELETE /api/v1/app/devices/{id}/members[...] List / share / revoke device access
POST /api/v1/app/devices/{id}/calibration/start · /stop Start / stop calibration mode
POST /api/v1/app/devices/{id}/firmware Upload a firmware binary (multipart) and register it
GET/POST /api/v1/app/devices/{id}/firmware/status · /approve OTA status / accept-to-apply approval
POST /api/v1/app/devices/{id}/commands/update-hiveinside Trigger a HiveInside OTA relay
GET /api/v1/app/devices/{id}/insights[/summary|/history] Rule-based colony insights, summary, and history

Local dashboard endpoints (optional, login-protected)

Enabled only when ENABLE_LOCAL_DASHBOARD=true; every route returns 404 when disabled. Access is gated by username + password login (HttpOnly session cookie): the first visit runs a setup wizard that creates the initial admin account, read endpoints need any valid session, and write/control endpoints need the admin role. The API is not scoped per user — one login covers every device on the server — so keep it off on multi-tenant deployments. These endpoints power the built-in /dashboard UI; see server/dashboard/README.md.

Method Endpoint Description
GET/POST /api/v1/local/auth/status · /setup · /login · /logout Session status / first-run wizard / sign in / sign out
POST /api/v1/local/auth/password · /email Change own password / set alert e-mail
GET/POST/DELETE /api/v1/local/auth/users[/{id}] Manage dashboard accounts (admin)
GET /api/v1/local/devices List all devices + hive display names
PATCH /api/v1/local/devices/{id}/visibility Hide / show a device in the hive picker (admin)
GET /api/v1/local/devices/{id}/measurements[/latest] Measurements (date filter, down-sampled for charts) / latest
GET /api/v1/local/export/measurements · /summary Download readings as an .ndjson backup (?device_id=, ?hive=, date filter, admin) / what the download would contain
POST /api/v1/local/devices/{id}/measurements/import Import an SD-card backup or a downloaded one (.ndjson / .tar, admin; route_by_device=true restores a multi-device backup)
POST /api/v1/local/devices/{id}/measurements/delete Delete readings in a time range (admin + device claim code)
POST /api/v1/local/devices/{id}/delete Erase a device and all of its rows (admin + device claim code + typed confirm_device_id; power the device down first or it re-registers)
POST /api/v1/local/devices/{id}/reseed Re-register a device under a claim code so HivePal can claim it again (admin; creates the device row if it is gone, releases one left claimed by nobody, refuses one still claimed in the app)
GET/PATCH /api/v1/local/devices/{id}/config Read / update device config (interval, scale offsets/factors, temp comp)
GET/PATCH /api/v1/local/devices/{id}/channels Read / rename the hive display names
GET /api/v1/local/devices/{id}/insights/summary · /history Highest-severity insight summary / persisted alert history
GET/POST /api/v1/local/devices/{id}/firmware/status · (upload) · /approve OTA status / upload binary / approve (admin)
GET /api/v1/local/publish/metrics Metrics that can be published as a public chart
GET/POST/PATCH/DELETE /api/v1/local/publish/charts[/{id}] List / publish / edit / revoke public chart embeds (writes: admin)
POST /api/v1/local/devices/{id}/hiveinside/update Queue a HiveInside BLE OTA relay to the node on ?slot= (hive index; ?force=, admin)
POST /api/v1/local/devices/{id}/calibration/start · /stop Start / stop calibration mode (admin)
POST /api/v1/local/devices/{id}/temp-compensation/fit Fit a load-cell temperature coefficient (admin)
GET /api/v1/local/notifications/config Which alert channels are enabled + VAPID public key
POST /api/v1/local/notifications/subscribe · /unsubscribe Register / forget this browser's Web Push subscription
POST /api/v1/local/notifications/test Send a test alert over every enabled channel

Public embed endpoints (no login)

The only routes on the server that need no credentials. Each serves exactly one chart an admin published through Publish data above, addressed by an unguessable token, and nothing else — no device IDs, no other readings, no access to any other endpoint. They 404 when ENABLE_PUBLIC_EMBEDS=false, when the publication is taken offline, and when it is revoked. See docs/publish-embed.md.

Method Endpoint Description
GET /embed/chart/{token} Self-contained page for <iframe> embedding in a website
GET /api/v1/public/charts/{token} The published chart as JSON (Access-Control-Allow-Origin: *)
GET /api/v1/public/charts/{token}.csv The same points as CSV

Measurement payload highlights

Core fields include weights, hive/ambient temperatures and humidity, raw HX711 values, firmware/config version, sensor status, boot count, and time source. Builds with optional hardware can also send:

  • Acoustic (INMP441): mic_ok, per-channel mic_left_* / mic_right_* RMS/peak levels and per-band FFT energy (*_band_sub_bass_dbfs, *_band_hum_dbfs, *_band_piping_dbfs, *_band_stress_dbfs, *_band_high_dbfs).
  • In-hive BLE sensor: hive_N_humidity_percent, ble_N_humidity_percent, ble_N_pressure_hpa, and the beehivemonitoring.com hiveheart_N_* / hivescale_N_* fields.
  • Vibration: accel_N_ok, broadband accel_N_rms_mg / accel_N_peak_mg, and per-band energy accel_N_band_swarm_mg (8–30 Hz) / _fanning_mg (30–100 Hz) / _activity_mg (100–200 Hz) — populated by the in-hive BLE sensor.
  • Entrance traffic (HiveTraffic BeeCounter, BLE/GATT): bee_counter_1_* / bee_counter_2_* lifetime totals, gate health, and status fields; the backend derives the interval in/out counts by differencing consecutive totals.
  • Power telemetry: battery_* (MAX17048) fields; the backend also still accepts the legacy solar_* fields.
  • Status: network_transport, calibration_mode, boot_count, time_source.

The backend accepts network_transport for the future Power Module; on-device firmware reports network_transport: "wifi". (The old cellular_ok / cellular_csq fields were removed from the schema and are silently ignored if sent.) Fields are stored in dedicated PostgreSQL columns (plus raw_json) and returned through the latest-measurements and HivePal app APIs.


Claim-code pairing

  1. Set CLAIM_CODE in secrets.h before flashing, for example ABCD-1234.
  2. The firmware includes the claim code in every measurement until the server confirms the device is claimed (the upload response carries "claimed": true), then stops sending it to limit exposure. Confirming rather than assuming matters: a rebuilt or restored backend has no record of the device, and a device that stopped sending its code could never be claimed again.
  3. The backend stores a hash of the claim code and creates an unclaimed device record on the first measurement.
  4. HivePal (or another app client) calls POST /api/v1/app/devices/claim with the code.
  5. The matched device is assigned to the user as owner.

Un-pairing and re-pairing

The pairing is fully reversible, from either end:

  • Removing the device in the app deletes your membership, and releases the device when you were the last member — claimed_at is cleared so the same claim code pairs it again. An owner can release a shared device in one step with DELETE /api/v1/app/devices/{id}/claim instead of waiting for every member to remove themselves.
  • The device notices on its next upload (firmware 0.24.9+): the server answers "claimed": false, the firmware drops its local "claim registered" latch, and the claim code starts flowing again automatically. No reflash, no factory reset.
  • Re-submitting the claim code in the setup portal also clears that latch (0.24.9+), which is how a device that latched under older firmware is brought back without a factory reset.
  • Older firmware (≤ 0.24.8) never clears the latch by itself, so a device that already stopped sending its claim code needs one of: the portal route above after an OTA to 0.24.9+, a CLAIM_CODE_REVISION bump plus re-flash, or a factory reset.
  • Re-seeding from the dashboard covers the cases where the server itself no longer holds the claim code and the app can only answer "no unclaimed device found": the device row was deleted here as well, the row was re-created by firmware that had already stopped sending the code, or the device was re-flashed with a new code that never replaced the stored one. Device & admin → Admin → Re-seed to HivePal (admin) takes the device ID and claim code and registers them exactly as a newly flashed device's first upload does, leaving the device unclaimed and claimable — no factory reset, no trip to the setup portal. It refuses a device that is still claimed in the app; remove it there first.
  • Readings survive an un-pair, so re-claiming restores the history. To erase a device for good, use the local dashboard's Delete device (admin + claim code + typed device ID).

Databases that predate this behaviour may hold devices left claimed with no members — claimed by nobody, visible to nobody, and un-claimable. Run server/migrations/022_release_orphaned_devices.sql once to release them; it only touches devices with zero members and changes no readings.

To push a new claim code through OTA, change CLAIM_CODE, increment CLAIM_CODE_REVISION, and publish a new firmware build.


Remote commands

Commands are queued by the server and picked up by the device on its next cycle.

Command type Payload Description
calibrate_scale_1 / calibrate_scale_2 {"known_weight_kg": 10.0} Calibrate a scale with a known weight
start_calibration_mode {"interval_seconds": 5, "timeout_seconds": 600} Temporarily use fast measurement cycles
stop_calibration_mode {} Return to normal interval and deep sleep
reboot {} Restart the ESP32
check_ota / ota_update {} Trigger an immediate OTA check
update_hiveinside {"slot": 1, "url": "...", "version": "...", "crc32": 123} Relay firmware to a HiveInside sensor over BLE GATT (usually via the update-hiveinside helper)
update_beecounter {"slot": 1, "url": "...", "version": "...", "crc32": 123} Relay firmware to a HiveTraffic counter over BLE GATT (usually via the update-beecounter helper). The counter stops counting for the transfer
start_provisioning {} Start the Wi-Fi provisioning AP
reset_wifi {} Clear saved Wi-Fi credentials and reboot
reset_preferences / factory_reset {} Clear all Preferences and reboot

PCB design

The pcb-design/ directory contains the KiCad designs — all published boards are tested and working:

  • ESP32-C6 Scale Module (V0.4, recommended) — the central board. Off-the-shelf modules on pin headers (no SMD soldering): XIAO ESP32-C6, NAU7802 for 2 scales, DS3231 RTC, SHT40, SD module, DS18B20 bus, MAX17048 battery gauge, TP4056 USB-C charger, and TPS63020 buck-boost regulator with a power-source selection jumper.
  • NAU7802 breakout PCB (v0.2) — I2C frontend for up to 16 wired scales (TCA9548A mux + up to 8× NAU7802; when it is used, no NAU7802 goes on the Scale Module); optionally carries its own XIAO MCU as a standalone BLE scale sensor.
  • Power Module (V0.3) — off-grid power (solar, battery) for a Scale Module; probably discontinued soon.
  • ESP32 30-pin Scale Module (V0.3) — the legacy board (ESP32 DevKit + 2× HX711 + INMP441 mics); obsolete and no longer recommended.

Start with pcb-design/README.md for the recommended setup, connector pinouts, fabrication files, and assembly notes.


Documentation

A full index is in docs/README.md. Highlights:


Testing and quality checks

CI (.github/workflows/ci.yml) builds both firmware targets, runs the host-level I2C-hardening and BLE-decoder tests, the Python server suite, and syntax-checks every website/dashboard script. To run the same checks locally:

# Server / insights suite (no database needed)
pip install -r server/requirements.txt pytest
DATABASE_URL=postgresql://unused/unused PYTHONPATH=server python3 test-data/test_counter_rules.py
DATABASE_URL=postgresql://unused/unused PYTHONPATH=server python3 test-data/test_accel_rules.py
DATABASE_URL=postgresql://unused/unused PYTHONPATH=server python3 test-data/test_ble_sensor_rules.py
DATABASE_URL=postgresql://unused/unused PYTHONPATH=server python3 test-data/test_sd_import.py
DATABASE_URL=postgresql://unused/unused PYTHONPATH=server python3 -m pytest -q test-data/test_tempcomp.py test-data/test_hive_tempcomp.py test-data/test_hiveheart_fft.py test-data/test_relay_ota_gate.py

# Claim / release lifecycle (needs a real PostgreSQL — it builds the schema and
# calls the endpoints for real). Skips with a message when the DB is unreachable;
# pass --require-db to make that a failure instead, as CI does.
docker run --rm -d -p 5433:5432 -e POSTGRES_PASSWORD=postgres -e POSTGRES_DB=hivehub_test --name hivehub-test-db postgres:16
DATABASE_URL=postgresql://postgres:postgres@localhost:5433/hivehub_test PYTHONPATH=server python3 test-data/test_pairing_lifecycle.py --require-db
docker rm -f hivehub-test-db

# Firmware host tests (plain g++, no Arduino toolchain)
./firmware/host_test/run_tests.sh
g++ -std=gnu++17 -I firmware/include -o /tmp/t1 test-data/test_ruuvi_decode.cpp   && /tmp/t1
g++ -std=gnu++17 -I firmware/include -o /tmp/t2 test-data/test_beehive_decode.cpp && /tmp/t2

# Firmware builds (needs PlatformIO)
cd firmware && pio run -e esp32dev && pio run -e xiao_esp32c6

curl examples for exercising a running backend are in docs/test-commands.md; a mock server with sample data lives in test-data/mock-server/.


Troubleshooting

  • Device shows no data — check docker compose logs hivescale-api for ingest errors; verify the device's API_BASE_URL is HTTPS with a valid certificate (the firmware verifies TLS) and NTP (UDP 123) is reachable.
  • OTA never triggers — updates need three things: the release's board must match the device (esp32 vs esp32-c6), the version must be newer, and the owner must have approved it (accept-to-apply gate). Also make sure PUBLIC_BASE_URL is set, otherwise download URLs are relative.
  • Firmware downloads 404 after a container rebuild — uploaded binaries live in FIRMWARE_DIR; make sure the firmware-data volume (or a bind mount) is attached.
  • Dashboard says "Local dashboard is disabled" — set ENABLE_LOCAL_DASHBOARD=true on the API container.
  • Locked out of the dashboard — accounts live in the dashboard_users table; deleting all rows re-runs the first-visit setup wizard.

Contributing

Issues and pull requests are welcome. Please run the test suite above before submitting, keep the docs in sync with behavior changes, and note that CLAUDE.md documents repository conventions for AI-assisted contributions. To request support for a new hive sensor, see docs/device-not-supported-yet.md.

Reporting security issues

Please do not open public issues for security vulnerabilities. Report them privately to the maintainer (see the commit history / GitHub profile for contact) so a fix can be released first.

License

MIT © 2026 Maximilian Nitschke

About

Open-source ESP32-C6 beehive monitoring hub for scales and sensors. Collects weight, climate, sound, vibration and bee traffic from up to 16 hives, with BLE, OTA, FastAPI/PostgreSQL backend, dashboards, MQTT and tested KiCad hardware.

Topics

Resources

Stars

6 stars

Watchers

0 watching

Forks

Releases

Sponsor this project

Packages

Contributors

Languages