4.8.1.beta

[va13k]

4.8.1.beta — Changes

(4.8.0.beta)

Scope

This branch collects the changes needed before the client could be used for an actual study. The previous state did not prevent the app from running; it prevented a study from producing trustworthy data, which is a different and harder problem to notice.

Three groups of change are included:

1. Data that was not being collected or delivered. An on-device schema upgrade had been failing for several releases; location collection stopped permanently at the first configuration update; a single table's upload failure was cancelled out by the other tables succeeding; an upload that hung stalled every table without reporting anything. Each of these produced no crash and no visible evidence.
2. Configuration that could silence a sensor without saying so. Sensitivity thresholds were entered as bare numbers with no unit and no range, and values larger than a sensor can physically produce filter out every sample while the sensor still reports itself as enabled.
3. Information the participant and the researcher did not have. Whether data is reaching the research database, which sensors are not delivering, and when the study needs something from the participant.

The common property of the first group is worth stating, because it determined where the effort went: these failures were silent. The app reported itself healthy throughout. They were found by comparing what a device had stored against what it claimed, not by using the app.

Status

Collection and delivery are substantially more reliable than before, and the failure modes that remain are documented rather than unknown. The work is not finished. The most significant outstanding items are architectural rather than incidental:

• Data is uploaded by connecting directly to MySQL over JDBC. The intended direction is HTTP(S) to the companion micro-server with a per-participant token. That single change would resolve several limitations listed below at once: the shared database credential that cannot be revoked per participant, the certificate authority that has to be distributed through the study configuration, the plaintext configuration fetch, and the single shared connection that serialises every table's upload.
• The screenshot sensor needs revision. Its upload is bounded, but its on-device storage is not, and it is the highest-volume sensor in the app by a wide margin.
• The Processor sensor should be removed. It reads /proc/stat, which Android has restricted since version 7, so it cannot collect on any currently supported device. It is disabled rather than removed, and still reports itself as active when started.

Two server-side database columns are required before this build is installed. The research
database and the study configuration it reads are provisioned by the companion dashboard project,
va13k/aware_dashboard, where these statements belong.
See Server-Side Prerequisites.

Principal Changes

Grouped by what they affect. Detail for each is in the sections below.

Collection and delivery correctness

• Fixed the on-device database never upgrading: a schema change was aborted part-way and silently rolled back, so several releases' worth of database changes had never reached a phone that already had AWARE installed.
• Fixed location collection stopping permanently at the first study-configuration update, and never resuming while the app kept running, while the sensor still logged itself as active.
• Fixed the magnetometer comparing one axis against the previous values of two other axes, which made any non-zero magnetometer sensitivity keep and drop readings unpredictably.
• Fixed a single sensor's delivery failure being cancelled out by the other sensors still delivering: delivery state is now tracked per sensor rather than as one flag for the whole upload.
• Fixed an upload waiting indefinitely when the research database accepts a connection and then stops answering, which stalled every table and reported nothing.
• Fixed sensors reporting that they had never collected anything once their data had been uploaded and cleared from the device.
• Fixed Network Traffic collection on current Android devices.

Configuration

• Replaced raw sensitivity-threshold entry with per-sensor presets stated in each sensor's own unit, and refused values larger than the sensor's readings ever change by.
• Added custom hardware-sensor sampling rates in Hz with live interval conversion.
• Sensors whose hardware the device does not have are excluded when a configuration is applied, not only on first sync.
• A study configuration is applied even while the research database is unreachable; only the upload waits for the database to return.
• Made study-join failures specific, distinguishing an invalid configuration, a missing or incorrect password, and an unreachable database.
• Made manual study updates reviewable before they replace an editable configuration.

Participant-facing

• Added persistent, per-sensor study consent, shown before joining rather than after, covering plugins as well as sensors.
• Added a researcher-controlled editable mode with participant-owned sensor settings.
• Added actionable collection states for collecting, delayed, blocked, event-driven, disabled, and unavailable sensors.
• Added a notification when the study needs something from the participant: a study password, or consent for newly added sensors. A frequency-only change is applied without interrupting them.
• Added a main-screen indication of how far collected data has reached the research database, naming any sensor that is not delivering.
• Added participant re-authentication when a researcher changes the study database password.
• Made leaving a study always succeed on the device, with researcher notification treated as best-effort.
• Clarified the ongoing background-collection notice and made it reliably visible on current Android versions.

Upload path

• Made upload batches land whole or not at all, so a partly delivered batch can no longer be duplicated by a retry.
• Bounded upload batches by payload size as well as row count, so a backlog of large rows cannot produce a batch too big to send.
• Made upload progress durable, so a table is no longer re-uploaded from the beginning whenever the study log was cleared.
• Made every upload encrypted, and added verification of the research database's identity for any study that publishes its database's certificate authority in its configuration. A study that publishes none still gets an encrypted connection, but the host it connects to is not verified.

Data model

• Reshaped device information into one row per device, rewritten when the device's hardware or Android version actually changes.
• Added an entry kind to the study log, so restarts, schedules, uploads, study changes, and sensor diagnostics can be counted separately.
• Added frequency-aware sensor-health reporting to the research database.
• Added Bluetooth connection-state data.

Security

• Removed database passwords from application logs, including the connection log.
• Masked password, secret, and token values in logged study configuration and study data.
• Removed a device-information ping that was sent to a third-party server.

General fixes

• Fixed background timers that consumed a CPU core for hours and could prevent a study from being joined.
• Reduced unnecessary preference refreshes and full-screen recreation.

Server-Side Prerequisites

Two columns must exist in the research database before this build is installed. The client writes both, and MySQL rejects an insert naming a column the table does not have — which fails the entire batch for that table, so aware_log and aware_studies would stop uploading until the columns exist.

ALTER TABLE aware_studies ADD COLUMN double_updated DOUBLE DEFAULT 0;
ALTER TABLE aware_log     ADD COLUMN log_type VARCHAR(32) DEFAULT '';

Both statements are backwards compatible: the columns are nullable with defaults, so a device still running an earlier build continues to upload normally. Applying them ahead of the app update is therefore the safe order, and no coordinated downtime is needed.

These belong with the rest of the research-database schema in the companion dashboard project, va13k/aware_dashboard.

No further server columns are needed. The per-sensor delivery state introduced here is held on the device only — the client's account has INSERT only and could not maintain mutable remote status, so what the researcher sees arrives through aware_log, which already uploads.

One column to verify rather than add: bluetooth.bt_status shipped with 4.8.0.beta, but a research database that predates it and was never migrated will reject every bluetooth insert. That is not hypothetical — it silently discarded two hours of Bluetooth data on a live study, because a single table's upload failure was masked by the other tables succeeding. Confirm the column exists before enrolling devices:

SHOW COLUMNS FROM bluetooth LIKE 'bt_status';

Two further server-side notes, neither of which blocks installation:

• Every table in the research database was confirmed to use InnoDB, and innodb_flush_log_at_trx_commit is 1. Both are required for the all-or-nothing upload batches described below to mean anything: MyISAM accepts a transaction and ignores it.
• The client's database account needs INSERT only. Any SELECT or UPDATE granted to it is unused by the app — every read and update the client performs is against its own on-device database.

Silent Failures Found and Fixed

• The on-device database never upgraded. A schema migration was aborted part-way and silently rolled back, so several releases' worth of changes had never reached a phone with an existing installation. Upgrades now complete and keep existing data, a failed upgrade is recorded instead of discarded, and the cached database handle is dropped so a retry re-attempts rather than serving a stale schema indefinitely.
• Location collection stopped at the first configuration update and never resumed — for the life of the process — while dumpsys showed the service running and the log read Location tracking with GPS is active: 30s. The GPS, network and passive providers are cancelled when the sensor restarts, but the static frequencies that guard re-registration were not reset, so the restarted sensor saw "no change" and registered nothing. Verified on a device: before the fix, zero rows in 100 seconds after a restart; after it, five rows at exactly the configured 30-second interval.
• The magnetometer compared its X axis against the previous Y and Z values, so any non-zero magnetometer sensitivity kept and dropped readings unpredictably.
• A single sensor's delivery failure was hidden by the other sensors still delivering. Upload health was one flag, set by any table's failure and cleared by any table's success, while ~30 sync adapters run in parallel. It is now tracked per table: a sensor's success clears only its own outage, and the main screen names the sensors that are not delivering.
• An upload that hung was never reported. The shared database connection had no read timeout, so a server that accepted a connection and then went quiet held the upload lock until TCP keepalive gave up — on the order of hours — while every other table waited behind it. Because the call returned neither success nor failure, nothing recorded or reported the stall. The connection now has bounded connect and read timeouts, and one connection-level failure skips the remaining tables for that cycle instead of each re-proving it.
• Data that had been uploaded and cleared was reported as never collected. The status and the
  researcher diagnostics now count delivered data as data the sensor collected.

Sensitivity Thresholds

A sensitivity threshold is a change filter: a reading is stored only when it differs from the last
stored reading by at least the threshold, in the sensor's own native unit, and on the three-axis
sensors only when every axis is within it. Entering it as a bare number invited values no sensor can
produce — a deployed study configuration carried threshold_accelerometer: 120 m/s² and
threshold_magnetometer: 1000000 µT, which filter out every sample and leave the sensor silent while
it still reports itself as enabled.

• Replaced raw numeric entry with per-sensor presets stated in the sensor's own unit — m/s², rad/s,
  µT, hPa, lux, °C — each naming what it keeps and what it drops, and anchored to published
  smartphone sensor-noise figures rather than an arbitrary scale.
• A typed-in value larger than the sensor's readings ever change by is refused, and one already
  set that way is reported as collecting nothing rather than displayed as a bare number.
• Proximity offers only "record every near/far change", because its hardware reports two quantised
  states and only on change — there is nothing in between for a threshold to remove.
• Light keeps a deliberately loose limit: illuminance genuinely spans five orders of magnitude, so a
  coarse light threshold is defensible where a coarse motion threshold is not.

The same presets were added to the study Configurator, since that is where the values are authored.

Permissions and Study Consent

Permission and consent requests now follow the sensors a study actually needs instead of presenting every possible request at startup.

• Startup requests only the permissions needed by the core app and sync framework.
• The legacy client path was aligned to enable only necessary sensors instead of starting unrelated collection.
• A per-sensor consent screen lets participants review each requested data source before enrollment is finalized.
• The consent screen covers the plugins a study requires as well as its sensors, presented in labelled sections rather than one undifferentiated list.
• Consent decisions are stored per study and sensor, enforced when configurations are applied, and retained across Activity recreation and app restarts.
• Runtime permissions are requested sequentially; already granted permissions are skipped and each missing permission is requested at most once per sequence.
• Accessibility and Location settings prompts are sequenced so one request does not obscure or interrupt another.
• If Android no longer allows an in-app permission prompt, the participant receives an explanation and a route to the application settings.
• Denying a permission no longer causes the requesting service to restart repeatedly or creates a permission-dialog loop.
• Fixed repeated Accessibility requests.
• Hardened Bluetooth behavior when runtime permissions are missing or revoked.
• Removed the unnecessary READ_CONTACTS request.
• Corrected GET_ACCOUNTS handling so it is requested only on Android versions where the legacy sync framework requires it.
• Wi‑Fi scanning now detects when Android Location services are disabled and guides the participant to enable them.
• Consent is not incorrectly activated for program joins that do not require the study-consent flow.
• Added missing sensor consent mappings and centralized consent checks so join, update, and editable-mode behavior agree.

Editable and Locked Study Modes

Researchers continue to control whether participants may edit study settings through enable_config_update.

Locked mode

• The server configuration remains authoritative.
• Participants see only sensors enabled for the study.
• Local attempts to change researcher-controlled settings are rejected.

Editable mode

• All supported and unsupported sensor options remain visible.
• Participant changes are persisted into the active study configuration rather than treated as temporary drift.
• Opening the app and scheduled configuration checks do not silently restore the server configuration.
• Each participant sensor change updates the effective study configuration and stamps the time it changed, then adds a compliance record naming the change. Researchers reconstruct the configuration in force at any moment from the most recent configuration at or before that time, so subsequently collected data remains interpretable without every compliance record restating the whole configuration.
• The sensor list, status text, and icon state update in place without recreating the whole Activity.

Manual study updates

• Check for study updates is the explicit way to compare an editable configuration with the server.
• The participant sees which sensors the server proposes to activate or deactivate before anything changes.
• Agree and update applies only the exact configuration the participant reviewed.
• Keep my settings retains the current device-specific configuration.
• Leave the study opens a separate confirmation before any exit action is taken.
• If the server configuration changes between preview and acceptance, the app requests a new review instead of applying an unseen version.
• Accepting an update opens the consent flow when newly enabled sensors require additional consent or permission.

Sensor Availability and Collection Status

The interface now distinguishes configuration state from actual data collection.

• Added a centralized hardware detector for physical sensors.
• Hardware checks used by the UI and configuration application now share the same helper instead of maintaining separate rules.
• Unsupported sensors remain visible in editable mode with an explicit explanation such as “This device has no gyroscope sensor.”
• An unavailable sensor cannot be activated: its checkbox is disabled, stale enabled values are cleared, and a write-time guard rejects activation through another UI path.
• Hardware-blocked sensors are excluded when a server configuration is applied to the device.
• Opening a sensor in editable mode shows an inline, non-editable collection-status row.
• Participant-facing states include Collecting, Delayed, Waiting for first sample, Enabled and waiting for an event, Blocked, Unavailable on this device, and Disabled.
• Blocked states identify missing runtime permission, Accessibility, or Android Location requirements.
• Status details include the latest sample time and, where possible, a concrete action the participant can take.
• Sensor icons use the same collection state as the status text, avoiding conflicting indicators.

Frequency and Sampling Settings

• Replaced ambiguous free-text hardware-sensor frequencies with sensor-specific presets and a Custom value… option.
• Added appropriate presets and defaults for individual sensors.
• Custom hardware-sensor rates are entered and displayed in Hz, including decimal values such as 2.5 Hz.
• The custom-rate dialog explains that Hz means samples per second and displays the corresponding interval live—for example, 20 Hz is one sample every 50 ms.
• Hardware-sensor rates are converted to Android's required microsecond sampling period only when stored; polling and scan frequencies remain expressed in seconds.
• Applied the same preset/custom interaction to Ambient Noise and OpenWeather settings.
• Corrected descriptions that previously confused sampling periods, polling intervals, and upload frequency.
• Centralized time-unit conversions and applied them consistently across hardware sampling, Location, Bluetooth, Wi‑Fi, Processor, Applications, and Network Traffic.

Frequency-Aware Sensor Health

• Sampled sensors are evaluated against their configured collection frequency rather than a fixed 30-minute timeout.
• A sample remains healthy for three expected intervals, with a two-minute minimum and one-day maximum to avoid impractically narrow or broad health windows.
• Event-driven sensors are reported as enabled and waiting for an event instead of becoming falsely delayed during a quiet period.
• Added a dedicated diagnostic utility shared by participant status and researcher logging.
• The existing ten-minute study-compliance job writes parseable sensor_status records to aware_log.
• Each record includes the sensor state, device setting, latest sample timestamp, expected freshness window, exclusion state, and reason.
• The same diagnostic records are emitted immediately after joining a study and after applying a configuration update.
• These diagnostics use the existing synchronized log table and add no table of their own. They are written with the entry kind diagnostics, so they can be selected without matching message text — which requires the log_type column from Server-Side Prerequisites.

Study Join, Information, Update, and Leave

• Built the foundation for study-information and enrollment-history improvements.
• Redesigned the study information experience with a reusable study card and a clearer joined-study list.
• Fixed join, leave, and rejoin tracking so the active enrollment is selected by enrollment state rather than a stale URL or historical row.
• Fixed the quitting process and synchronization of study-exit state.
• Fixed study configuration serialization and restoration during reset.
• Corrected configuration synchronization and ensured the current configuration state is saved.
• Participants are reliably notified when a study configuration changes, including when no study screen was open during synchronization.
• Configuration updates apply only changed settings and report which sensors were activated or deactivated.
• Missing or differently shaped sensor settings are parsed through shared helpers instead of being dropped or causing update crashes.
• Live settings that drift from an otherwise unchanged locked configuration are reconciled safely, with backoff to avoid repeated service restarts for unsatisfiable hardware.
• When several configuration checks are triggered at once, the app now runs one check instead of queuing every request. Repeated screen taps and restarts can no longer pile up a backlog of outdated checks.
• The app now stops waiting for a study configuration that does not answer in time, and refuses a configuration file larger than 5 MB. A slow or oversized file can no longer leave the app waiting or use up its memory.

Joining a study

• The study address is validated before anything is downloaded, so a missing, malformed, or non-web address produces a message instead of an unexplained failure.
• Join failures are now distinguished from one another: an unreachable or invalid configuration, a required password that was not supplied, an incorrect password, and an unreachable study database each produce their own explanation.
• An unreachable research database is no longer reported to the participant as an incorrect password.
• A study opened from a link or QR code that requires a participant-supplied password now asks for it, with the study address already filled in, instead of ending in a failure the participant cannot act on. Such studies could not previously be joined by link or QR code at all.
• Validation runs entirely in the background, and progress and result messages are delivered only while the originating screen is still present.
• Validation no longer waits indefinitely on an unresponsive database; the connection check is bounded by a short timeout.
• Every join entry point—the in-app dialog, study links, and QR codes—now uses the same validation path and produces the same result for the same input.

Study database password

• The app detects when a study that relies on a participant-supplied database password has had that password changed by the researcher, and distinguishes a rejected password from a database that is merely unreachable.
• Only a rejected password prompts the participant to re-enter it. A database outage never produces a password prompt.
• Data upload pauses while re-authentication is pending and resumes once the new password is accepted; collection continues in the meantime, so no data is lost.
• The participant is prompted immediately when a study screen is open, and on next app open otherwise.

Leaving a study

• Leaving requires explicit confirmation explaining that collection stops, local study settings are removed, and previously uploaded data remains on the server.
• The app uploads only the quit study exit record rather than waiting for every provider to finish a full synchronization.
• Leaving always completes on the device. A participant can withdraw even when the research database is temporarily unavailable or permanently gone.
• Notifying the researcher is best-effort and bounded by a short timeout, so an unreachable database cannot delay or prevent a withdrawal.
• The exit is recorded locally either way. When the researcher could not be notified, the exit record says so and the participant is told plainly rather than left to assume the withdrawal failed.

Data Upload and Delivery

The connection to the research database, and the batches sent over it, were reworked so that a delivery either happens or is retried cleanly.

• The database connection now verifies the server's certificate against a certificate authority that ships inside the app. The connection was already encrypted, but the certificate was not checked, so a host positioned on the network path could present its own certificate and read or alter a participant's data in transit. A certificate from any other authority now ends the handshake.
• The require_ssl field in a study configuration was never read by the app; the connection is now configured explicitly rather than relying on a driver default.
• Upload batches are wrapped in a transaction and committed once. A batch that fails part-way is rolled back, so retrying it cannot leave the earlier part of the batch stored twice. This matters because the client's account is insert-only: duplicates it created could never be removed.
• Warnings returned by the database after an insert are now read and reported instead of discarded, and are attributed to the batch that caused them rather than to a later one on the same connection.
• Batch size is bounded by estimated payload size as well as by row count, with an 8 MB ceiling. A row count alone is payload-blind: the same limit covered a sensor row of about a hundred bytes and a screenshot row of over a megabyte, which at a real backlog exceeded both the phone's memory and the server's packet limit.
• A batch that cannot fit its rows now reports how many it actually carried, and paging advances by that count. Advancing by the requested count could step past rows that were never uploaded, which then went on to be deleted locally.
• Upload progress is recorded in a table of its own rather than as an entry in the study log. The log is uploaded and cleared on the same schedule as any other table, which removed the progress records with it; every table whose rows are deliberately kept on the device was then uploaded again from the beginning on the following synchronization.
• A single un-uploadable batch no longer blocks local cleanup for its whole table, so a table can drain instead of only growing.
• FAILED to sync. Server down? no longer names a cause it cannot know. Failures report the batch's row count and size and defer to the connection log.
• Two debug messages claiming the server had been queried for the latest record were removed. Nothing was queried; the messages misdirected anyone reading the log to diagnose a synchronization problem.
• A missing progress record now ends a synchronization pass the same way in release and debug builds. The check previously sat inside a debug-only block.
• Removed a connection test that assigned and then closed the shared upload connection, tearing down an in-progress synchronization when it ran, and which had no timeout. Its only caller was itself unused.

Data Model and Schema

The device, study, and log tables were reviewed column by column against what they actually record. No sensor table changed.

Device information

• aware_device is now one row per device, rewritten in place when the device's facts change and carrying the time of that change. The table has always permitted a single row per device, so the previous attempt to add a row for a changed device was silently discarded — meaning an Android upgrade was detected, dropped, and re-detected on every service start, and never reached the server.
• Because a rewrite carries a new timestamp, each real change is uploaded once and the research database accumulates one row per change. Comparing the Android version across two rows for a device therefore shows a mid-study upgrade.
• Previously the table gained a row on every service start — app open, keep-alive, boot, configuration check — because the uploaded row was deleted locally and the emptiness read as "never recorded". The row is now retained on the device.
• Four columns were removed because they carried no information: the serial number, which Android no longer reports to applications; the build type, which is user on every production device; the device label, which was never populated; and the brand, which restates the manufacturer. board, device, and product were kept.
• Existing rows in the research database are unaffected. The removed columns simply stop receiving values.

Study records

• aware_studies records when a configuration was last replaced, on all three paths that can replace one: a server update, a plugin difference reconciled on the device, and a participant edit in editable mode.
• Joining and re-joining a study now name themselves in the compliance record. A first join previously wrote an empty compliance value, so it was only visible indirectly through the join timestamp.
• The configuration is written on the records where it changed rather than copied onto every compliance record. A study configuration is several kilobytes, and repeating it on each consent decision, plugin install, and cancelled quit made those records an order of magnitude larger than the events they describe.
• Lookups for the active study now select the most recent record that carries a configuration. Compliance records copy the join timestamp of the enrolment they describe, so they matched "joined and not exited" too and, being newer, could be mistaken for the enrolment itself.

Study log

• Each log entry now carries its kind: lifecycle, scheduler, sync, study, diagnostics, context, or event. The message column previously held every kind of record as free text, so counting app restarts meant matching a literal string, and the vocabulary could only be discovered by reading the client source.
• Entries written by plugins keep working unchanged and are recorded as event.

On-device database upgrade

• A schema upgrade that removes a column now completes. The upgrade carried data across using a column list read back from the newly created table, which yields the previous column set while that table is still empty; the resulting statement named a removed column, the upgrade aborted, and the transaction rolled back. The database was then a version behind the application querying it. This had never been reachable before, because no schema change since 2022 had removed a column.

Network Traffic Collection

• Replaced telephony data-activity callbacks with periodic TrafficStats sampling.
• Network Traffic now records Wi‑Fi and mobile received/sent bytes and packets at frequency_network_traffic.
• Wi‑Fi-only devices and sessions no longer depend on a cellular callback before data is written.
• Unsupported mobile counters fall back safely while total traffic remains collectable.
• Counter resets do not create negative traffic values.
• Repeated service starts replace the previous callback instead of creating duplicate collection schedules, and stopping the service removes pending callbacks.
• Network Traffic participates in the same frequency-aware collection diagnostics as other sampled sensors.

Bluetooth Data and Reliability

• Added a Bluetooth data column that records connection status, giving researchers more context than discovery data alone.
• Hardened Bluetooth runtime-permission handling.
• Applied the shared frequency-unit conversion to Bluetooth scan scheduling.
• Preserved hardware and permission explanations in participant-facing sensor status.

Background Collection Notice

• The notice shown while AWARE collects data in the background now always appears, and appears quietly: no sound, no vibration, and no badge on the app icon. It previously shared a notification setting group that Android can leave hidden, and which cannot be corrected once the device has created it.
• The notice now says what is happening — “Collecting study data in the background” — instead of “Data collection active”.
• Tapping the notice opens the app. It previously poked the background service and appeared to do nothing.
• Added the notification permission that Android 13 and later require, so the notice can be shown at all on current devices.

Background Work and Battery

• Fixed a timer that kept a processor core busy for the entire questionnaire waiting period — often hours — for every pending questionnaire. It now waits idly between checks.
• A pending questionnaire could stop other short tasks from ever starting, including joining a study after consent, because both waited in the same single-file queue. Long timers now run on their own, so joining is never held up behind one.
• Only one questionnaire expiry timer is kept. Replacing a questionnaire that is still waiting now cancels the old timer instead of leaving it running alongside the new one.
• Joining a study, leaving a study, and scanning a QR code no longer share that single-file queue either.
• Replaced a similar busy wait in the speech utility with a completion callback, and tied stopping the service to the phrase that started it.
• Fixed a leak where every restart of the background service created another worker thread, even when the restart changed nothing.
• A scheduled action that points at something no longer installed, or carries an unreadable value, is now handled instead of failing the whole scheduler.
• MQTT port, keep-alive, and quality-of-service values are checked and brought into a valid range before use. An empty or malformed value previously stopped the service outright.
• Repeated starts of screenshot capture reuse the session already running instead of building a new one each time, and a session Android has already invalidated is no longer restarted with an expired permission. That combination previously produced a stop-and-restart loop.

Interface Responsiveness and Consistency

• Added consistent dialog sizing and window behavior across the phone interface.
• Removed manual onContentChanged() calls where preference setters already notify their own rows.
• Sensor categories refresh once per synchronization pass even when several status_* settings belong to the same category.
• Unchanged setShouldDisableView() state is not written repeatedly.
• Added and removed study sensors are reconciled in place instead of restarting the interface.
• Applied redundant-refresh fixes to both the current settings interface and the maintained legacy path.
• Ambient Noise and OpenWeather retain preference references on their Activity and use lifecycle-managed listeners.
• Plugin preference rows no longer refresh unnecessarily when values have not changed.

Plugin Stability

• Fixed crash loops in Ambient Noise and OpenWeather.
• Fixed Ambient Noise silence detection.
• Fixed an Ambient Noise sample-size timing race.
• Corrected plugin preference refresh behavior.
• Added frequency dropdowns and defaults for Ambient Noise and OpenWeather.

Security

• Removed the exposed API key from the interface and source.
• Removed database passwords from log output, including the connection logging that previously wrote the configured or participant-entered database password to the device log.
• Added a shared log redactor that masks password, secret, and token values inside logged JSON, applied to the study-configuration, study-data, and studies-record log sites.
• Study-database credentials are excluded from join and validation diagnostics: a failure logs its reason, and where useful the exception type, but never the password.
• The research-database connection is always encrypted, and verifies the server's certificate when the study's configuration publishes its database's certificate authority. Traffic was already encrypted; without verification, encryption protected against an observer but not against a host that answered in the database's place.
• The authority is read from the study configuration rather than bundled in the app, so rotating the database server's certificates needs no new build — the study publishes the new authority and devices pick it up on their next configuration sync. A study that publishes no authority is not verified, and the app records that it is not.
• Removed a request that sent the full device record — identifier, model, build, brand, serial, and Android version — to awareframework.com whenever a device was first recorded. It had no setting in front of it, unlike the separate usage-donation ping, which remains gated and is still worth a deliberate decision against the study's ethics approval.
• The client's database account requires INSERT only; nothing in the app reads or updates the research database.

Build, Dependencies, and Release Automation

• Stopped tracking generated APK and JAR artifacts.
• Replaced vendored dependencies with declared build dependencies where appropriate.
• Removed dangling Anko dependency references.
• Removed dead Gradle configuration, redundant declarations, and an obsolete data-offload path.
• Removed a redundant provider _ID declaration.
• Added a tag-triggered GitHub Actions release workflow.
• Updated release CI SDK setup for 2026 runner images.

Researcher and Data Impact

• Participant-specific editable configurations are retained on the device and included in compliance history.
• Sensor-health records provide context for missing or delayed data without requiring a new database table; they are carried by aware_log, which gains one column this release.
• Study exit is visible as an aware_studies record with an exit timestamp and study_compliance = "quit study". When the database could not be reached at the moment of withdrawal, the record instead reads quit study (server unreachable, not notified), so a withdrawal the researcher was never told about is distinguishable from one that was reported.
• Data collected while a password re-authentication was pending is retained on the device and uploaded once the participant re-enters the password, rather than being dropped.
• Network Traffic produces one Wi‑Fi and one mobile row per configured interval, including zero-value rows when a transport had no traffic, making collection cadence explicit.
• Bluetooth records now include connection status.
• Existing configuration formats remain compatible: hardware-sensor sampling periods are still stored in microseconds even though custom values are presented in Hz.
• aware_device becomes a device dimension rather than a stream: one row per device, plus a row for each genuine hardware or Android change. Queries that counted rows in this table as a proxy for app restarts should read aware_log entries of kind lifecycle instead, which is the correct source and was always uploaded.
• Rows already in aware_device are left as they are. A view selecting the most recent row per device presents the table consistently across the change, and is worth keeping permanently so that analysis depends on a stated contract rather than on row counts.
• aware_log gains log_type, making it possible to select or aggregate a kind of entry without matching message text.
• aware_studies gains double_updated and explicit joined study and rejoined study compliance values. The study configuration now appears on the records where it changed; the configuration in force at a given moment is the most recent one at or before that moment.
• Duplicate rows previously accumulated in aware_studies and aware_device on every synchronization, because the records tracking upload progress were removed by the log's own cleanup. Historical duplicates remain and are byte-identical, so they can be collapsed on device_id and timestamp without losing anything.

Tests and Validation

• Added unit coverage for generalized configuration parsing and missing-sensor handling.
• Added unit coverage for configuration synchronization and saved state.
• Added unit coverage for hardware availability and configuration exclusion.
• Added unit coverage for sensor diagnostics and actionable participant status text.
• Added unit coverage for frequency-unit conversion, Hz conversion, and freshness windows.
• Added unit coverage for permission request sequencing.
• Added unit coverage for consent persistence and enforcement.
• Added unit coverage for editable configuration persistence, preview, approval, and manual synchronization.
• Added unit coverage for Network Traffic counter deltas and unsupported-counter handling.
• Added unit coverage for study-configuration validation: absent, incomplete, and malformed configurations, database fields reported individually by name, and detection of studies that require a participant-supplied password.
• Added unit coverage for study-address validation, including null, empty, blank, scheme-less, non-web, and malformed addresses, and for the address forms that must keep working such as plain HTTP, non-standard ports, and shared-drive links.
• Added unit coverage for distinguishing a rejected database password from an unreachable database.
• Added unit coverage for redaction of password, secret, and token values in log output.
• Added unit coverage for the device-facts comparison that decides whether the stored device row still describes the device, including that every compared field can trigger a rewrite and that a null and an empty value count as equal — a mismatch there would rewrite the row on every service start.
• Added unit coverage for the upload payload budget: that a batch always takes at least one row however large, that a screenshot backlog splits into several batches each inside the cap with every row eventually taken, and that a full batch of sensor rows does not reach the cap.
• Added unit coverage for summarizing database warnings, including a bounded walk so a self-referencing warning chain cannot hang the synchronization thread.
• Added unit coverage for the connection parameters that require certificate verification, asserted literally rather than by shape: a connection string missing the verification flag still connects, to any host that answers.
• Added unit coverage tying the column names the framework writes to the schema it creates, for both the compared device fields and the columns whose absence server-side would fail an upload batch.
• Added unit coverage for the column list a schema upgrade carries data across, including that a trailing table constraint is not mistaken for a column and that the removed device columns are absent.
• Added unit coverage for per-sensor delivery health, including the case that caused the field failure: one table failing, then another succeeding, must leave the first still recorded as failing. Also covers repeated failures keeping their original start time, several tables failing at once, the stored state surviving a save-and-reload, and a malformed stored value being skipped rather than guessed.
• Added unit coverage for the upload connection's bounds: that both the connect and read timeouts are present and non-zero in the connection string, that a communications failure is treated as connection-level while a rejected statement is not — an unknown column must not suppress every other table's upload — and that the cooldown ends exactly at its length and is not held open by a clock that steps backwards.
• Added unit coverage for the sensitivity thresholds: that every threshold setting has a unit and a usable limit, that no recommended preset exceeds its own limit, and that each value from the deployed study configuration is rejected as out of range.
• Core and phone unit-test suites pass: 267 tests.
• Ambient Noise and OpenWeather builds pass.
• Debug APK assembly passes.
• Network Traffic was verified on a physical Android device: after one configured 30-second interval, the previously empty database contained separate Wi‑Fi and mobile rows with non-zero Wi‑Fi byte and packet deltas.
• Certificate verification was verified against the live research database on a physical Android device: the trust store was built in the app's private storage from the bundled authority, and uploads completed over a verified TLS 1.2 connection. The rejection case was confirmed separately with the same database driver — verification against the public authorities refuses the connection, while verification against the study's own authority succeeds.
• The schema upgrade was replayed against a copy of a real pre-upgrade device database: the four device columns were dropped, all study, settings, and device rows were retained, and the database reached the new version.
• A clean installation was verified on a physical Android device: all six tables created in their current shape, and no crash on launch.

Architectural — needs a decision and a plan

Replace direct database access with HTTP(S) to the micro-server. The client currently opens a JDBC
connection to MySQL and inserts rows directly. The intended direction is an HTTP(S) endpoint on the
companion micro-server, authenticated with a per-participant token. This is the single change that
would resolve the most limitations at once:

• the database credential is shared by every participant in a study, so one recovered from a device is
  valid for all of them and cannot be revoked individually
• the database's certificate authority has to be distributed through the study configuration, which is
  itself fetched over plain HTTP where a study is hosted that way — so a fetch that can be altered in
  transit undermines the verification it delivers
• every table's upload serialises through one shared JDBC connection, because a JDBC connection is not
  thread-safe; roughly 30 sync adapters take turns on it
• the client's account is INSERT-only, which is correct for a shared credential but means the client
  cannot maintain any state server-side, and cannot remove rows it duplicated in the past

Revise the screenshot sensor. Its upload is bounded by payload size, but its on-device storage is
not, and it is by a wide margin the highest-volume sensor in the app — measured at roughly 69 MB/hour
while uploads could not keep up. If storage fills, collection stops for every sensor, not just this
one. It is currently left enabled by decision. A cap, a retention policy, or a resolution/interval
reduction all need considering together.

Remove the Processor sensor. It reads /proc/stat, which Android has restricted since version 7,
returning Permission denied. It cannot collect on any currently supported device. It is disabled
rather than removed, and still logs itself as active when started, which is misleading.

Decide the local-cleanup floor. Uploads are floored at the newest study record's timestamp, while
the local cleanup that follows a successful upload is not floored at all. Rows below the floor are
therefore skipped by every batch and then removed — observed on a device as 26 location rows deleted
without having been uploaded. The window is one sync interval and it opens on each configuration
update. The choice is whether the floor should be the participant's original enrolment rather than the
newest configuration change; a configuration change is arguably not a new enrolment.

Serve study configurations over HTTPS. Outside this branch, but it is what makes the certificate
authority delivered in that configuration trustworthy.

Verification still owed

• Per-sensor delivery health is covered by unit tests but has not been exercised on a device where a
  single table fails while the others succeed, which is the case it exists for. Reproducing it needs
  one table's schema broken server-side deliberately.
• The upload connection's timeouts have not been tested against a real hang. Stopping the database
  refuses connections, which already failed fast; the case that needed fixing is a server that accepts
  a connection and then goes quiet, which needs the port dropped rather than closed.
• The sensitivity-threshold presets have been validated on hardware for eight of the ten sensors.
  Barometer and ambient temperature could not be tested — neither available device has that hardware.
• Study-configuration change notifications, consent-screen behaviour under rotation, and applying a
  configuration while the database is down are implemented but not yet walked through on a device.

Platform constraints, not defects

Recording these because each one was initially mistaken for a fault during this work.

• Periodic uploads cannot run more often than every 15 minutes. Android floors periodic sync
  intervals at 15 minutes, so any frequency_webservice below 15 behaves identically to 15. The
  setting should be written as 15 or more so it states what actually happens. Data accumulates for up
  to one interval before delivery — for a high-rate sensor such as the magnetometer that is tens of
  thousands of rows per cycle, which is expected rather than a backlog. Manual synchronisation is
  unaffected, as it is not a periodic request.
• Sensors the device does not have are dropped when a configuration is applied. This is intended,
  and it means a configuration enabling ten hardware sensors can legitimately produce data for fewer.
  An empty table is therefore not by itself evidence of a fault: establish whether the hardware exists
  first. Neither available device had a barometer or ambient-temperature sensor; one had no gyroscope
  or magnetometer either.
• Proximity reporting no data is usually correct. It is an on-change sensor with two quantised
  states, so a stationary device produces nothing.
• The accelerometer delivers fewer samples than its configured rate, by a large factor on both
  available devices. Whether this is Android's background sensor throttling, the service's own
  batching, or the platform not honouring the requested period is not established. It does not affect
  correctness, but analysis should not assume the configured rate.

Carried over from earlier releases

• When a participant enables a sensor that requires a runtime permission or Accessibility service, the app opens the appropriate grant flow. Fully reverting every just-enabled checkbox immediately after every possible denial or settings-return path remains follow-up work; the collection-status row still reports the sensor as blocked and explains the missing requirement.
• Rows in aware_device and aware_studies that were duplicated by the previous upload-progress behaviour remain in the research database. They are byte-identical and can be collapsed; an insert-only client cannot remove them.