Skip to content

feat(data): make the site correct for 2027+ seasons - #2

Open
Persblack wants to merge 2 commits into
mainfrom
frontend-engineer/multi-year-hardening
Open

feat(data): make the site correct for 2027+ seasons#2
Persblack wants to merge 2 commits into
mainfrom
frontend-engineer/multi-year-hardening

Conversation

@Persblack

Copy link
Copy Markdown
Owner

Why

The site assumed a single festival year everywhere. A 2027 dataset would not have failed — it would have quietly lied: the calendar grouped by month index alone, so July 2026 and July 2027 merged into one section labelled July 2026; the analytics histogram summed both years into the same bars; the month slider matched every year with no way to scope one; five copy strings hardcoded 2026. Independently, every festival date went through new Date("2026-07-03") — UTC midnight read back with local getters, which shifts the day for anyone west of Greenwich and mis-buckets month-boundary festivals. tsc --noEmit was already red on three errors, because no script ran it.

Decision (agreed up front): rolling window — the default view is everything from today forward, whatever year it lands in, with year chips to narrow. Adding a season is a file drop; src/data/festivals*.json is globbed, merged and de-duplicated by id, so 2028 needs no code change.

What changed

Data layer (new)

  • lib/festival-normalize.ts — pure validator. Drops only rows that cannot be identified or placed in time (missing id/name, unparseable start date, duplicate id); repairs everything else and reports each repair at build time. Fails the build above a 5% drop ratio, so a gutted scrape cannot ship as a stub site.
  • lib/festival-data.ts — build-time-only loading seam (never imported from a React island, which would bundle the 1.1 MB dataset). year is derived from start_date, never trusted from the file.
  • lib/dates.ts — local-midnight parsing, year-qualified YYYY-MM bucket keys, cross-year ranges (Dec 30, 2027 - Jan 2, 2028), rejects non-dates instead of producing Invalid Date.
  • lib/guards.ts — canonical isRecord / hasCoordinates.
  • Festival.latitude/longitude/website are nullable, because the data already was (28 rows without coordinates, 47 without a usable URL). Map, Google Maps link, share sheet and recommendations now guard instead of rendering null,null.

Views

  • lib/filters.ts — one filter/sort/group implementation replacing four divergent copies of the same predicate chain; prebuilt lowercase search index instead of lowercasing ~8k lineup strings on every keystroke.
  • Season chips in FilterBar; calendar sections, jump navigation and planner timeline markers carry the real year; analytics draws one <Area> per season.
  • All year-specific copy derives from the data; footer states dataset size, span and freshness.
  • Country flags derive from the ISO code (regional indicator symbols) instead of a 9-entry table, so a new country renders without a code change.
  • Leaflet popup HTML escapes scraped names/cities.

Gates

  • npm run typecheck, npm run test, npm run check; 63 unit tests over dates, filters and the validator. Tests do not read the gitignored dataset.

Dependencies

Dev-only: vitest, @types/node. No env vars, no migrations.

Review checklist

  • Rolling-window default is the intended UX (past festivals hidden until "Show past festivals")
  • 5% drop ratio is the right build-failure threshold for scraper regressions
  • Analytics deliberately covers the whole dataset (past included) while the homepage stats cover only upcoming — the subtitle says "all N festivals"
  • Pre-existing any at the Leaflet/recharts boundaries left untouched (out of scope)

Evidence

$ npm run check
tsc --noEmit                       exit 0   (was exit 2 on 3 errors before this branch)
vitest run                         3 files, 63 tests passed
astro build                        7 pages, exit 0
[festival-data] kept 916/916 rows from ../data/festivals.json (2026: 916)
  repaired 47x no usable website URL
  repaired 28x missing or implausible coordinates (excluded from map)

Verified against a synthetic second season (150 shifted rows + 7 deliberate defects) dropped into src/data/festivals_2027.json:

[festival-data] kept 1071/1073 rows from ../data/festivals.json, ../data/festivals_2027.json (2026: 916, 2027: 155)
  dropped 2:
    - defect-no-date-2027: unparseable start_date ("summer 2027")
    - row #1072: missing id
  repaired 155x year field disagreed with start_date (derived from start_date)
  repaired 1x unrecognized genre "Shoegaze" (add it to GENRE_CATEGORY_MAP)
  repaired 1x end_date before start_date (clamped to start)
  repaired 1x unknown size (fell back to medium)

Browser-verified (headless Chromium, 1440x1000) with that two-season dataset:

  • Calendar: separate June 2026 … December 2026 / March 2027 … December 2027 sections, jump nav Jun '26 … Dec '27, header badge 2026-2027 Festivals, footer 1071 festivals · 2026-2027 · Data updated 14 Feb 2027
  • Season chips All years / 2026 / 2027; clicking 2027 narrows 266 → 155 and every section header is 2027; active chip rgb(59,130,246) vs muted rgb(37,37,37)
  • Analytics: 2 <Area> series with a 2026/2027 legend
  • Map: 11 festivals without coordinates aren't shown, 236 markers
  • Timeline: 570-day span with Jul '26 … Dec '27, Jan '28 markers — no duplicate bare Jan
  • Cross-year festival renders Dec 30, 2027 - Jan 2, 2028
  • No Invalid Date / NaN on any of the 7 routes

With the synthetic file removed, single-year output is unchanged: no season chips, bare month names in the jump nav, one chart series, 2026 Festivals badge.

The only script was `astro build`, which does not typecheck — `tsc --noEmit`
was already failing on three pre-existing errors nobody could see. Adds
typecheck, vitest and a combined `check` script so date and filter logic has a
gate that fails instead of a page that looks plausible.
The site assumed one year everywhere, so a second season would not have
failed — it would have quietly lied. Calendar grouped by month index alone
(July 2026 and July 2027 merging into one section labelled "July 2026"), the
analytics histogram summed years into the same bars, the month slider matched
every year with no way to scope one, and five copy strings hardcoded 2026.
Festival dates were parsed with `new Date("2026-07-03")` — UTC midnight read
back through local getters, which shifts the day for every visitor west of
Greenwich and mis-buckets month-boundary festivals.

Rolling window is now the model: the default view is everything from today
forward whatever year it lands in, with year chips to narrow. Adding a season
is a file drop — `src/data/festivals*.json` is globbed, merged and
de-duplicated by id, so 2028 needs no code change.

Data layer
- `lib/festival-normalize.ts`: pure validator. Drops only rows that cannot be
  identified or placed in time (no id/name, unparseable start date, duplicate
  id) and repairs the rest, reporting every repair at build time. Fails the
  build past a 5% drop ratio so a gutted scrape cannot ship as a stub site.
  Today's data reports 47 unusable website URLs and 28 missing coordinates.
- `lib/festival-data.ts`: build-time-only loading seam (never imported from an
  island, which would bundle the dataset). Year is derived from `start_date`,
  never trusted from the file.
- `lib/dates.ts`: local-midnight parsing, year-qualified `YYYY-MM` bucket keys,
  cross-year date ranges ("Dec 30, 2027 - Jan 2, 2028").
- `Festival.latitude/longitude/website` are now nullable, because the data
  already was; map, maps link, share sheet and recommendations guard instead of
  rendering `null,null`.

Views
- `lib/filters.ts`: one filter/sort/group implementation, replacing four
  divergent copies of the same predicate chain, plus a prebuilt lowercase
  search index instead of lowercasing ~8k lineup strings per keystroke.
- Season chips in FilterBar; calendar sections and jump navigation carry the
  real year; analytics draws one series per season.
- All year-specific copy derives from the data, and the footer states dataset
  size, span and freshness.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant