Skip to content

Geocode addresses locally against the regional address points - #167

Merged
jinskeep-morpc merged 2 commits into
mainfrom
local-geocoder-addresspoints
Aug 4, 2026
Merged

Geocode addresses locally against the regional address points#167
jinskeep-morpc merged 2 commits into
mainfrom
local-geocoder-addresspoints

Conversation

@jinskeep-morpc

Copy link
Copy Markdown
Collaborator

Closes #161.

morpc.geocode() wraps Nominatim, an external rate-limited service. The group quarters source strategy calls a local geocoder a prerequisite rather than a refinement: eight of the proposed facility-source workflows emit geometrybasis = geocoded, and none of them should depend on a public API to do it. This adds geocode_addresspoints(), which matches against morpc-addresspoints-standardize instead — offline, reproducible, and better on in-region addresses.

geocode() is unchanged, so the Nominatim path in morpc-mls-standardize keeps working.

Why an index has to be built

The published address point database cannot be queried directly. It carries no index on the fields a match joins on, so every lookup scans 1.25 million rows; its geometry is a WKB blob rather than coordinates; and its components are normalized inconsistently between counties. It also cannot be indexed in place — its Frictionless descriptor records a hash of the file, so adding an index makes resolve_data_path() refuse to load it.

So build_geocode_index() derives a smaller database once: match fields normalized through the same functions the query side uses, geometry decoded to coordinates, indexed over the join. 29 seconds, 114 MB, source file untouched. It rebuilds itself when either the source release or CONST_GEOCODE_INDEX_VERSION moves on — the latter because the index stores names already normalized, so a change to the normalization silently invalidates it in a way the source hash cannot detect.

Matching, and what it refuses to do

Three tiers, from every component down to house number and street name alone. The tier that fired is reported, along with how many points it found and how far apart they were.

Points metres apart are the units of one building or the buildings of one campus, and are returned as their centre. Points kilometres apart are different places, and the address comes back unmatched with a note saying why. The default tolerance of 500 m sits above the widest campus in the validation set (353 m across 133 points) and well below the closest genuine collision (two 3000 BETHEL RD addresses 75 km apart).

That refusal is the point. These outputs feed geometrybasis = geocoded in eight downstream repositories, and a null with a stated reason is worth more there than a point that is quietly wrong.

Measured

Against the 305 facilities in morpc-doddfacilities-standardize and morpc-cmsnursinghomes-standardize that publish both a street address and a point — so the published coordinate scores the match:

crude baseline this PR median error
DODD (surveyed points) 79% 91% 4 m
CMS (self-geocoded) 88% 95% 117 m

The crude baseline is a component join with no address parsing, which is roughly what the earlier Ohio Department of Health attempt did. It put one point 126 km away; the worst case here is 4,070 m.

CMS carries a geocodingfootnote column and geocodes its own addresses, so its 117 m median is largely CMS's error rather than ours. DODD is the honest measure.

Of the 22 remaining misses, most are addresses the counties never published — the street is in the data, that house number is not. Two are source typos (BELLFONTAINE for Bellefontaine, JOHN SHIELD for John Shields) that only a fuzzy tier would catch.

Deliberately not included

  • A fuzzy street-name tier. Needs rapidfuzz, a new dependency, to recover roughly 2 of 305. The miss analysis does not justify it yet.
  • The parcel fallback named alongside address points in the strategy doc. Same reasoning — most misses are absent house numbers, which no second reference recovers.

Two things found along the way

  • morpc/morpc-addresspoints-standardize#10. Knox County is unjoinable as published — ZIP float tail, null city and fulladdress, zero-padded house numbers — and mixed-case streetname costs matches in Franklin too. This PR normalizes past all of it on both sides, which is most of the 79% → 91%. Fixing it upstream removes the workaround.
  • resolve_data_path() cannot fetch a private repository's release asset. The URL 404s anonymously, so the address point database has to be downloaded with gh first. morpc-addresspoints-standardize's README documents load_data() as the way to get it, and that path does not work for anyone today. Not fixed here; worth its own issue.

Tests

32 in tests/test_geocode.py, up from 10. The matcher tests build a small index in a fixture and pass it via indexPath, so they exercise tiers, centroid collapse and ambiguity refusal without the 183 MB download.

The 4 failures in tests/test_utils.py (datetime parsing) predate this branch and are untouched by it.

🤖 Generated with Claude Code

jinskeep-morpc and others added 2 commits August 4, 2026 10:31
The address point data is published as normalized components, but every facility registry
publishes a single free-text line. Matching one against the other needs a parser on the query
side that produces exactly the vocabulary the reference side already uses, which is why this
lives beside normalize_street_type and normalize_directional rather than in a source repository.

Two defects in the published data are reduced here rather than worked around at each call site.
Counties that publish the house number as a numeric column leave a float tail, and Knox zero pads
to five digits, so a literal join against Knox fails for all 26,990 of its records. Numbered
routes are spelled three ways across the region -- "STATE ROUTE 33", "ST RT 33", "SR 33" -- and
are folded to one prefix so that a name joins across a county line.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
morpc.geocode() calls Nominatim, an external rate limited service, which the group quarters source
strategy identifies as a prerequisite to be replaced: eight of the proposed source repositories
emit geometrybasis = geocoded, and none of them should depend on a public API to do it.

geocode_addresspoints() matches against morpc-addresspoints-standardize instead. The published
database cannot be queried directly -- no index on the join fields, geometry as a WKB blob, and
components normalized inconsistently between counties -- and it cannot be indexed in place, because
its descriptor records a hash of the file. So build_geocode_index() derives a smaller database from
it once, normalizing both sides through the same functions, and rebuilds when the release or the
normalization version moves on.

Matching runs in tiers and reports which one fired, how many points it found and how far apart they
were. Points that are metres apart are units of one building and are returned as their centre;
points that are kilometres apart are different places and the address is left unmatched with a note
saying so. A null with a reason is worth more here than a point that is quietly wrong.

Measured against the 305 facilities that publish both an address and a point: DODD 91% matched,
median 4 m from the published coordinate; CMS 95%, median 117 m, where the spread is largely CMS's
own geocoding error, which it flags in geocodingfootnote. A crude match without address parsing
reached 79% and 88% on the same facilities and put one point 126 km away.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@jinskeep-morpc
jinskeep-morpc merged commit 01f47bc into main Aug 4, 2026
1 check passed
@jinskeep-morpc
jinskeep-morpc deleted the local-geocoder-addresspoints branch August 4, 2026 14:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add a local geocoder that matches addresses against MORPC address points

1 participant