issue #15 [RFC] Include Istiaatha & Bismillah in Phonetic Search

[amiraelgarf]

Add Istiaatha & Bismillah support to PhoneticSearch

Summary

PhoneticSearch previously indexed only the core Quranic text (6,236 ayat), so queries containing الاستعاذة ("أعوذ بالله من الشيطان الرجيم") or البسملة ("بسم الله الرحمن الرحيم") ( both extremely common in real recitation audio ) could never match, since those phonemes simply didn't exist in the reference string.

This PR adds both as first-class, filterable segments in the phoneme index, without breaking any existing query.

What changed

All changes are contained to src/quran_transcript/phonetics/search.py.

• ph_index.npy gains an 8th column, segment_type (0=quran, 1=istiaatha, 2=bismillah). Istiaatha is now indexed before every sura's first aya; Bismillah is indexed before every sura's first aya except sura 9 (not recited) and sura 1 (it's textually aya 1 itself, already indexed as segment_type=quran).
• PhoneticSearch.__init__(data_dir=None, start=(1,1), end=(114,6)): new optional start/end bounds to scope search to a (sura, aya) range. Rejects old 7-column indexes with a clear ValueError instead of silently misbehaving.
• search(..., include_istiaatha=False, include_bismillah=False): both default off (unchanged behavior for existing callers); matches touching an excluded segment are filtered out.
• get_uthmani_from_result(): now reconstructs Uthmani text correctly for matches inside Istiaatha, inside Bismillah, or spanning Istiaatha → Bismillah → Quran text in a single result. Raises NotImplementedError (documented, not silent) for the one unhandled edge case: a match starting in one sura's Quran text and ending in the next sura's Istiaatha/Bismillah.

Design notes

• Bismillah's sura-1/sura-9 exclusion isn't hardcoded: it reuses AyaFormat.bismillah_uthmani, which is already None for those two suras in utils.py, avoiding a second source of truth.
• Istiaatha is indexed once per sura (not once globally) so a search bounded to any single sura can still match Istiaatha said before it.
• phonemes_idx was always aya-local (reset per aya), not a global string offset: so prepending Istiaatha/Bismillah before each sura does not shift any existing aya's indices. Verified empirically (see Testing).

Migration

Requires a one-time index rebuild:

from quran_transcript.phonetics.search import create_phonemes_index
create_phonemes_index()

Old (7-column) ph_index.npy files are rejected at load time with a clear message rather than producing incorrect results.

Testing

Verified against a real rebuild of the full Quran index (not just unit-level):

• Existing test_phonemes_search_pytest.py: 2/2 pass, unmodified, against the rebuilt 8-column index.
• New test_istiaatha_bismillah_search.py (17 tests): index shape/row counts, inclusion/exclusion for both segment types, no false-positive Bismillah match in sura 9, correct handling of sura 1 and sura 27's literal Quranic Bismillah occurrences, boundary filtering, full Istiaatha+Bismillah+aya round-trip reconstruction (sura 112), the NotImplementedError edge case, and old-index rejection.
• Manual sanity check: rebuilt index has 311,886 Quran rows (exact match to the pre-change total: confirms zero drift in existing content), 2,166 Istiaatha rows (114 × 19), 1,680 Bismillah rows (112 × 15).

limitation

get_uthmani_from_result() doesn't yet resolve a match that starts in one sura's Quran text and ends in the following sura's Istiaatha/Bismillah, it raises NotImplementedError rather than returning wrong text. Flagging as a follow-up rather than blocking this PR on it, since it's a narrow edge case at sura boundaries specifically.