Skip to content

Implement dynamic SEO pre-rendering engine - #257

Merged
kargig merged 7 commits into
mainfrom
feature/seo-dynamic-rendering
Aug 16, 2026
Merged

Implement dynamic SEO pre-rendering engine#257
kargig merged 7 commits into
mainfrom
feature/seo-dynamic-rendering

Conversation

@kargig

@kargig kargig commented Aug 14, 2026

Copy link
Copy Markdown
Owner

Replace the static-generation pre-rendering strategy with a live, database-backed Dynamic Rendering Engine.

Intercept public GET requests for catalogs, details, public logs, and user directories at the Nginx gateway, proxying them to a fast FastAPI endpoint. Pre-render page elements (metadata, breadcrumbs, JSON-LD schemas, and body content) into the DOM using the React SPA index.html skeleton, resolved dynamically and cached in memory.

Specifically, pre-render catalogs, directories, and detail views for:

  • Dive Sites (/dive-sites)
  • Diving Centers (/diving-centers)
  • Dive Routes (/dive-routes)
  • Public logbooks (/dives, filtered for non-private entries)
  • User profiles, analytics, and custom lists (/users)
  • Dynamic calculators, certifications, and tags (/resources)
  • Static informational pages (/about, /help, /privacy)

Configure a 10-minute Nginx proxy cache zone to handle search bot spikes without overloading the database. Cleanly handle HTTP 404 responses for deleted or missing entities

Add a comprehensive integration test suite to verify route resolution, metadata correctness, caching, and custom 404 responses.

@kargig
Implement dynamic SEO pre-rendering engine
1a15fd7 
Replace the static-generation pre-rendering strategy with a live,
database-backed Dynamic Rendering Engine.

Intercept public GET requests for catalogs, details, public logs,
and user directories at the Nginx gateway, proxying them to a fast
FastAPI endpoint. Pre-render page elements (metadata, breadcrumbs,
JSON-LD schemas, and body content) into the DOM using the React SPA
index.html skeleton, resolved dynamically and cached in memory.

Specifically, pre-render catalogs, directories, and detail views for:
- Dive Sites (`/dive-sites`)
- Diving Centers (`/diving-centers`)
- Dive Routes (`/dive-routes`)
- Public logbooks (`/dives`, filtered for non-private entries)
- User profiles, analytics, and custom lists (`/users`)
- Dynamic calculators, certifications, and tags (`/resources`)
- Static informational pages (`/about`, `/help`, `/privacy`)

Configure a 10-minute Nginx proxy cache zone to handle search bot
spikes without overloading the database. Cleanly handle HTTP 404
responses for deleted or missing entities, and revert the obsolete
static pre-rendering generation and R2 Worker routing pipeline.

Add a comprehensive integration test suite to verify route resolution,
metadata correctness, caching, and custom 404 responses.
@github-actions

github-actions Bot commented Aug 14, 2026 •
edited
Loading

Copy link
Copy Markdown

PR Reviewer Guide 🔍

(Review updated until commit 784444c)

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 4 🔵🔵🔵🔵⚪
🧪 PR contains tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Possible Issue

In the get_prerendered_page endpoint, the parts list is parsed from the URL path. For the resources route, the code checks parts[1] without first verifying that len(parts) >= 2. If a request is made to /resources (which is handled by the len(parts) == 1 branch), the code correctly handles it. However, if a request is made to /resources/ (with a trailing slash), clean_path becomes an empty string, parts becomes an empty list, and the code falls into the if not parts branch, treating it as the homepage instead of returning a 404 or redirecting. This is a minor edge case but could lead to incorrect SEO behavior for a URL that should not be treated as the homepage.

 
elif parts[0] == "resources":
    if len(parts) == 1:
        page_title = "Divemap - Resources"
        description = "Access helpful diving resources, organizations, and planning tools."
        main_content = """<main class="seo-prerender">
            <h1>Diving Resources</h1>
            <p>Explore our compiled databases of global diving organizations, system tags, and digital planning calculators.</p>
            <ul>
                <li><a href="/resources/tags">Diving Tags</a></li>
                <li><a href="/resources/diving-organizations">Diving Organizations</a></li>
                <li><a href="/resources/tools/mod">Max Depth (MOD) Planning Calculator</a></li>
                <li><a href="/resources/tools/bestmix">Nitrox Best Mix Planning Calculator</a></li>
            </ul>
        </main>"""
        canonical = f"{base_url}/resources"

    elif parts[1] == "tags":
        page_title = "Divemap - Diving Tags"
        description = "Browse official community tags used to categorize dive sites and marine life."
        main_content = """<main class="seo-prerender">
            <h1>Diving Tags</h1>
            <p>System-wide taxonomy for categorizing dive sites based on attributes, access difficulty, and marine life characteristics.</p>
            <nav>
                <a href="/resources">Back to Resources</a>
            </nav>
        </main>"""
        canonical = f"{base_url}/resources/tags"

    elif parts[1] == "diving-organizations":
        orgs = db.query(DivingOrganization).all()
        org_links = []
        for o in orgs:
            org_links.append((o.name, f"/resources/diving-organizations#{slugify(o.name)}"))

        page_title = "Divemap - Diving Organizations"
        description = "Directory of international scuba diving training organizations and certification bodies."
        main_content = render_listing_main(
            "Diving Organizations",
            "Directory of international scuba diving training and certification agencies.",
            org_links,
        )
        canonical = f"{base_url}/resources/diving-organizations"

    elif parts[1] == "tools" and len(parts) >= 3:
        tool_id = parts[2]
        page_title = f"Divemap - Scuba Calculator: {tool_id.upper()}"
        description = f"Interactive dive planning calculator for {tool_id.upper()} calculations."
        main_content = f"""<main class="seo-prerender">
            <h1>Scuba Calculator: {tool_id.upper()}</h1>
            <p>Digital planning tool for estimating diving limits, depths, and gas mixtures.</p>
            <nav>
                <a href="/resources">Back to Resources</a>
            </nav>
        </main>"""
        canonical = f"{base_url}/resources/tools/{tool_id}"
    else:
        raise HTTPException(status_code=404, detail="Page not found")
Performance Concern

The get_prerendered_page endpoint performs multiple database queries for listing pages (e.g., /dive-sites, /diving-centers, /dive-routes, /dives) without any pagination or limit beyond the hardcoded 100. While this is acceptable for the initial implementation, it could become a performance bottleneck as the database grows. The queries also lack joinedload for related objects in some cases (e.g., DivingCenter listing does not eager-load ratings or other relations), which could lead to N+1 query problems when rendering the listing HTML. Consider adding eager loading for all relations used in the rendering functions and implementing pagination or a more efficient query strategy.

 
sites = (
    db.query(DiveSite)
    .filter(DiveSite.status == "approved", DiveSite.deleted_at.is_(None))
    .limit(100)
    .all()
)
Possible Issue

In the get_prerendered_page endpoint, the parts list is parsed from the URL path. For the resources route, the code checks parts[1] without first verifying that len(parts) >= 2. If a request is made to /resources (which is handled by the len(parts) == 1 branch), the code correctly handles it. However, if a request is made to /resources/ (with a trailing slash), clean_path becomes an empty string, parts becomes an empty list, and the code falls into the if not parts branch, treating it as the homepage instead of returning a 404 or redirecting. This is a minor edge case but could lead to incorrect SEO behavior for a URL that should not be treated as the homepage.

 
elif parts[0] == "resources":
    if len(parts) == 1:
        page_title = "Divemap - Resources"
        description = "Access helpful diving resources, organizations, and planning tools."
        main_content = """<main class="seo-prerender">
            <h1>Diving Resources</h1>
            <p>Explore our compiled databases of global diving organizations, system tags, and digital planning calculators.</p>
            <ul>
                <li><a href="/resources/tags">Diving Tags</a></li>
                <li><a href="/resources/diving-organizations">Diving Organizations</a></li>
                <li><a href="/resources/tools/mod">Max Depth (MOD) Planning Calculator</a></li>
                <li><a href="/resources/tools/bestmix">Nitrox Best Mix Planning Calculator</a></li>
            </ul>
        </main>"""
        canonical = f"{base_url}/resources"

    elif parts[1] == "tags":
        page_title = "Divemap - Diving Tags"
        description = "Browse official community tags used to categorize dive sites and marine life."
        main_content = """<main class="seo-prerender">
            <h1>Diving Tags</h1>
            <p>System-wide taxonomy for categorizing dive sites based on attributes, access difficulty, and marine life characteristics.</p>
            <nav>
                <a href="/resources">Back to Resources</a>
            </nav>
        </main>"""
        canonical = f"{base_url}/resources/tags"

    elif parts[1] == "diving-organizations":
        orgs = db.query(DivingOrganization).all()
        org_links = []
        for o in orgs:
            org_links.append((o.name, f"/resources/diving-organizations#{slugify(o.name)}"))

        page_title = "Divemap - Diving Organizations"
        description = "Directory of international scuba diving training organizations and certification bodies."
        main_content = render_listing_main(
            "Diving Organizations",
            "Directory of international scuba diving training and certification agencies.",
            org_links,
        )
        canonical = f"{base_url}/resources/diving-organizations"

    elif parts[1] == "tools" and len(parts) >= 3:
        tool_id = parts[2]
        page_title = f"Divemap - Scuba Calculator: {tool_id.upper()}"
        description = f"Interactive dive planning calculator for {tool_id.upper()} calculations."
        main_content = f"""<main class="seo-prerender">
            <h1>Scuba Calculator: {tool_id.upper()}</h1>
            <p>Digital planning tool for estimating diving limits, depths, and gas mixtures.</p>
            <nav>
                <a href="/resources">Back to Resources</a>
            </nav>
        </main>"""
        canonical = f"{base_url}/resources/tools/{tool_id}"
    else:
        raise HTTPException(status_code=404, detail="Page not found")

⚠️ Review coverage: The following files were not included in this review because of the token budget:

  • docs/superpowers/specs/2026-08-14-seo-dynamic-rendering-design.md
  • docs/superpowers/plans/2026-08-14-seo-dynamic-rendering-plan.md
  • nginx/dev.conf

@kargig
Secure and optimize SEO pre-rendering engine
56230af 
Mitigate security vulnerabilities and remove dead code from the
SEO pre-rendering layer.

Apply strict Host header validation in the dynamic HTML pre-render
router to prevent Host Header Injection attacks. Untrusted production
requests are forced to the canonical production domain.

Wrap user-controlled strings (usernames, dive titles, and logged notes)
with HTML escaping to eliminate Stored XSS risks on profiles and logs.

Prune obsolete static HTML compiler helper functions to keep the
pre-rendering codebase lightweight and free of orphaned dependencies.
@kargig

kargig commented Aug 14, 2026

Copy link
Copy Markdown
Owner Author

/review

@github-actions

github-actions Bot commented Aug 14, 2026

Copy link
Copy Markdown

Persistent review updated to latest commit 56230af

@kargig
Add input validation and SEO 301 redirects to router
295d854 
Enhance security, optimize performance, and align canonical URL routing
inside the SEO pre-rendering router.

Validate usernames against an alphanumeric/dash regex format to prevent
URL-encoded parsing exploits. Ensure list IDs are parsed as integers,
returning a clean HTTP 404 for invalid, non-numeric collections.

Implement HTTP 301 redirects on details pages when requested with a
missing or mismatched slug, safely steering search engines to the true
canonical URL and protecting against duplicate content indexing.

Prune redundant SQLAlchemy joinedload options on catalog listing queries
to reduce database join and memory overhead under bot crawling traffic.

Add detailed test cases in the integration suite to verify redirect
headers, username pattern restrictions, and non-numeric list limits.
@kargig

kargig commented Aug 14, 2026

Copy link
Copy Markdown
Owner Author

/review

@github-actions

github-actions Bot commented Aug 14, 2026

Copy link
Copy Markdown

Persistent review updated to latest commit 295d854

@kargig
Mitigate thundering herd and secure user queries
2df051c 
Add asyncio lock and filter out inactive accounts in SEO pre-render.

Prevent a thundering herd on first-start template fetches by
implementing an asyncio double-checked lock inside the loader.

Filter out public dive logs and profile details associated with
disabled, banned, or deleted user accounts, safeguarding privacy
and preventing inactive content from appearing on search engines.

Defensively escape numerical depth and duration metrics inside the
pre-rendered dive detail template to protect against HTML injection.
@kargig

kargig commented Aug 15, 2026

Copy link
Copy Markdown
Owner Author

/review

@github-actions

github-actions Bot commented Aug 15, 2026

Copy link
Copy Markdown

Persistent review updated to latest commit 2df051c

@kargig
Add asyncio lazy locks and secure user pre-rendering URLs
784444c 
Avoid 'RuntimeError: Event loop is closed' warnings in tests by
instantiating the template caching asyncio.Lock lazily inside the
load function rather than at module import time.

Isolate and secure URL parameters on user routes. Use raw validated
usernames for canonical URLs and navigation hyperlinks to prevent
HTML entity leakage, while keeping escaped variants strictly for
display inside HTML tags.

Add Nginx locations in both dev and prod configuration gateways to
block external access to '/api/v1/seo/html', returning 403. This
seals the pre-rendering route from external abuse and mitigates
database-flooding risks.
@kargig

kargig commented Aug 15, 2026

Copy link
Copy Markdown
Owner Author

/review

@github-actions

github-actions Bot commented Aug 15, 2026

Copy link
Copy Markdown

Persistent review updated to latest commit 784444c

kargig added 2 commits August 15, 2026 10:07
@kargig
Guard resources routing and confirm listings load
d863910 
Add explicit list length checks on resources sub-routes.

Enforce strict index bounds checks on resource sub-path slices to
prevent IndexError exceptions when parsing '/resources/tags' or
any dynamic children under rare pathing edge cases.

Confirm that catalog and database queries for public listings do
not trigger any N+1 query patterns during active crawls. All
relationship access is either restricted to direct columns or
successfully pre-loaded via eager SQLAlchemy options.
@kargig
Style pre-rendered SEO pages to match React app
1cafcbf 
Integrate styled navbar headers and brand colors into HTML.

Style unstyled semantic HTML elements inside pre-rendered pages to
bypass Tailwind's Preflight CSS reset defaults. Inject custom CSS
rules to provide bold typography and card listings.

Pre-render a fixed, styled top navigation header matching the real
React app's '#0072b2' ocean-blue theme, complete with logo, mock
search box, nav links, and authorization button skeletons.

Render a high-contrast hero section on the homepage and structured
card layouts on detail pages inside '<div id="root">'. This
guarantees a seamless, pixel-perfect visual match and
zero-layout-shift hydration when React boots.
@kargig
kargig merged commit 3745e1b into main Aug 16, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

Review effort 4/5

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@kargig