feat: implement robots.txt and dynamic sitemap.xml

[Rayan9064]

Summary by CodeRabbit

• New Features
  • Added search engine guidance to control which site areas are crawled.
  • Added an automatically generated sitemap covering key pages and public user profiles.
  • Sitemap entries include page priorities, update frequency, and last-modified dates.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
dumpit	Ready	Preview, Comment	Jul 20, 2026 6:01pm

[coderabbitai]

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: e1f10ce6-ca3f-4373-bb7f-bb32f237ab88

📥 Commits

Reviewing files that changed from the base of the PR and between 2820507 and 2101117.

📒 Files selected for processing (2)

• app/robots.ts
• app/sitemap.ts

---

📝 Walkthrough

Walkthrough

Adds Next.js robots and sitemap metadata routes. Robots rules allow the site root while excluding dashboard, API, and login paths. The sitemap combines static routes with Firestore user profiles and uses an environment-configured app URL.

Changes

SEO metadata routes

Layer / File(s)	Summary
Crawler directives app/robots.ts	Defines wildcard crawler rules, allows the site root, disallows selected paths, and configures the sitemap URL with an environment fallback.
Sitemap generation app/sitemap.ts	Combines static routes with Firestore user profile URLs, converts updated_at values, and returns static routes when Firestore access fails.

Estimated code review effort: 2 (Simple) | ~10 minutes

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/seo-sitemap

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 ESLint

If the error stems from missing dependencies, add them to the package.json file. For unrecoverable errors (e.g., due to private dependencies), disable the tool in the CodeRabbit configuration.

ESLint install timed out. The project may have too many dependencies for the sandbox.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}