User-agent: * Disallow: /auth/ # Real functional/app routes (routes/web.php, Category 4 comment ~line 199) - # these are intentionally not redirected, blocked from crawling here instead. Disallow: /api/stc/upload-form16 Disallow: /kyt/complete Disallow: /setup/verify Disallow: /public/setup/verify Disallow: /user/signup Disallow: /register-after-otp Disallow: /account/ Disallow: /add-listing/ Disallow: /tools/ # Confirmed 2026-09-05: /public/resources/app/data/service_details_full.json # 404s -- never a real URL, removed. The file is actually served as a static # asset directly from Laravel's public/ webroot at this path (no /public/ # prefix), which is the real URL Google crawled: Disallow: /resources/app/data/service_details_full.json # NOTE: /login, /register, and /sign-in intentionally NOT blocked here. # /login and /register 301-redirect to /?auth=login and /?auth=register. # /sign-in (added 2026-09-05) 301-redirects to /login for the same reason. # Blocking these via robots.txt would prevent Googlebot from ever seeing # those redirects, undermining the fix. Let the 301s do the consolidation work.