User-agent: * Allow: / # Do not crawl admin/authentication pages Disallow: /admin Disallow: /login Disallow: /otp Disallow: /dashboard # Do not crawl API or internal resources Disallow: /api/ Sitemap: https://www.mwsgkp.in/sitemap.xml