# betaider — calibrated multi-sport predictions User-agent: * Allow: / # Not pages: the admin panel, JSON endpoints, and per-action query forms that # would multiply one page into thousands of near-duplicates in an index. Disallow: /admin/ Disallow: /*.json.php$ Disallow: /*?*submitted= Disallow: /ticket_fix.json.php # The outbound bookmaker redirect. Every link into it already carries # rel="nofollow sponsored", and it answers a 302 rather than a page, so there is # nothing here for an index — but a crawler following it would spend requests # and inflate the click counts the affiliate page is read from. Disallow: /go Disallow: /go.php # /slip, /search and /predict are deliberately NOT listed here: they carry a # noindex meta tag, and a crawler can only read that if it is allowed to fetch # the page. Disallowing a URL does not keep it out of the index — it can still # be indexed from links, just with no title — so the meta tag is the tool. # The parameters below choose a view of a page rather than a different page. # Left crawlable they turn one radar into thousands of near-identical URLs. Disallow: /*?*leagues%5B%5D= Disallow: /*?*families%5B%5D= Disallow: /*?*page= # Be polite to aggressive crawlers without blocking the majors. Crawl-delay: 2 Sitemap: https://betaider.com/sitemap.xml