# www.robotstxt.org/ # www.google.com/support/webmasters/bin/answer.py?hl=en&answer=156449 # https://developers.google.com/search/reference/robots_txt?csw=1#url-matching-based-on-path-values # https://support.google.com/webmasters/answer/6080548?hl=en # https://www.searchenginejournal.com/technical-seo/url-parameter-handling/?amp # Used for many other (non-commercial) purposes as well User-agent: CCBot Disallow: / # For new training only User-agent: GPTBot Disallow: / # Not for training, only for user requests User-agent: ChatGPT-User Disallow: / # Marker for disabling Bard and Vertex AI User-agent: Google-Extended Disallow: / # Speech synthesis only? User-agent: FacebookBot Disallow: / # Multi-purpose, commercial uses; including LLMs User-agent: Omgilibot Disallow: / User-agent: Omgili Disallow: / User-agent: * Disallow: /show/sendto/* Disallow: /adformats/* Disallow: /biznes/profile/* Sitemap: https://www.dnevnik.bg/dnevnik_all0.xml