Skip to main content
Loading...

https://www.PetAdsHub.com/488/posts/3/20/2922405.html

URL is Crawlable
# Blocks robots from specific folders / directories

User-agent: googlebot
User-agent: google
User-agent: bingbot
User-agent: bing
User-agent: Mediapartners-Google

Disallow: /cgi-bin/
Disallow: /cron/
Disallow: /log/
Crawl-delay: 15

User-agent: *
Disallow: /
robots.txt
Lines in robots.txt

Page tested on 27th March 2025 at 10:46