URL is Crawlable
# Yahoo! Slurp
User-agent: Slurp
Crawl-delay: 15
# SemrushBot (General scraping)
User-agent: SemrushBot
Crawl-delay: 15
# SemrushBot-SA (SEO / technical crawling)
User-agent: SemrushBot-SA
Crawl-delay: 15
# MJ12bot (https://mj12bot.com/)
User-agent: MJ12bot
Crawl-delay: 15
# AhrefsBot (https://ahrefs.com/robot)
User-agent: AhrefsBot
Crawl-delay: 15
# All
User-agent: *
Disallow: /ab_monitor
robots.txt
Lines in robots.txt
Page tested on 29th December 2024 at 06:03
What Else Could We Find on This Page?
| Page Authority: | 0 | View Report |
|---|---|---|
| Page Speed: | 84/100 Mobile / 98/100 Desktop | View Report |
| Page Language: | View Report | |
| Meta Description: | 50 characters | View Report |