Skip to main content
Loading...

https://www.beautikue.com/could-hawaii-be-the-perfect-destination-for-a-family-getaway/

URL is Crawlable
User-agent: * # applies to all robots
Disallow: /*/feed/$
Disallow: /?nonamp=1/
Disallow: /?amp=1
Disallow: /?noamp=mobile
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php

# Adding Multiple Sitemaps
Sitemap: https://www.beautikue.com/sitemap_index.xml
Sitemap: https://www.beautikue.com/post-sitemap.xml
Sitemap: https://www.beautikue.com/category-sitemap.xml
Sitemap: https://www.beautikue.com/author-sitemap.xml

# Allowed Good User Agents for better Crawl

User-agent: Googlebot
Disallow:
Allow: /*

User-agent: Mediapartners-Google*
Disallow:
Allow: /*

User-agent: Boomtrain-Content-Bot*
Disallow:
Allow: /*

User-agent: Googlebot-Image
Disallow:
Allow: /*

User-agent: Adsbot-Google
Disallow:
Allow: /*

User-agent: Googlebot-News
Disallow:
Allow: /*

User-agent: Bingbot
Disallow:

User-agent: MSNBot
Disallow:

User-agent: Slurp
Disallow:

User-agent: DuckDuckBot
Disallow:

User-agent: Baiduspider
Disallow:

User-agent: YandexBot
Disallow:

User-agent: ia_archiver
Disallow:

User-agent: Teoma
Disallow:

User-agent: Rogerbot
Disallow:

User-agent: Rogerbot/1.2
Disallow:

User-agent: DotBot
Disallow:

User-agent: DotBot/1.1
Disallow:

User-agent: AhrefsBot
Disallow:
Allow: /*

User-agent: MJ12bot
Disallow:
Allow: /*

User-agent: SemrushBot
Disallow:
Allow: /*

User-agent: NinjaBot
Disallow:

User-agent: Facebot
Disallow:

User-agent: Twitterbot
Disallow:

User-agent: LinkedInBot
Disallow:
robots.txt
Lines in robots.txt

Page tested on 10th December 2024 at 00:14