URL is Crawlable
User-agent: *
Disallow: /assets
Disallow: /cache
Disallow: /sources
Disallow: /api
Disallow: /script_backups
Disallow: /updates
Disallow: /install
Disallow: /admincp
Disallow: /admin-panel
Disallow: /ajax_loading.php
Disallow: /api.php
Disallow: /xml
Disallow: /system_status.php
Disallow: /nodejs
Disallow: /?cache=
Sitemap: https://newyorktimesnow.com/sitemap.xml
Sitemap: https://newyorktimesnow.com/sitemap-1.xml
Sitemap: https://newyorktimesnow.com/sitemap-2.xml
robots.txt
Lines in robots.txt
Page tested on 6th May 2025 at 05:27
What Else Could We Find on This Page?
| Page Authority: | 0 | View Report |
|---|---|---|
| Page Speed: | 7/100 Mobile / 84/100 Desktop | View Report |
| Page Language: | en | View Report |
| Meta Description: | 143 characters | View Report |