Is 172.17.0.1 visible to AI?
Single-page scan of http://172.17.0.1:8080/robots.txt · checked 2026-10-30
90AI visibility
AI crawler access — 8 engines
| Crawler | Org | Role | robots.txt | Live fetch | Verdict |
|---|---|---|---|---|---|
| GPTBot | OpenAI | training | allowed* | HTTP 200 · 1 KB | Open |
| ClaudeBot | Anthropic | training | allowed* | HTTP 200 · 1 KB | Open |
| Google-Extended | training | allowed* | — | Open | |
| CCBot | Common Crawl | training | allowed* | HTTP 200 · 1 KB | Open |
| Bytespider | ByteDance | training | allowed* | HTTP 200 · 1 KB | Open |
| PerplexityBot | Perplexity | assistant | allowed* | HTTP 200 · 1 KB | Open |
| OAI-SearchBot | OpenAI | search | allowed* | HTTP 200 · 1 KB | Open |
| ChatGPT-User | OpenAI | agent | allowed* | HTTP 200 · 1 KB | Open |
What we found
!
No JSON-LD structured data found. Schema markup makes content easier for AI systems to interpret and cite.
-10 pts
!
No robots.txt found. Everything is allowed by default, but you are not steering any crawler explicitly.
0 pts
✓
llms.txt found.
0 pts
Embed your score badge
Paste this into your site, README or docs — it links back to this public report:
<a href="https://api.crawlcheck.dev/check/172.17.0.1" title="AI visibility score for 172.17.0.1"><img src="https://api.crawlcheck.dev/badge/172.17.0.1" alt="AI visibility score 90/100 — 172.17.0.1" width="154" height="24"></a>
FAQ
Can AI crawlers access 172.17.0.1?
As of 2026-10-30, 8 of 8 major AI crawlers can access the homepage. The test covers GPTBot (OpenAI), ClaudeBot (Anthropic), Google-Extended, CCBot, Bytespider (ByteDance), PerplexityBot, OAI-SearchBot and ChatGPT-User.
What does a score of 90/100 mean?
This site is highly accessible to AI crawlers — robots.txt allows most AI agents, pages return real content to bots, and the content is easy for AI systems to extract and quote.
How can 172.17.0.1 improve its AI visibility?
Allow AI crawlers in robots.txt, remove noindex from key pages, publish substantive well-structured content, and add JSON-LD structured data. Re-run the free scan after each change to track the score.