Crawlability Checker

Check if your URLs can be indexed by search engine bots

Enter URLs to Check

Add one or multiple URLs (separated by commas or new lines)

URLs

Protocol (https://) is optional - will be added automatically

User Agent (optional)

Specify a user agent to check specific bot permissions (e.g., Googlebot, Bingbot)

Status Legend

check_circle

CRAWLABLE

Legally allowed and practically accessible

warning

LIMITED

Allowed but CDN may rate-limit

cancel

NOT CRAWLABLE

Blocked by robots.txt, headers, or meta tags

error

ERROR

Failed to check - unable to determine status