Name: AI-Crawler Access Audit 2026: robots.txt and llms.txt records for 500 websites
Creator: Asfand Yar Junejo
Published: 2026-11-01

Return to the Lab · Independent crawler-policy research

Research by Asfand Yar Junejo | Independent AI search and technical SEO investigation
Published: 1 November 2026 | Edition: November 2026 | Sample: 500 websites | Baseline determined robots.txt verdicts: 448
Evidence: Study methodology and evidence provenance

Are websites blocking AI crawlers?

Some are, but the answer depends on which crawler, which kind of restriction, and which denominator you use. Among 448 websites for which this audit could determine a robots.txt policy, 152 (33.9%) explicitly named at least one of 22 tracked AI-related tokens. The other 296 (66.1%) named none. This is not proof that those sites lack an AI policy. A website can restrict requests through its wildcard User-agent: * group, its firewall, terms of use, licensing agreements, or controls not evaluated here.

A smaller number restrict the engines used for AI-assisted search. 58 of 448 determined sites (12.9%) fully disallowed at least one of three named search tokens, OAI-SearchBot, Claude-SearchBot, or PerplexityBot. News and media sites stood out: 41 of 59 (69.5%) in that sampling bucket fully restricted at least one of those three search bots.

The main pattern is selective rather than universal exclusion. GPTBot was fully blocked by 76/448 (17.0%) while the same provider’s search token, OAI-SearchBot, was blocked by 34/448 (7.6%). ClaudeBot was blocked by 79/448 (17.6%) versus 33/448 (7.4%) for Claude-SearchBot. These are descriptive comparisons, not proof of the operators’ motives or the effects of their policies.

Evidence boundary: The frozen baseline is a cross-sectional observational audit of published robots.txt instructions and the presence of /llms.txt. It did not measure crawler compliance, model training, WAF behavior, AI referral traffic, AI citations, AI Overview rankings, or changes over 12 months. Its sampling frame is purposive, not representative of the entire web.

ASFAND YAR JUNEJO / RESEARCH LAB · FIGURE 01

Study coverage and analysis populations

500 domains attempted

13 purposive buckets: 11 industry groups and two regional oversamples

Determined policy

448/500 · 89.6%

433 usable robots.txt bodies
15 recorded HTTP 404/410 no-file outcomes

Undetermined policy

52/500 · 10.4%

44 unresolved requests
8 invalid / HTML response bodies

The robots-policy denominator is 448. Unknown is not scored as allowed.

No-file responses mean no restrictions stated in robots.txt; they do not establish actual crawler access. The sample does not estimate prevalence across the public web.

Figure 1. Population accounting. The 448 determined outcomes include 433 usable bodies and 15 no-file responses; 52 outcomes remain unknown.
Source: Frozen baseline 500-site dataset, v3.1.2. Capture provenance is preserved in the evidence archive.

Seven findings that matter

33.9% explicitly addressed at least one AI-related token. That is 152 of 448 determined sites. Not naming a token is not the same as deciding to allow AI training.
Training and collection restrictions were more frequent than search restrictions. Across site-by-token verdict pairs, full blocks occurred in 14.3% of the 12-token training/collection/usage group, compared with 7.8% of six search-token pairs and 7.8% of four user-triggered-token pairs. Because group sizes differ, same-operator comparisons are more informative.
The most frequently blocked token was CCBot: 83/448 (18.5%), followed by ClaudeBot and Bytespider at 79/448 each (17.6%), and GPTBot at 76/448 (17.0%). Small differences in this deliberately selected sample should not be treated as a global ranking.
12.9% fully restricted a core AI search bot: 58/448. Twenty-four sites blocked all three. The broader six-token search group yielded 67/448 (15.0%).
News was the clearest sampling-bucket outlier: 41/59 (69.5%) blocked a core search token, compared with 0/56 among SaaS and technology sites. Neither figure estimates a sector-wide population rate.
Two sites fully blocked all 22 tracked tokens under the provider-aware parser: USA Today and Reddit. Applying Apple’s documented Googlebot fallback changes the Applebot verdict at 14 domains; without it, X and the UAE telecom operator du would also appear to block all 22 tokens.
100/500 (20.0%) had a detected /llms.txt file meeting the strict-MIME rule. Of those, 94 began with a conventional Markdown heading, five were link lists, and one used an unusual # banner. Ninety passed a strict Markdown-link or standalone-URL test, while 99 contained at least one absolute HTTP(S) URL. Neither measurement establishes usefulness or AI consumption.

What do robots.txt restrictions actually control?

The Robots Exclusion Protocol (RFC 9309) defines how cooperating automated fetchers interpret a site’s robots.txt path rules. It is a crawl-access convention, not a cryptographic access-control mechanism, copyright license, or guarantee that a bot will follow instructions. A server can publish an Allow rule but still deliver a challenge or HTTP 403 response to a particular network. A bot may also have learned about a URL through other indexes or references.

Three distinctions prevent the most common mistakes:

Training access is not identical to search access. OpenAI documents independent controls for GPTBot and OAI-SearchBot. Anthropic likewise distinguishes ClaudeBot, Claude-SearchBot, and Claude-User. A training opt-out does not automatically remove a site from an answer engine’s retrieval system.

User-triggered fetches are not ordinary scheduled crawls. OpenAI says robots.txt rules may not apply to ChatGPT-User, because the interaction originates from a user’s request. Perplexity’s crawler documentation separately describes Perplexity-User and PerplexityBot, with different behavior for user-initiated requests. Therefore the audit reports user-triggered tokens for completeness, not as proof that these systems obey every rule or honor the same semantics as background crawlers.

A usage-control token is not necessarily a fetching crawler. Google’s Google-Extended and Apple’s Applebot-Extended are robots.txt controls for certain downstream uses of already crawled content. Treating them as standalone HTTP clients would misrepresent what a server log can show.

Which tokens were included?

Table 1. The 22 AI-related robots.txt tokens tracked in this audit, by function
Functional group	Tokens in the audit	How to interpret a full `Disallow: /`
Training, collection, and usage controls (12)	GPTBot, ClaudeBot, CCBot, Google-Extended, Meta-ExternalAgent, Applebot-Extended, Bytespider, Amazonbot, Diffbot, ImagesiftBot, AI2Bot, cohere-ai	A declared restriction for the applicable purpose. Exact product effects vary; Google-Extended and Applebot-Extended are usage-control tokens.
Search and retrieval-related (6)	OAI-SearchBot, Claude-SearchBot, PerplexityBot, Applebot, YouBot, DuckAssistBot	A declared restriction on this fetch/index access. The headline core search measure uses only the first three.
User-triggered fetch (4)	ChatGPT-User, Claude-User, Perplexity-User, MistralAI-User	A published directive, not necessarily one a provider promises to honor for direct user-initiated requests.

This was a fixed set of 22 AI-related robots.txt tokens at collection time. It is not an exhaustive census of every AI service or bot on the internet. Other operator tokens, including Mistral’s documented indexing and training identifiers, are outside this release’s fixed scope.

Only full root-path blocks are counted below. An entry marked Partial has one or more path restrictions but does not disallow / and is not counted as a full-site block.

Table 2. Full root-path blocks by token, among 448 sites with a determined policy (baseline)
Token	Purpose in this study	Fully blocked	Share of 448
CCBot	Common Crawl collection	83	18.5%
ClaudeBot	Anthropic training-related	79	17.6%
Bytespider	Collection-related	79	17.6%
GPTBot	OpenAI training-related	76	17.0%
Applebot-Extended	Apple usage-control token	68	15.2%
Diffbot	Data extraction / collection	64	14.3%
Meta-ExternalAgent	Meta collection-related	61	13.6%
Amazonbot	Amazon collection-related	61	13.6%
Google-Extended	Google usage-control token	57	12.7%
PerplexityBot	Perplexity search-related	55	12.3%
OAI-SearchBot	OpenAI search-related	34	7.6%
Claude-SearchBot	Anthropic search-related	33	7.4%
Applebot	Apple search, provider-aware rules	7	1.6%

The full release CSV includes all 22 tokens, plus Allowed, Partial, and Blocked outcomes. That distinction matters: generic User-agent: * rules commonly disallow URLs such as /checkout, /admin, or internal-search paths, meaning Partial is the most common verdict for many tokens. Calling every partial restriction an AI-bot block would dramatically exaggerate the phenomenon.

ASFAND YAR JUNEJO / RESEARCH LAB · FIGURE 02

Published policy outcomes across all 22 tokens

Among 448 determined sites. Blocked / Partial / Allowed. Every row totals 448.

Token	Share of 448 (0–100%)	Blocked / Partial / Allowed
TRAINING / COLLECTION / USAGE CONTROL
GPTBot		76 / 301 / 71
ClaudeBot		79 / 300 / 69
CCBot		83 / 301 / 64
Google-Extended		57 / 321 / 70
Meta-ExternalAgent		61 / 319 / 68
Applebot-Extended		68 / 311 / 69
Bytespider		79 / 305 / 64
Amazonbot		61 / 316 / 71
Diffbot		64 / 320 / 64
ImagesiftBot		45 / 338 / 65
AI2Bot		37 / 348 / 63
cohere-ai		58 / 325 / 65
SEARCH / RETRIEVAL
OAI-SearchBot		34 / 335 / 79
Claude-SearchBot		33 / 345 / 70
PerplexityBot		55 / 318 / 75
Applebot		7 / 367 / 74
YouBot		46 / 333 / 69
DuckAssistBot		34 / 348 / 66
USER-TRIGGERED FETCH
ChatGPT-User		41 / 332 / 75
Claude-User		35 / 346 / 67
Perplexity-User		34 / 343 / 71
MistralAI-User		29 / 354 / 65

Blocked means an effective root disallow. Partial means the root is allowed but at least one nonempty path restriction exists. Allowed means no applicable disallow. Applebot uses its documented Googlebot fallback. Provider enforcement is not measured.

Figure 2. All 22 token classifications, with exact Blocked / Partial / Allowed counts. Partial restrictions are not counted as full root blocks.
Source: Frozen baseline 500-site dataset, v3.1.2. Capture provenance is preserved in the evidence archive.

Applebot is a special case

Apple’s published crawler guidance says that when a file does not mention Applebot but does mention Googlebot, Applebot follows the Googlebot instructions. Generic exact-token matching would instead fall back to User-agent: *. These interpretations are not interchangeable.

In this archive, the vendor-specific rule alters Applebot’s verdict on 14 domains. Under the generic parser, 10/448 sites fully blocked Applebot and four appeared to block all 22 tokens. Under the provider-aware interpretation used in the final dataset, Applebot is fully blocked on 7/448, and only USA Today and Reddit block every tracked token. These results reflect Apple’s documented policy interpretation, not an experiment observing real Applebot requests.

Which types of websites restrict AI search bots?

The sampling frame deliberately included major industries plus Pakistan and Gulf/MENA regional oversamples. The latter contain mixed industries and must not be interpreted as mutually comparable industrial sectors. Every row below uses the number of websites with determined robots.txt results. The /llms.txt column has a different denominator: conclusive /llms.txt checks within those determined websites. Numerator and denominator are both shown.

Table 3. Results by industry sampling bucket (baseline, determined sites)
Industry sampling bucket	Determined n	Names AI token	GPTBot fully blocked	Blocks at least one core search token	Strict-MIME `/llms.txt` among conclusive
Global news & media	59	51/59 (86.4%)	35/59 (59.3%)	41/59 (69.5%)	4/42 (9.5%)
Content platforms & communities	34	21/34 (61.8%)	16/34 (47.1%)	11/34 (32.4%)	3/31 (9.7%)
E-commerce & retail	41	15/41 (36.6%)	6/41 (14.6%)	3/41 (7.3%)	9/37 (24.3%)
Marketing, SEO & media industry	38	12/38 (31.6%)	3/38 (7.9%)	0/38 (0.0%)	18/37 (48.6%)
SaaS & technology platforms	56	16/56 (28.6%)	2/56 (3.6%)	0/56 (0.0%)	28/54 (51.9%)
Healthcare & pharma	28	7/28 (25.0%)	3/28 (10.7%)	0/28 (0.0%)	3/24 (12.5%)
Travel, hospitality & airlines	23	5/23 (21.7%)	1/23 (4.3%)	0/23 (0.0%)	4/18 (22.2%)
Education & research	29	5/29 (17.2%)	3/29 (10.3%)	1/29 (3.4%)	4/26 (15.4%)
Government & public sector (global)	14	2/14 (14.3%)	0/14 (0.0%)	0/14 (0.0%)	0/13 (0.0%)
Finance, banking & insurance	36	4/36 (11.1%)	1/36 (2.8%)	0/36 (0.0%)	8/32 (25.0%)
AI companies	14	1/14 (7.1%)	0/14 (0.0%)	0/14 (0.0%)	6/14 (42.9%)

Table 4. Results for the two regional oversamples, mixed industries (baseline, determined sites)
Regional oversample, mixed industries	Determined n	Names AI token	GPTBot fully blocked	Blocks at least one core search token	Strict-MIME `/llms.txt` among conclusive
Gulf/MENA	24	6/24 (25.0%)	2/24 (8.3%)	2/24 (8.3%)	7/22 (31.8%)
Pakistan top sites	52	7/52 (13.5%)	4/52 (7.7%)	0/52 (0.0%)	6/51 (11.8%)

Small cells matter. For a bucket containing 14 determined sites, one additional site changes its percentage by approximately 7.1 points. These figures describe these sampled domains in the baseline collection window, not all newsrooms, governments, national websites, or SaaS firms.

ASFAND YAR JUNEJO / RESEARCH LAB · FIGURE 03

Core AI search restrictions by sampling bucket

At least one full root block for OAI-SearchBot, Claude-SearchBot or PerplexityBot. Bars use a common 0–100% scale.

Global news & media41/59 (69.5%)

Content platforms & communities11/34 (32.4%)

E-commerce & retail3/41 (7.3%)

Education & research1/29 (3.4%)

AI companies0/14 (0.0%)

Finance, banking & insurance0/36 (0.0%)

Government & public sector (global)0/14 (0.0%)

Healthcare & pharma0/28 (0.0%)

Marketing, SEO & media industry0/38 (0.0%)

SaaS & technology platforms0/56 (0.0%)

Travel, hospitality & airlines0/23 (0.0%)

REGIONAL OVERSAMPLES · MIXED INDUSTRIES

Gulf/MENA2/24 (8.3%)

Pakistan top sites0/52 (0.0%)

The exact numerator and determined denominator are shown for each bucket. Zero is an observed sample count. Purposive, uneven bucket sizes do not support sector-wide prevalence or significance claims.

Figure 3. Descriptive core-search block rates by sampling bucket. Regional oversamples are separated from industry buckets; counts are not population estimates.
Source: Frozen baseline 500-site dataset, v3.1.2. Capture provenance is preserved in the evidence archive.

Why do news publishers stand apart?

News and media was the most restrictive bucket for the three designated search bots, with 41 of 59 sites fully blocking one or more. The archived instructions show stronger training-collection restrictions at multiple major outlets as well. It is reasonable to discuss publishers’ licensing and content-control incentives as context, but this dataset does not identify the motivations of individual organizations or prove that restricted bots reduce traffic, citations, or unauthorized reuse.

What about Pakistan, finance, and AI-company websites?

Of the 52 determined websites in the Pakistan oversample, seven explicitly named at least one tracked token and four fully blocked GPTBot. None fully blocked a core search token. Eight of the original 60 sites lacked determined robots verdicts; readers should not mistake this purposive regional sample for national coverage.

In the 36 determined finance, banking, and insurance sites, one fully blocked GPTBot, and none fully blocked a core search token. In the 14 determined AI-company websites, none fully blocked GPTBot or the three core search tokens, although that does not mean they imposed no path-specific or network restrictions. A provider’s own publishing policy is not evidence that its training or search product has the same policy for other websites.

How common is `/llms.txt`, and is it worth creating?

A detected /llms.txt file meeting the strict-MIME rule appeared on 100 of the 500 sampled domains (20.0%). One additional Markdown body had a content-type mismatch, 311 checks were classified absent, and 88 remained inconclusive. Among the 412 conclusive checks, 100/412 (24.3%) had a file. The conservative all-frame presence bound is 20.0%, and if every inconclusive check eventually proved positive, it would be 37.6%. Those are missing-data bounds, not confidence intervals.

Classification clarification (v3.1.2): The 100-file headline is a strict-MIME metric: these Markdown-like files were served without an HTML content type. One additional Markdown document, SAP’s 9,578-byte response, was served as text/html and is recorded as present_content_type_mismatch, not absent. There are 311 absent results and 88 unverified checks. Content-based presence is therefore 101/500 (20.2%), or 101/412 (24.5%) among conclusive checks. Strict-MIME presence remains 100/500 (20.0%), or 100/412 (24.3%). The strict-MIME upper missingness bound is 37.6%; the content-based bound is 37.8%. No archived source bytes were changed.

The format breakdown for the 100 strict-MIME files was:

94 files beginning with a Markdown heading after removal of a UTF-8 byte-order mark.
5 link-list style files without an initial Markdown heading.
1 custom # banner format, published by Lufthansa.

Do not confuse link-pattern detection with the presence of links. A strict parser counted Markdown hyperlinks or standalone absolute-URL lines in 90 files, while a broader scan found at least one absolute HTTP(S) URL anywhere in 99. One minimal file, AFP’s # AFP.com, contained no absolute HTTP(S) URL. Neither measure tests link validity, relevance, completeness, or whether an AI product consumes the content.

The llms.txt proposal is a convention, not an IETF robots standard. Google’s guide to generative AI features in Search says Google Search ignores llms.txt files, so maintaining one neither helps nor harms visibility or rankings in Google Search. It may be useful to other services or workflows that choose to ingest it, but this audit makes no causal claim about citation or traffic benefits.

For a site owner, the responsible sequence is to maintain accessible canonical HTML pages, accurate site navigation, a supported sitemap, and valid structured data when appropriate. If there is a documented consumer for a curated /llms.txt manifest, treat it as a maintained optional interface, not as a substitute for those foundations.

ASFAND YAR JUNEJO / RESEARCH LAB · FIGURE 04

llms.txt classification and missingness bounds

Strict-MIME detections

100/500 · 20.0%

Markdown, MIME mismatch

1/500 · 0.2%

Absent / non-manifest

311/500 · 62.2%

Unverified

88/500 · 17.6%

Strict-MIME: 100/500 = 20.0%. Content-based: 101/500 = 20.2%.

Among 412 conclusive checks: 24.3% strict-MIME and 24.5% content-based.

All-frame arithmetic bounds

Names at least one AI token: 30.4–40.8%

GPTBot root block: 15.2–25.6%

Strict-MIME llms.txt: 20.0–37.6%

Content-based llms.txt: 20.2–37.8%

Interval graphic scale: 0% at left; 50% at right.

Bounds assign every unresolved outcome first negative, then positive. They are not confidence intervals. The additional Markdown document is SAP’s 9,578-byte response served as text/html. Presence does not establish quality, ingestion, citations or traffic effects.

Figure 4. Strict-MIME presence remains 100/500; content-based presence includes SAP and is 101/500. Arithmetic bounds address missing outcomes, not statistical sampling uncertainty.
Source: Frozen baseline 500-site dataset, v3.1.2. Capture provenance is preserved in the evidence archive.

What should technical SEO and AI visibility teams do with these findings?

1. Write a crawler-access policy before editing directives

Decide separately whether each vendor may fetch for model training, search discovery, and user-triggered retrieval. Include legal and security stakeholders where intellectual property, licensing, privacy, or paid content is involved. A B2B software company seeking product discovery may reasonably make different choices from a publisher protecting licensed content. The dataset cannot prescribe a universal allow-or-block answer.

For implementation support, see the technical SEO and AI visibility audit services, which cover crawler access, rendering, indexing and measurement.

2. Inspect effective rules, not just named user-agent lines

For each prioritized token, evaluate its applicable group, the wildcard fallback, rule precedence, and the exact path being tested. For Applebot, check whether Apple’s special Googlebot inheritance rule applies. Confirm that an apparently permissive named group does not conceal problematic route behavior, and evaluate key pages rather than only /.

The following is an illustrative policy skeleton, not a copy-and-paste configuration. It demonstrates how a site might distinguish training opt-outs from one search-crawler allowance:

User-agent: GPTBot
Disallow: /

User-agent: ClaudeBot
Disallow: /

User-agent: OAI-SearchBot
Allow: /

User-agent: Claude-SearchBot
Allow: /

User-agent: *
Disallow: /internal-search/
Disallow: /checkout/

Before deployment, test your real crawler policy with a competent robots parser: named-group handling, other bots not listed, third-party access requirements, staging versus production differences, and the possibility that a specific named group replaces wildcard housekeeping rules. Blocking GPTBot does not necessarily undo historical training or prevent indirect mentions of your brand.

For an initial path-level check, use the robots.txt checker for AI crawlers. Its current bot list and classifications differ from this research release; it is not a replay of the study parser or a test of live WAF access.

3. Check the WAF, CDN, and bot verification layer separately

robots.txt is an instruction. An HTTP 403, 429, or challenge page is an actual access outcome. Review server and CDN logs for verified crawler IP ranges or validated reverse DNS rather than trusting a spoofable user-agent string alone. Protect sensitive routes; do not indiscriminately allowlist all traffic pretending to be a named AI bot.

For example, Cloudflare’s Bot Fight Mode cannot be bypassed by normal WAF Skip rules, while Super Bot Fight Mode supports more granular exceptions. Product-specific rules and plan capabilities matter. This study did not measure WAF outcomes, so the point here is a practical audit recommendation supported by provider documentation, not a finding from the 500-site sample.

4. Make important answers accessible in initial HTML

A crawl permission does not prove that a retrieval system can read a dynamically rendered product specification, pricing explanation, or research methodology. Compare the initial HTML with a rendered view for representative pages, especially client-side applications; ensure canonical facts and links are discoverable without relying entirely on late JavaScript execution. Google’s AI feature guidance emphasizes crawl accessibility, internal linking, and available textual content. This study did not independently benchmark AI crawlers’ render engines or timeout budgets here.

5. Measure actual search outcomes separately

If visibility is the business goal, the next study must record prompt text, engine/version, location, date, answer appearance, target-domain link, mention without link, and actual referrals. For causal questions, compare controlled interventions with suitable controls. Do not infer citation lifts from this crawler-policy audit.

For the broader distinction between discovery, retrieval and citation measurement, see how AI search visibility is diagnosed and measured.

Research data, method, and reproducibility

The study uses a preassembled, purposive frame of 500 well-known websites assigned to 13 mutually exclusive sampling buckets. It is intentionally enriched with Pakistan and Gulf/MENA sites, and is not statistically representative of the public web. Robots files were requested with retry and validity checks; capture dates are preserved in the accompanying evidence archive. The release preserves 433 usable HTTP 200 robots.txt bodies, 15 recorded 404/410 no-file outcomes, eight quarantined HTML/invalid bodies, and 44 unresolved requests. Thus 448 of 500 (89.6%) have a determined policy classification.

The replayable parser reports Blocked when the effective rule disallows the root path /, Partial when / is not blocked but one or more subpaths are restricted, and Allowed when no applicable disallow rule is found. Apple’s documented Googlebot fallback is applied for Applebot alone when its own group is absent. The 22-token matrix, source capture indexes, and SHA-256 checks support inspection and reanalysis.

The archived /llms.txt collection includes a source index with stored-size and cryptographic checksum fields. All 433 usable robots.txt bodies and all 156 stored HTTP 200 /llms.txt response bodies passed the current internal archive-integrity checks. This demonstrates the release is internally consistent; it does not establish that a verified vendor crawler saw the same response when visiting from its own infrastructure.

A third-party parser comparison previously reported 9,517 agreements out of 9,526 generic binary root-path block comparisons (99.91%), with nine prefix-matching differences. That test checked only blocked versus not blocked, not the three-class Blocked/Partial/Allowed result. The Applebot vendor-specific fallback is tested separately in this release and is not counted as an error in the generic-parser comparison. See parser_validation.json and applebot_vendor_fallback.json for the exact scope. The external library comparison has not been freshly rerun in this release.

Missingness matters. Fifty-two sites could not be classified: 44 unresolved and eight unusable-body cases. The observed 152 named-token sites therefore imply a full-frame bound of 30.4% to 40.8% if all missing sites were negative or positive. Likewise, the GPTBot full-block rate lies between 15.2% and 25.6% under those extremes. These are arithmetic sensitivity bounds, not intervals derived from a sampling model. Failure may be related to the sites’ defenses, so it is inappropriate to assume missing sites resemble successful fetches.

Important remaining limits include missing early-pass HTTP response headers and exact per-fetch clock times, a mix of network vantage points, classification at the root path rather than every URL, and provider-specific enforcement that was not directly tested. Missing early-pass metadata has not been reconstructed. The separate crawler API follow-up resolves some failed resource checks; it does not retroactively complete the original capture metadata.

The broader evidence-led search methodology explains how findings are separated from interpretation. The study-specific methods and limits above govern this release.

The accompanying package contains the full CSV, per-file archives and indexes, replay scripts, validation report, and release changelog. Reuse should cite this archived snapshot and the precise version, rather than silently treating the numbers as current live website policies.

ASFAND YAR JUNEJO / RESEARCH LAB · FIGURE 05

What the research can and cannot establish

01 · Archived responses (Observed)

433 usable robots.txt bodies and 156 stored HTTP 200 llms.txt bodies. Hashes test consistency against their source indexes.

02 · Published policy interpretation (Derived)

Root-path classifications for 22 tokens, including Applebot fallback. Replay reproduces the derivation; non-root URL behavior remains outside the headline test.

03 · Actual crawler access (Not measured)

Verified vendor requests, WAF/CDN challenges, response variation and compliance require separate network-level observations.

04 · AI visibility and business effects (Not measured)

Citations, mentions, referrals, training use and commercial outcomes require separate measurements and suitable causal designs.

These are distinct evidence questions, not measured causal steps. Sampling limits, unknown outcomes and incomplete early capture metadata remain part of the interpretation.

Figure 5. The evidence boundary separates stored responses and reproducible interpretations from unmeasured access behavior and visibility effects.
Source: Frozen baseline 500-site dataset, v3.1.2. Capture provenance is preserved in the evidence archive.

Crawler API verification: what changed?

41 of 140 previously unresolved resource checks are now conclusive: eight robots.txt checks and 33 llms.txt checks. The pass used 247 target requests across 99 domains, plus four control/pilot requests. Each remaining inconclusive resource received an alternate-profile attempt with the study’s browser user agent and a U.S. crawler location. 44 robots policies and 55 llms.txt checks remain undetermined or unverified.

Frozen baseline versus reconciled follow-up view
Measure	Frozen baseline v3.1.2	Reconciled verification view
Determined robots policies	448 / 500	456 / 500
Undetermined robots policies	52	44
Usable robots bodies	433	439
Recorded no-file robots responses	15	17
GPTBot root blocks	76 / 448 (17.0%)	78 / 456 (17.1%)
Any core-search token blocked	58 / 448 (12.9%)	59 / 456 (12.9%)
Strict-rule llms.txt detections	100 / 500 (20.0%)	108 / 500 (21.6%)
Content-based llms document presence	101 / 500 (20.2%)	110 / 500 (22.0%)
llms.txt absent / non-manifest	311	335
Unverified llms.txt checks	88	55

These are different evidence views. The article’s original findings and Figures 1–5 report the frozen baseline. The register and exports below combine that baseline with newly resolved crawler API responses. They do not represent a single simultaneous crawl, and the small changes must not be interpreted as a trend in website policy.

Six recovered robots bodies were parsed with the same fixed 22-token, provider-aware parser. Two HTTP 404 outcomes use the study’s no-declared-restrictions convention. A control robots retrieval reproduced the archived body hash. All 34 stored HTTP 200 response texts matched the crawler API’s reported byte lengths. Stored hashes identify the API-returned text encoded as UTF-8; they do not independently prove original wire bytes, complete headers or exact server response times.

Two content-based exceptions remain separate from strict detections: SAP’s Markdown body served with an HTML MIME type, and CVS.com’s Site/Owner/Purpose reference document with bulleted bare URLs. CVS is present in a nonstandard format, not absent. TIME’s detected file is a policy notice, so file detection must not be mistaken for a useful navigation manifest or AI consumption.

The reconciled llms results are 108 strict detections, one MIME mismatch, one nonstandard-format document, 335 absent/non-manifest outcomes and 55 unverified checks. Among 445 conclusive checks, content-based presence is 110/445 (24.7%). Its all-frame missingness bounds are 22.0–33.0%; strict-rule bounds are 21.6–32.6%. These arithmetic bounds are not confidence intervals.

HTTP denials, timeouts, authentication gates, missing target metadata, HTML challenges and invalid robots bodies remain unresolved. Provider-recorded 404s are observations from that request profile; changes between profiles remain in the attempt log. No claim of 100% coverage, crawler compliance or AI visibility is made.

Inspect the reconciled 500-site register · Read the baseline methodology

RESEARCH LAB · NOVEMBER 2026 EDITION

500-Website Evidence Register

Reconciled verification view: 439 usable robots bodies, 17 no-file outcomes and 44 undetermined policies. Baseline values are retained in every record and export. This is a mixed-pass view, not a new simultaneous 500-site snapshot. Compare the two evidence views.

Inspect every website in the fixed sampling frame, including unsuccessful checks. Expand a record for all 22 token classifications, stored-file references and SHA-256 hashes. Use the token filter to examine any tracked crawler.

439 usable responses · 17 no-file responses · 7 invalid responses · 37 unresolved requests. Policy classifications are determined for 456 websites and undetermined for 44. The 13 sampling buckets are purposive, not population weights.

Results describe archived website directives interpreted by the study parser. They do not establish crawler compliance, successful access, AI citations, or current website policy. Allowed means no applicable disallow rule in the study interpretation. Partial means subpath restrictions without a root block. A missing file is treated as no declared restrictions; failed or invalid checks remain Undetermined. Core search means OAI-SearchBot, Claude-SearchBot and PerplexityBot.

Archive references identify files inside the research package. They are not live evidence links. The standalone register downloads below contain all 500 records, all 22 token results and provenance metadata, including original capture timestamps. Raw response files are supplied separately in the research package.

Download full 500-site CSV (0.9 MB)Download 140-check verification CSV (61 KB)

Downloads include all 500 records regardless of filters. The CSV files are fixed copies of this release; the JSON file is built in your browser when you choose it.

All 500 websites are listed below. JavaScript enables search, filters, pagination, detailed evidence and downloads.

Reconciled results for all 500 sampled websites, with frozen baseline values retained
Website	Sampling bucket	Robots fetch	GPTBot	Core-search root block	llms.txt	Full record
nytimes.comThe New York Times	Global news & media	Usable responseHTTP 200	Blocked	At least one blocked	Absent / non-manifest	Inspect nytimes.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
washingtonpost.comThe Washington Post	Global news & media	Usable responseHTTP 200	Partial	At least one blocked	Unverified	Inspect washingtonpost.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
wsj.comThe Wall Street Journal	Global news & media	Usable responseHTTP 200	Partial	At least one blocked	Unverified	Inspect wsj.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
usatoday.comUSA Today	Global news & media	Usable responseHTTP 200	Blocked	At least one blocked	Absent / non-manifest	Inspect usatoday.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
latimes.comLos Angeles Times	Global news & media	Usable responseHTTP 200	Blocked	None fully blocked	Absent / non-manifest	Inspect latimes.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
npr.orgNPR	Global news & media	Usable responseHTTP 200	Blocked	At least one blocked	Absent / non-manifest	Inspect npr.org: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
cnn.comCNN	Global news & media	Usable responseHTTP 200	Blocked	At least one blocked	Absent / non-manifest	Inspect cnn.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
nbcnews.comNBC News	Global news & media	Usable responseHTTP 200	Blocked	At least one blocked	Absent / non-manifest	Inspect nbcnews.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
foxnews.comFox News	Global news & media	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect foxnews.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
cbsnews.comCBS News	Global news & media	Usable responseHTTP 200	Blocked	None fully blocked	Absent / non-manifest	Inspect cbsnews.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
bloomberg.comBloomberg	Global news & media	Usable responseHTTP 200	Blocked	At least one blocked	Detected (strict rule)	Inspect bloomberg.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
forbes.comForbes	Global news & media	Usable responseHTTP 200	Blocked	At least one blocked	Absent / non-manifest	Inspect forbes.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
time.comTIME	Global news & media	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict rule)	Inspect time.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
theatlantic.comThe Atlantic	Global news & media	Usable responseHTTP 200	Allowed	At least one blocked	Absent / non-manifest	Inspect theatlantic.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
politico.comPolitico	Global news & media	Usable responseHTTP 200	Blocked	At least one blocked	Absent / non-manifest	Inspect politico.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
axios.comAxios	Global news & media	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect axios.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
huffpost.comHuffPost	Global news & media	Usable responseHTTP 200	Blocked	At least one blocked	Absent / non-manifest	Inspect huffpost.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
businessinsider.comBusiness Insider	Global news & media	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect businessinsider.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
bbc.comBBC	Global news & media	Usable responseHTTP 200	Blocked	At least one blocked	Absent / non-manifest	Inspect bbc.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
theguardian.comThe Guardian	Global news & media	Usable responseHTTP 200	Partial	At least one blocked	Absent / non-manifest	Inspect theguardian.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
telegraph.co.ukThe Telegraph	Global news & media	Usable responseHTTP 200	Allowed	None fully blocked	Absent / non-manifest	Inspect telegraph.co.uk: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
thetimes.co.ukThe Times	Global news & media	Usable responseHTTP 200	Partial	At least one blocked	Absent / non-manifest	Inspect thetimes.co.uk: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
independent.co.ukThe Independent	Global news & media	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect independent.co.uk: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
ft.comFinancial Times	Global news & media	Usable responseHTTP 200	Partial	At least one blocked	Absent / non-manifest	Inspect ft.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
economist.comThe Economist	Global news & media	Usable responseHTTP 200	Blocked	At least one blocked	Absent / non-manifest	Inspect economist.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
dailymail.co.ukDaily Mail	Global news & media	Usable responseHTTP 200	Blocked	At least one blocked	Detected (strict MIME)	Inspect dailymail.co.uk: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
mirror.co.ukDaily Mirror	Global news & media	Usable responseHTTP 200	Blocked	At least one blocked	Absent / non-manifest	Inspect mirror.co.uk: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
itv.comITV	Global news & media	Unresolved requestHTTP not recorded	Undetermined	Undetermined	Unverified	Inspect itv.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
reuters.comReuters	Global news & media	Usable responseHTTP 200	Blocked	At least one blocked	Unverified	Inspect reuters.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
apnews.comAssociated Press	Global news & media	Usable responseHTTP 200	Blocked	At least one blocked	Absent / non-manifest	Inspect apnews.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
afp.comAFP	Global news & media	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect afp.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
lemonde.frLe Monde	Global news & media	Usable responseHTTP 200	Partial	At least one blocked	Absent / non-manifest	Inspect lemonde.fr: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
france24.comFrance 24	Global news & media	Usable responseHTTP 200	Blocked	At least one blocked	Unverified	Inspect france24.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
euronews.comEuronews	Global news & media	Usable responseHTTP 200	Blocked	None fully blocked	Absent / non-manifest	Inspect euronews.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
spiegel.deDer Spiegel	Global news & media	Usable responseHTTP 200	Blocked	At least one blocked	Absent / non-manifest	Inspect spiegel.de: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
dw.comDeutsche Welle	Global news & media	Usable responseHTTP 200	Blocked	At least one blocked	Absent / non-manifest	Inspect dw.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
bild.deBild	Global news & media	Usable responseHTTP 200	Partial	At least one blocked	Absent / non-manifest	Inspect bild.de: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
faz.netFrankfurter Allgemeine Zeitung	Global news & media	Usable responseHTTP 200	Blocked	At least one blocked	Absent / non-manifest	Inspect faz.net: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
sueddeutsche.deSueddeutsche Zeitung	Global news & media	Usable responseHTTP 200	Blocked	At least one blocked	Absent / non-manifest	Inspect sueddeutsche.de: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
elpais.comEl Pais	Global news & media	Usable responseHTTP 200	Partial	At least one blocked	Absent / non-manifest	Inspect elpais.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
rtve.esRTVE	Global news & media	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect rtve.es: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
corriere.itCorriere della Sera	Global news & media	Usable responseHTTP 200	Blocked	None fully blocked	Absent / non-manifest	Inspect corriere.it: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
repubblica.itLa Repubblica	Global news & media	Usable responseHTTP 200	Blocked	At least one blocked	Detected (strict rule)	Inspect repubblica.it: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
nos.nlNOS	Global news & media	Usable responseHTTP 200	Blocked	At least one blocked	Absent / non-manifest	Inspect nos.nl: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
cbc.caCBC	Global news & media	Usable responseHTTP 200	Blocked	At least one blocked	Absent / non-manifest	Inspect cbc.ca: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
theglobeandmail.comThe Globe and Mail	Global news & media	Usable responseHTTP 200	Blocked	At least one blocked	Absent / non-manifest	Inspect theglobeandmail.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
abc.net.auABC Australia	Global news & media	Usable responseHTTP 200	Blocked	At least one blocked	Absent / non-manifest	Inspect abc.net.au: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
smh.com.auThe Sydney Morning Herald	Global news & media	Usable responseHTTP 200	Blocked	At least one blocked	Absent / non-manifest	Inspect smh.com.au: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
theage.com.auThe Age	Global news & media	Usable responseHTTP 200	Blocked	At least one blocked	Absent / non-manifest	Inspect theage.com.au: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
stuff.co.nzStuff	Global news & media	Usable responseHTTP 200	Blocked	At least one blocked	Absent / non-manifest	Inspect stuff.co.nz: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
nzherald.co.nzThe New Zealand Herald	Global news & media	Usable responseHTTP 200	Blocked	At least one blocked	Absent / non-manifest	Inspect nzherald.co.nz: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
japantimes.co.jpThe Japan Times	Global news & media	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect japantimes.co.jp: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
asahi.comAsahi Shimbun	Global news & media	Usable responseHTTP 200	Blocked	At least one blocked	Absent / non-manifest	Inspect asahi.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
scmp.comSouth China Morning Post	Global news & media	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect scmp.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
straitstimes.comThe Straits Times	Global news & media	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect straitstimes.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
thehindu.comThe Hindu	Global news & media	Usable responseHTTP 200	Blocked	At least one blocked	Absent / non-manifest	Inspect thehindu.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
hindustantimes.comHindustan Times	Global news & media	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict rule)	Inspect hindustantimes.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
indianexpress.comThe Indian Express	Global news & media	Usable responseHTTP 200	Partial	At least one blocked	Unverified	Inspect indianexpress.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
koreaherald.comThe Korea Herald	Global news & media	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect koreaherald.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
bangkokpost.comBangkok Post	Global news & media	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect bangkokpost.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
microsoft.comMicrosoft	SaaS & technology platforms	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect microsoft.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
google.comGoogle	SaaS & technology platforms	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect google.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
apple.comApple	SaaS & technology platforms	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect apple.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
salesforce.comSalesforce	SaaS & technology platforms	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect salesforce.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
adobe.comAdobe	SaaS & technology platforms	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect adobe.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
oracle.comOracle	SaaS & technology platforms	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect oracle.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
sap.comSAP	SaaS & technology platforms	Usable responseHTTP 200	Partial	None fully blocked	Present, MIME mismatch	Inspect sap.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
ibm.comIBM	SaaS & technology platforms	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect ibm.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
cisco.comCisco	SaaS & technology platforms	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect cisco.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
intel.comIntel	SaaS & technology platforms	Usable responseHTTP 200	Allowed	None fully blocked	Detected (strict MIME)	Inspect intel.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
nvidia.comNVIDIA	SaaS & technology platforms	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect nvidia.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
dell.comDell Technologies	SaaS & technology platforms	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect dell.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
hp.comHP	SaaS & technology platforms	Usable responseHTTP 200	Allowed	None fully blocked	Absent / non-manifest	Inspect hp.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
lenovo.comLenovo	SaaS & technology platforms	Usable responseHTTP 200	Allowed	None fully blocked	Detected (strict MIME)	Inspect lenovo.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
samsung.comSamsung	SaaS & technology platforms	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect samsung.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
sony.comSony	SaaS & technology platforms	Unresolved requestHTTP not recorded	Undetermined	Undetermined	Unverified	Inspect sony.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
atlassian.comAtlassian	SaaS & technology platforms	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect atlassian.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
slack.comSlack	SaaS & technology platforms	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect slack.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
zoom.usZoom	SaaS & technology platforms	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect zoom.us: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
dropbox.comDropbox	SaaS & technology platforms	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect dropbox.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
box.comBox	SaaS & technology platforms	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict rule)	Inspect box.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
servicenow.comServiceNow	SaaS & technology platforms	Unresolved requestHTTP not recorded	Undetermined	Undetermined	Unverified	Inspect servicenow.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
workday.comWorkday	SaaS & technology platforms	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect workday.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
intuit.comIntuit	SaaS & technology platforms	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect intuit.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
autodesk.comAutodesk	SaaS & technology platforms	Unresolved requestHTTP not recorded	Undetermined	Undetermined	Unverified	Inspect autodesk.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
vmware.comVMware	SaaS & technology platforms	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect vmware.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
redhat.comRed Hat	SaaS & technology platforms	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect redhat.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
gitlab.comGitLab	SaaS & technology platforms	Usable responseHTTP 200	Partial	None fully blocked	Unverified	Inspect gitlab.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
docker.comDocker	SaaS & technology platforms	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect docker.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
cloudflare.comCloudflare	SaaS & technology platforms	Usable responseHTTP 200	Allowed	None fully blocked	Detected (strict MIME)	Inspect cloudflare.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
akamai.comAkamai	SaaS & technology platforms	Unresolved requestHTTP not recorded	Undetermined	Undetermined	Unverified	Inspect akamai.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
netflix.comNetflix	SaaS & technology platforms	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect netflix.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
spotify.comSpotify	SaaS & technology platforms	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect spotify.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
uber.comUber	SaaS & technology platforms	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect uber.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
snowflake.comSnowflake	SaaS & technology platforms	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect snowflake.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
datadoghq.comDatadog	SaaS & technology platforms	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect datadoghq.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
mongodb.comMongoDB	SaaS & technology platforms	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect mongodb.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
twilio.comTwilio	SaaS & technology platforms	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect twilio.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
zendesk.comZendesk	SaaS & technology platforms	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect zendesk.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
asana.comAsana	SaaS & technology platforms	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect asana.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
monday.commonday.com	SaaS & technology platforms	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect monday.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
notion.soNotion	SaaS & technology platforms	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect notion.so: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
figma.comFigma	SaaS & technology platforms	Usable responseHTTP 200	Blocked	None fully blocked	Absent / non-manifest	Inspect figma.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
canva.comCanva	SaaS & technology platforms	Usable responseHTTP 200	Blocked	None fully blocked	Absent / non-manifest	Inspect canva.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
miro.comMiro	SaaS & technology platforms	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect miro.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
airtable.comAirtable	SaaS & technology platforms	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect airtable.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
webflow.comWebflow	SaaS & technology platforms	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect webflow.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
wix.comWix	SaaS & technology platforms	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect wix.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
squarespace.comSquarespace	SaaS & technology platforms	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect squarespace.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
docusign.comDocuSign	SaaS & technology platforms	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect docusign.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
okta.comOkta	SaaS & technology platforms	Usable responseHTTP 200	Allowed	None fully blocked	Detected (strict MIME)	Inspect okta.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
crowdstrike.comCrowdStrike	SaaS & technology platforms	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect crowdstrike.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
splunk.comSplunk	SaaS & technology platforms	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect splunk.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
paloaltonetworks.comPalo Alto Networks	SaaS & technology platforms	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect paloaltonetworks.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
amd.comAMD	SaaS & technology platforms	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect amd.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
qualcomm.comQualcomm	SaaS & technology platforms	Usable responseHTTP 200	Allowed	None fully blocked	Detected (strict MIME)	Inspect qualcomm.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
broadcom.comBroadcom	SaaS & technology platforms	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect broadcom.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
huawei.comHuawei	SaaS & technology platforms	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect huawei.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
ericsson.comEricsson	SaaS & technology platforms	No file recordedHTTP 404	Allowed	None fully blocked	Absent / non-manifest	Inspect ericsson.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
jetbrains.comJetBrains	SaaS & technology platforms	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect jetbrains.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
amazon.comAmazon	E-commerce & retail	Usable responseHTTP 200	Blocked	At least one blocked	Absent / non-manifest	Inspect amazon.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
ebay.comeBay	E-commerce & retail	Usable responseHTTP 200	Blocked	At least one blocked	Absent / non-manifest	Inspect ebay.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
walmart.comWalmart	E-commerce & retail	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect walmart.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
target.comTarget	E-commerce & retail	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect target.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
etsy.comEtsy	E-commerce & retail	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect etsy.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
bestbuy.comBest Buy	E-commerce & retail	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect bestbuy.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
homedepot.comThe Home Depot	E-commerce & retail	Usable responseHTTP 200	Partial	None fully blocked	Unverified	Inspect homedepot.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
lowes.comLowe's	E-commerce & retail	Unresolved requestHTTP not recorded	Undetermined	Undetermined	Unverified	Inspect lowes.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
costco.comCostco	E-commerce & retail	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect costco.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
wayfair.comWayfair	E-commerce & retail	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect wayfair.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
chewy.comChewy	E-commerce & retail	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect chewy.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
kroger.comKroger	E-commerce & retail	Unresolved requestHTTP not recorded	Undetermined	Undetermined	Unverified	Inspect kroger.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
walgreens.comWalgreens	E-commerce & retail	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect walgreens.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
cvs.comCVS Pharmacy	E-commerce & retail	Usable responseHTTP 200	Partial	None fully blocked	Present, nonstandard format	Inspect cvs.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
macys.comMacy's	E-commerce & retail	Unresolved requestHTTP not recorded	Undetermined	Undetermined	Unverified	Inspect macys.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
nordstrom.comNordstrom	E-commerce & retail	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect nordstrom.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
nike.comNike	E-commerce & retail	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect nike.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
adidas.comAdidas	E-commerce & retail	Unresolved requestHTTP not recorded	Undetermined	Undetermined	Unverified	Inspect adidas.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
ikea.comIKEA	E-commerce & retail	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect ikea.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
alibaba.comAlibaba	E-commerce & retail	Usable responseHTTP 200	Blocked	None fully blocked	Absent / non-manifest	Inspect alibaba.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
jd.comJD.com	E-commerce & retail	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect jd.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
aliexpress.comAliExpress	E-commerce & retail	Usable responseHTTP 200	Blocked	None fully blocked	Absent / non-manifest	Inspect aliexpress.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
flipkart.comFlipkart	E-commerce & retail	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect flipkart.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
mercadolibre.comMercado Libre	E-commerce & retail	Usable responseHTTP 200	Blocked	At least one blocked	Absent / non-manifest	Inspect mercadolibre.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
rakuten.comRakuten	E-commerce & retail	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect rakuten.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
zalando.deZalando	E-commerce & retail	Usable responseHTTP 200	Blocked	None fully blocked	Detected (strict MIME)	Inspect zalando.de: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
asos.comASOS	E-commerce & retail	Unresolved requestHTTP not recorded	Undetermined	Undetermined	Unverified	Inspect asos.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
shopify.comShopify	E-commerce & retail	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect shopify.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
shein.comShein	E-commerce & retail	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect shein.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
temu.comTemu	E-commerce & retail	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect temu.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
wish.comWish	E-commerce & retail	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect wish.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
overstock.comOverstock	E-commerce & retail	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect overstock.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
zappos.comZappos	E-commerce & retail	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect zappos.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
sephora.comSephora	E-commerce & retail	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect sephora.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
ulta.comUlta Beauty	E-commerce & retail	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect ulta.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
gap.comGap	E-commerce & retail	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect gap.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
hm.comH&M	E-commerce & retail	Unresolved requestHTTP not recorded	Undetermined	Undetermined	Unverified	Inspect hm.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
zara.comZara	E-commerce & retail	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect zara.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
uniqlo.comUniqlo	E-commerce & retail	Usable responseHTTP 200	Partial	None fully blocked	Unverified	Inspect uniqlo.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
lego.comLEGO	E-commerce & retail	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect lego.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
decathlon.comDecathlon	E-commerce & retail	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect decathlon.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
carrefour.comCarrefour	E-commerce & retail	Usable responseHTTP 200	Partial	None fully blocked	Unverified	Inspect carrefour.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
tesco.comTesco	E-commerce & retail	Unresolved requestHTTP not recorded	Undetermined	Undetermined	Unverified	Inspect tesco.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
sainsburys.co.ukSainsbury's	E-commerce & retail	Unresolved requestHTTP not recorded	Undetermined	Undetermined	Unverified	Inspect sainsburys.co.uk: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
argos.co.ukArgos	E-commerce & retail	Unresolved requestHTTP not recorded	Undetermined	Undetermined	Unverified	Inspect argos.co.uk: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
otto.deOtto	E-commerce & retail	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect otto.de: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
bol.comBol	E-commerce & retail	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect bol.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
cdiscount.comCdiscount	E-commerce & retail	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect cdiscount.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
allegro.plAllegro	E-commerce & retail	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect allegro.pl: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
pinduoduo.comPinduoduo	E-commerce & retail	Usable responseHTTP 200	Allowed	None fully blocked	Absent / non-manifest	Inspect pinduoduo.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
jpmorganchase.comJPMorgan Chase	Finance, banking & insurance	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect jpmorganchase.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
bankofamerica.comBank of America	Finance, banking & insurance	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect bankofamerica.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
wellsfargo.comWells Fargo	Finance, banking & insurance	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect wellsfargo.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
citi.comCiti	Finance, banking & insurance	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect citi.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
goldmansachs.comGoldman Sachs	Finance, banking & insurance	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect goldmansachs.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
morganstanley.comMorgan Stanley	Finance, banking & insurance	Usable responseHTTP 200	Allowed	None fully blocked	Absent / non-manifest	Inspect morganstanley.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
americanexpress.comAmerican Express	Finance, banking & insurance	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect americanexpress.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
visa.comVisa	Finance, banking & insurance	Usable responseHTTP 200	Allowed	None fully blocked	Absent / non-manifest	Inspect visa.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
mastercard.comMastercard	Finance, banking & insurance	Unresolved requestHTTP not recorded	Undetermined	Undetermined	Unverified	Inspect mastercard.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
paypal.comPayPal	Finance, banking & insurance	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect paypal.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
capitalone.comCapital One	Finance, banking & insurance	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect capitalone.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
discover.comDiscover	Finance, banking & insurance	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect discover.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
hsbc.comHSBC	Finance, banking & insurance	Usable responseHTTP 200	Allowed	None fully blocked	Absent / non-manifest	Inspect hsbc.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
barclays.comBarclays	Finance, banking & insurance	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect barclays.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
lloydsbank.comLloyds Bank	Finance, banking & insurance	Invalid responseHTTP 200	Undetermined	Undetermined	Absent / non-manifest	Inspect lloydsbank.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
santander.comSantander	Finance, banking & insurance	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect santander.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
bnpparibas.comBNP Paribas	Finance, banking & insurance	Unresolved requestHTTP not recorded	Undetermined	Undetermined	Unverified	Inspect bnpparibas.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
ing.comING	Finance, banking & insurance	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect ing.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
ubs.comUBS	Finance, banking & insurance	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect ubs.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
deutschebank.comDeutsche Bank	Finance, banking & insurance	Unresolved requestHTTP not recorded	Undetermined	Undetermined	Unverified	Inspect deutschebank.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
societegenerale.comSociete Generale	Finance, banking & insurance	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect societegenerale.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
bbva.comBBVA	Finance, banking & insurance	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect bbva.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
unicreditgroup.euUniCredit	Finance, banking & insurance	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect unicreditgroup.eu: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
allianz.comAllianz	Finance, banking & insurance	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect allianz.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
axa.comAXA	Finance, banking & insurance	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect axa.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
metlife.comMetLife	Finance, banking & insurance	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect metlife.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
prudential.comPrudential Financial	Finance, banking & insurance	Usable responseHTTP 200	Blocked	None fully blocked	Absent / non-manifest	Inspect prudential.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
aig.comAIG	Finance, banking & insurance	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect aig.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
progressive.comProgressive	Finance, banking & insurance	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect progressive.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
geico.comGEICO	Finance, banking & insurance	Usable responseHTTP 200	Allowed	None fully blocked	Detected (strict rule)	Inspect geico.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
statefarm.comState Farm	Finance, banking & insurance	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect statefarm.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
fidelity.comFidelity Investments	Finance, banking & insurance	Usable responseHTTP 200	Partial	None fully blocked	Unverified	Inspect fidelity.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
vanguard.comVanguard	Finance, banking & insurance	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect vanguard.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
schwab.comCharles Schwab	Finance, banking & insurance	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect schwab.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
blackrock.comBlackRock	Finance, banking & insurance	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect blackrock.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
stripe.comStripe	Finance, banking & insurance	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect stripe.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
squareup.comSquare	Finance, banking & insurance	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect squareup.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
robinhood.comRobinhood	Finance, banking & insurance	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect robinhood.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
coinbase.comCoinbase	Finance, banking & insurance	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect coinbase.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
rbc.comRoyal Bank of Canada	Finance, banking & insurance	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect rbc.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
pfizer.comPfizer	Healthcare & pharma	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect pfizer.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
novartis.comNovartis	Healthcare & pharma	Usable responseHTTP 200	Allowed	None fully blocked	Absent / non-manifest	Inspect novartis.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
roche.comRoche	Healthcare & pharma	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect roche.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
merck.comMerck & Co.	Healthcare & pharma	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect merck.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
astrazeneca.comAstraZeneca	Healthcare & pharma	Usable responseHTTP 200	Allowed	None fully blocked	Absent / non-manifest	Inspect astrazeneca.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
gsk.comGSK	Healthcare & pharma	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect gsk.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
sanofi.comSanofi	Healthcare & pharma	Usable responseHTTP 200	Allowed	None fully blocked	Absent / non-manifest	Inspect sanofi.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
bayer.comBayer	Healthcare & pharma	Unresolved requestHTTP not recorded	Undetermined	Undetermined	Unverified	Inspect bayer.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
jnj.comJohnson & Johnson	Healthcare & pharma	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect jnj.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
abbvie.comAbbVie	Healthcare & pharma	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect abbvie.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
amgen.comAmgen	Healthcare & pharma	Usable responseHTTP 200	Allowed	None fully blocked	Absent / non-manifest	Inspect amgen.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
gilead.comGilead Sciences	Healthcare & pharma	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect gilead.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
lilly.comEli Lilly	Healthcare & pharma	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect lilly.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
novonordisk.comNovo Nordisk	Healthcare & pharma	Usable responseHTTP 200	Allowed	None fully blocked	Absent / non-manifest	Inspect novonordisk.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
takeda.comTakeda	Healthcare & pharma	Usable responseHTTP 200	Allowed	None fully blocked	Detected (strict MIME)	Inspect takeda.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
boehringer-ingelheim.comBoehringer Ingelheim	Healthcare & pharma	Usable responseHTTP 200	Partial	None fully blocked	Unverified	Inspect boehringer-ingelheim.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
moderna.comModerna	Healthcare & pharma	No file recordedHTTP 404	Allowed	None fully blocked	Absent / non-manifest	Inspect moderna.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
biogen.comBiogen	Healthcare & pharma	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect biogen.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
regeneron.comRegeneron	Healthcare & pharma	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect regeneron.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
mayoclinic.orgMayo Clinic	Healthcare & pharma	Unresolved requestHTTP not recorded	Undetermined	Undetermined	Unverified	Inspect mayoclinic.org: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
clevelandclinic.orgCleveland Clinic	Healthcare & pharma	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect clevelandclinic.org: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
hopkinsmedicine.orgJohns Hopkins Medicine	Healthcare & pharma	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect hopkinsmedicine.org: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
kaiserpermanente.orgKaiser Permanente	Healthcare & pharma	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect kaiserpermanente.org: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
hcahealthcare.comHCA Healthcare	Healthcare & pharma	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect hcahealthcare.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
cvshealth.comCVS Health	Healthcare & pharma	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict rule)	Inspect cvshealth.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
unitedhealthgroup.comUnitedHealth Group	Healthcare & pharma	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect unitedhealthgroup.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
webmd.comWebMD	Healthcare & pharma	Usable responseHTTP 200	Blocked	None fully blocked	Absent / non-manifest	Inspect webmd.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
healthline.comHealthline	Healthcare & pharma	Usable responseHTTP 200	Blocked	None fully blocked	Absent / non-manifest	Inspect healthline.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
medicalnewstoday.comMedical News Today	Healthcare & pharma	Usable responseHTTP 200	Blocked	None fully blocked	Absent / non-manifest	Inspect medicalnewstoday.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
nhs.ukNHS	Healthcare & pharma	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect nhs.uk: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
mit.eduMassachusetts Institute of Technology	Education & research	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect mit.edu: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
stanford.eduStanford University	Education & research	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect stanford.edu: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
harvard.eduHarvard University	Education & research	Usable responseHTTP 200	Allowed	None fully blocked	Absent / non-manifest	Inspect harvard.edu: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
ox.ac.ukUniversity of Oxford	Education & research	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect ox.ac.uk: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
cam.ac.ukUniversity of Cambridge	Education & research	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect cam.ac.uk: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
caltech.eduCalifornia Institute of Technology	Education & research	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect caltech.edu: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
princeton.eduPrinceton University	Education & research	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect princeton.edu: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
yale.eduYale University	Education & research	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect yale.edu: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
columbia.eduColumbia University	Education & research	No file recordedHTTP 404	Allowed	None fully blocked	Absent / non-manifest	Inspect columbia.edu: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
uchicago.eduUniversity of Chicago	Education & research	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect uchicago.edu: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
imperial.ac.ukImperial College London	Education & research	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect imperial.ac.uk: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
ucl.ac.ukUniversity College London	Education & research	Usable responseHTTP 200	Partial	None fully blocked	Unverified	Inspect ucl.ac.uk: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
ethz.chETH Zurich	Education & research	No file recordedHTTP 404	Allowed	None fully blocked	Absent / non-manifest	Inspect ethz.ch: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
tsinghua.edu.cnTsinghua University	Education & research	No file recordedHTTP 404	Allowed	None fully blocked	Absent / non-manifest	Inspect tsinghua.edu.cn: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
utoronto.caUniversity of Toronto	Education & research	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect utoronto.ca: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
mcgill.caMcGill University	Education & research	Usable responseHTTP 200	Allowed	None fully blocked	Absent / non-manifest	Inspect mcgill.ca: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
nature.comNature	Education & research	Usable responseHTTP 200	Blocked	At least one blocked	Absent / non-manifest	Inspect nature.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
arxiv.orgarXiv	Education & research	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect arxiv.org: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
science.orgScience	Education & research	Usable responseHTTP 200	Blocked	None fully blocked	Unverified	Inspect science.org: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
plos.orgPLOS	Education & research	Usable responseHTTP 200	Allowed	None fully blocked	Absent / non-manifest	Inspect plos.org: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
springer.comSpringer Nature	Education & research	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect springer.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
elsevier.comElsevier	Education & research	Usable responseHTTP 200	Allowed	None fully blocked	Absent / non-manifest	Inspect elsevier.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
wiley.comWiley	Education & research	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect wiley.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
researchgate.netResearchGate	Education & research	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect researchgate.net: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
academia.eduAcademia.edu	Education & research	Usable responseHTTP 200	Blocked	None fully blocked	Absent / non-manifest	Inspect academia.edu: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
coursera.orgCoursera	Education & research	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect coursera.org: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
edx.orgedX	Education & research	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect edx.org: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
khanacademy.orgKhan Academy	Education & research	Invalid responseHTTP 200	Undetermined	Undetermined	Absent / non-manifest	Inspect khanacademy.org: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
mpg.deMax Planck Society	Education & research	Usable responseHTTP 200	Allowed	None fully blocked	Absent / non-manifest	Inspect mpg.de: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
cern.chCERN	Education & research	No file recordedHTTP 404	Allowed	None fully blocked	Absent / non-manifest	Inspect cern.ch: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
booking.comBooking.com	Travel, hospitality & airlines	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect booking.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
expedia.comExpedia	Travel, hospitality & airlines	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect expedia.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
airbnb.comAirbnb	Travel, hospitality & airlines	Usable responseHTTP 200	Blocked	None fully blocked	Absent / non-manifest	Inspect airbnb.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
marriott.comMarriott	Travel, hospitality & airlines	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect marriott.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
hilton.comHilton	Travel, hospitality & airlines	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect hilton.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
hyatt.comHyatt	Travel, hospitality & airlines	Unresolved requestHTTP not recorded	Undetermined	Undetermined	Unverified	Inspect hyatt.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
ihg.comIHG Hotels & Resorts	Travel, hospitality & airlines	Unresolved requestHTTP not recorded	Undetermined	Undetermined	Unverified	Inspect ihg.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
wyndhamhotels.comWyndham Hotels & Resorts	Travel, hospitality & airlines	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect wyndhamhotels.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
delta.comDelta Air Lines	Travel, hospitality & airlines	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect delta.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
united.comUnited Airlines	Travel, hospitality & airlines	Unresolved requestHTTP not recorded	Undetermined	Undetermined	Unverified	Inspect united.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
aa.comAmerican Airlines	Travel, hospitality & airlines	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect aa.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
southwest.comSouthwest Airlines	Travel, hospitality & airlines	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect southwest.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
jetblue.comJetBlue	Travel, hospitality & airlines	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect jetblue.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
alaskaair.comAlaska Airlines	Travel, hospitality & airlines	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect alaskaair.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
britishairways.comBritish Airways	Travel, hospitality & airlines	Invalid responseHTTP 200	Undetermined	Undetermined	Absent / non-manifest	Inspect britishairways.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
lufthansa.comLufthansa	Travel, hospitality & airlines	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect lufthansa.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
airfrance.comAir France	Travel, hospitality & airlines	Unresolved requestHTTP not recorded	Undetermined	Undetermined	Unverified	Inspect airfrance.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
klm.comKLM	Travel, hospitality & airlines	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect klm.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
ryanair.comRyanair	Travel, hospitality & airlines	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect ryanair.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
easyjet.comeasyJet	Travel, hospitality & airlines	Unresolved requestHTTP not recorded	Undetermined	Undetermined	Unverified	Inspect easyjet.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
virginatlantic.comVirgin Atlantic	Travel, hospitality & airlines	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect virginatlantic.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
qantas.comQantas	Travel, hospitality & airlines	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect qantas.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
singaporeair.comSingapore Airlines	Travel, hospitality & airlines	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect singaporeair.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
cathaypacific.comCathay Pacific	Travel, hospitality & airlines	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect cathaypacific.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
ana.co.jpANA – All Nippon Airways	Travel, hospitality & airlines	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect ana.co.jp: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
jal.co.jpJapan Airlines	Travel, hospitality & airlines	Unresolved requestHTTP not recorded	Undetermined	Undetermined	Unverified	Inspect jal.co.jp: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
trip.comTrip.com	Travel, hospitality & airlines	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect trip.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
agoda.comAgoda	Travel, hospitality & airlines	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect agoda.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
hotels.comHotels.com	Travel, hospitality & airlines	Usable responseHTTP 200	Partial	None fully blocked	Unverified	Inspect hotels.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
vrbo.comVrbo	Travel, hospitality & airlines	Usable responseHTTP 200	Partial	None fully blocked	Unverified	Inspect vrbo.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
searchengineland.comSearch Engine Land	Marketing, SEO & media industry	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect searchengineland.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
searchenginejournal.comSearch Engine Journal	Marketing, SEO & media industry	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect searchenginejournal.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
moz.comMoz	Marketing, SEO & media industry	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect moz.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
ahrefs.comAhrefs	Marketing, SEO & media industry	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect ahrefs.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
semrush.comSemrush	Marketing, SEO & media industry	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect semrush.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
hubspot.comHubSpot	Marketing, SEO & media industry	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect hubspot.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
neilpatel.comNeil Patel	Marketing, SEO & media industry	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect neilpatel.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
backlinko.comBacklinko	Marketing, SEO & media industry	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect backlinko.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
contentmarketinginstitute.comContent Marketing Institute	Marketing, SEO & media industry	Usable responseHTTP 200	Blocked	None fully blocked	Absent / non-manifest	Inspect contentmarketinginstitute.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
marketingprofs.comMarketingProfs	Marketing, SEO & media industry	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect marketingprofs.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
socialmediaexaminer.comSocial Media Examiner	Marketing, SEO & media industry	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect socialmediaexaminer.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
buffer.comBuffer	Marketing, SEO & media industry	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect buffer.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
hootsuite.comHootsuite	Marketing, SEO & media industry	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect hootsuite.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
sproutsocial.comSprout Social	Marketing, SEO & media industry	Usable responseHTTP 200	Allowed	None fully blocked	Detected (strict MIME)	Inspect sproutsocial.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
copyblogger.comCopyblogger	Marketing, SEO & media industry	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect copyblogger.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
convinceandconvert.comConvince & Convert	Marketing, SEO & media industry	Invalid responseHTTP 200	Undetermined	Undetermined	Absent / non-manifest	Inspect convinceandconvert.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
adweek.comAdweek	Marketing, SEO & media industry	Usable responseHTTP 200	Allowed	None fully blocked	Absent / non-manifest	Inspect adweek.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
adage.comAd Age	Marketing, SEO & media industry	Unresolved requestHTTP not recorded	Undetermined	Undetermined	Unverified	Inspect adage.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
thedrum.comThe Drum	Marketing, SEO & media industry	Usable responseHTTP 200	Blocked	None fully blocked	Absent / non-manifest	Inspect thedrum.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
econsultancy.comEconsultancy	Marketing, SEO & media industry	Usable responseHTTP 200	Blocked	None fully blocked	Absent / non-manifest	Inspect econsultancy.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
smartinsights.comSmart Insights	Marketing, SEO & media industry	Usable responseHTTP 200	Allowed	None fully blocked	Absent / non-manifest	Inspect smartinsights.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
digitalmarketer.comDigitalMarketer	Marketing, SEO & media industry	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect digitalmarketer.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
cxl.comCXL	Marketing, SEO & media industry	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect cxl.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
unbounce.comUnbounce	Marketing, SEO & media industry	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect unbounce.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
leadpages.comLeadpages	Marketing, SEO & media industry	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect leadpages.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
optinmonster.comOptinMonster	Marketing, SEO & media industry	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect optinmonster.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
yoast.comYoast	Marketing, SEO & media industry	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect yoast.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
screamingfrog.co.ukScreaming Frog	Marketing, SEO & media industry	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect screamingfrog.co.uk: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
brightedge.comBrightEdge	Marketing, SEO & media industry	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect brightedge.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
conductor.comConductor	Marketing, SEO & media industry	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect conductor.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
similarweb.comSimilarweb	Marketing, SEO & media industry	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect similarweb.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
hotjar.comHotjar	Marketing, SEO & media industry	Usable responseHTTP 200	Allowed	None fully blocked	Absent / non-manifest	Inspect hotjar.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
crazyegg.comCrazy Egg	Marketing, SEO & media industry	Usable responseHTTP 200	Allowed	None fully blocked	Detected (strict MIME)	Inspect crazyegg.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
vwo.comVWO	Marketing, SEO & media industry	Usable responseHTTP 200	Allowed	None fully blocked	Detected (strict MIME)	Inspect vwo.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
gtmetrix.comGTmetrix	Marketing, SEO & media industry	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect gtmetrix.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
wordstream.comWordStream	Marketing, SEO & media industry	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect wordstream.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
spyfu.comSpyFu	Marketing, SEO & media industry	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect spyfu.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
mangools.comMangools	Marketing, SEO & media industry	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect mangools.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
buzzsumo.comBuzzSumo	Marketing, SEO & media industry	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect buzzsumo.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
growthhackers.comGrowthHackers	Marketing, SEO & media industry	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect growthhackers.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
dawn.comDawn	Pakistan top sites	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect dawn.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
geo.tvGeo TV	Pakistan top sites	Usable responseHTTP 200	Allowed	None fully blocked	Absent / non-manifest	Inspect geo.tv: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
arynews.tvARY News	Pakistan top sites	Usable responseHTTP 200	Blocked	None fully blocked	Detected (strict MIME)	Inspect arynews.tv: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
tribune.com.pkThe Express Tribune	Pakistan top sites	Usable responseHTTP 200	Allowed	None fully blocked	Absent / non-manifest	Inspect tribune.com.pk: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
thenews.com.pkThe News International	Pakistan top sites	Usable responseHTTP 200	Allowed	None fully blocked	Absent / non-manifest	Inspect thenews.com.pk: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
dunyanews.tvDunya News	Pakistan top sites	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect dunyanews.tv: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
samaa.tvSamaa TV	Pakistan top sites	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect samaa.tv: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
aaj.tvAaj TV	Pakistan top sites	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect aaj.tv: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
jang.com.pkJang	Pakistan top sites	Usable responseHTTP 200	Allowed	None fully blocked	Absent / non-manifest	Inspect jang.com.pk: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
brecorder.comBusiness Recorder	Pakistan top sites	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect brecorder.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
pakistantoday.com.pkPakistan Today	Pakistan top sites	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect pakistantoday.com.pk: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
urdupoint.comUrduPoint	Pakistan top sites	Usable responseHTTP 200	Blocked	None fully blocked	Absent / non-manifest	Inspect urdupoint.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
hbl.comHBL	Pakistan top sites	Usable responseHTTP 200	Allowed	None fully blocked	Absent / non-manifest	Inspect hbl.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
meezanbank.comMeezan Bank	Pakistan top sites	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect meezanbank.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
ubl.com.pkUnited Bank Limited	Pakistan top sites	Unresolved requestHTTP not recorded	Undetermined	Undetermined	Unverified	Inspect ubl.com.pk: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
mcb.com.pkMCB Bank	Pakistan top sites	Usable responseHTTP 200	Allowed	None fully blocked	Absent / non-manifest	Inspect mcb.com.pk: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
nbp.com.pkNational Bank of Pakistan	Pakistan top sites	No file recordedHTTP 404	Allowed	None fully blocked	Absent / non-manifest	Inspect nbp.com.pk: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
bankalfalah.comBank Alfalah	Pakistan top sites	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect bankalfalah.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
abl.comAllied Bank	Pakistan top sites	Usable responseHTTP 200	Allowed	None fully blocked	Detected (strict MIME)	Inspect abl.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
askaribank.comAskari Bank	Pakistan top sites	Usable responseHTTP 200	Partial	None fully blocked	Unverified	Inspect askaribank.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
habibmetro.comHabib Metropolitan Bank	Pakistan top sites	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect habibmetro.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
faysalbank.comFaysal Bank	Pakistan top sites	Unresolved requestHTTP not recorded	Undetermined	Undetermined	Unverified	Inspect faysalbank.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
jazz.com.pkJazz	Pakistan top sites	Usable responseHTTP 200	Blocked	None fully blocked	Absent / non-manifest	Inspect jazz.com.pk: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
telenor.com.pkTelenor Pakistan	Pakistan top sites	Unresolved requestHTTP not recorded	Undetermined	Undetermined	Unverified	Inspect telenor.com.pk: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
zong.com.pkZong	Pakistan top sites	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect zong.com.pk: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
ptcl.com.pkPTCL	Pakistan top sites	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect ptcl.com.pk: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
ufone.comUfone	Pakistan top sites	No file recordedHTTP 404	Allowed	None fully blocked	Absent / non-manifest	Inspect ufone.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
daraz.pkDaraz Pakistan	Pakistan top sites	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect daraz.pk: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
foodpanda.pkfoodpanda Pakistan	Pakistan top sites	Usable responseHTTP 200	Blocked	None fully blocked	Absent / non-manifest	Inspect foodpanda.pk: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
olx.com.pkOLX Pakistan	Pakistan top sites	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect olx.com.pk: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
zameen.comZameen.com	Pakistan top sites	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect zameen.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
pakwheels.comPakWheels	Pakistan top sites	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect pakwheels.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
rozee.pkROZEE.PK	Pakistan top sites	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect rozee.pk: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
priceoye.pkPriceOye	Pakistan top sites	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect priceoye.pk: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
nu.edu.pkFAST National University	Pakistan top sites	No file recordedHTTP 404	Allowed	None fully blocked	Absent / non-manifest	Inspect nu.edu.pk: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
lums.edu.pkLUMS	Pakistan top sites	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect lums.edu.pk: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
comsats.edu.pkCOMSATS University Islamabad	Pakistan top sites	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect comsats.edu.pk: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
pu.edu.pkUniversity of the Punjab	Pakistan top sites	No file recordedHTTP 404	Allowed	None fully blocked	Absent / non-manifest	Inspect pu.edu.pk: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
nust.edu.pkNUST	Pakistan top sites	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect nust.edu.pk: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
uok.edu.pkUniversity of Karachi	Pakistan top sites	No file recordedHTTP 404	Allowed	None fully blocked	Absent / non-manifest	Inspect uok.edu.pk: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
qau.edu.pkQuaid-i-Azam University	Pakistan top sites	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect qau.edu.pk: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
vu.edu.pkVirtual University of Pakistan	Pakistan top sites	No file recordedHTTP 404	Allowed	None fully blocked	Absent / non-manifest	Inspect vu.edu.pk: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
aiou.edu.pkAllama Iqbal Open University	Pakistan top sites	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect aiou.edu.pk: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
aku.eduAga Khan University	Pakistan top sites	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect aku.edu: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
pakistan.gov.pkGovernment of Pakistan	Pakistan top sites	Usable responseHTTP 200	Allowed	None fully blocked	Absent / non-manifest	Inspect pakistan.gov.pk: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
fbr.gov.pkFederal Board of Revenue	Pakistan top sites	No file recordedHTTP 404	Allowed	None fully blocked	Absent / non-manifest	Inspect fbr.gov.pk: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
sbp.org.pkState Bank of Pakistan	Pakistan top sites	Unresolved requestHTTP not recorded	Undetermined	Undetermined	Unverified	Inspect sbp.org.pk: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
nadra.gov.pkNADRA	Pakistan top sites	Unresolved requestHTTP not recorded	Undetermined	Undetermined	Unverified	Inspect nadra.gov.pk: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
hec.gov.pkHigher Education Commission Pakistan	Pakistan top sites	No file recordedHTTP 404	Allowed	None fully blocked	Absent / non-manifest	Inspect hec.gov.pk: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
punjab.gov.pkGovernment of Punjab	Pakistan top sites	Unresolved requestHTTP not recorded	Undetermined	Undetermined	Absent / non-manifest	Inspect punjab.gov.pk: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
sindh.gov.pkGovernment of Sindh	Pakistan top sites	No file recordedHTTP 404	Allowed	None fully blocked	Absent / non-manifest	Inspect sindh.gov.pk: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
kp.gov.pkGovernment of Khyber Pakhtunkhwa	Pakistan top sites	Invalid responseHTTP 200	Undetermined	Undetermined	Absent / non-manifest	Inspect kp.gov.pk: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
balochistan.gov.pkGovernment of Balochistan	Pakistan top sites	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect balochistan.gov.pk: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
ecp.gov.pkElection Commission of Pakistan	Pakistan top sites	Usable responseHTTP 200	Allowed	None fully blocked	Absent / non-manifest	Inspect ecp.gov.pk: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
propakistani.pkProPakistani	Pakistan top sites	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect propakistani.pk: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
techjuice.pkTechJuice	Pakistan top sites	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect techjuice.pk: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
hamariweb.comHamariweb	Pakistan top sites	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect hamariweb.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
whatmobile.com.pkWhatMobile	Pakistan top sites	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect whatmobile.com.pk: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
dailypakistan.com.pkDaily Pakistan	Pakistan top sites	Usable responseHTTP 200	Allowed	None fully blocked	Absent / non-manifest	Inspect dailypakistan.com.pk: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
pakobserver.netPakistan Observer	Pakistan top sites	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect pakobserver.net: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
gulfnews.comGulf News	Gulf/MENA	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect gulfnews.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
khaleejtimes.comKhaleej Times	Gulf/MENA	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect khaleejtimes.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
thenationalnews.comThe National	Gulf/MENA	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect thenationalnews.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
emirates.comEmirates	Gulf/MENA	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect emirates.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
etihad.comEtihad Airways	Gulf/MENA	Unresolved requestHTTP not recorded	Undetermined	Undetermined	Unverified	Inspect etihad.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
noon.comNoon	Gulf/MENA	Unresolved requestHTTP not recorded	Undetermined	Undetermined	Unverified	Inspect noon.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
namshi.comNamshi	Gulf/MENA	Unresolved requestHTTP not recorded	Undetermined	Undetermined	Unverified	Inspect namshi.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
emiratesnbd.comEmirates NBD	Gulf/MENA	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect emiratesnbd.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
adcb.comAbu Dhabi Commercial Bank	Gulf/MENA	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect adcb.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
bankfab.comFirst Abu Dhabi Bank	Gulf/MENA	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect bankfab.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
eand.come&	Gulf/MENA	Invalid responseHTTP 200	Undetermined	Undetermined	Absent / non-manifest	Inspect eand.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
du.aedu	Gulf/MENA	Usable responseHTTP 200	Blocked	At least one blocked	Absent / non-manifest	Inspect du.ae: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
u.aeUAE Government Portal	Gulf/MENA	No file recordedHTTP 404	Allowed	None fully blocked	Absent / non-manifest	Inspect u.ae: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
bayut.comBayut	Gulf/MENA	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect bayut.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
propertyfinder.aeProperty Finder	Gulf/MENA	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect propertyfinder.ae: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
dubizzle.comDubizzle	Gulf/MENA	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect dubizzle.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
arabnews.comArab News	Gulf/MENA	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict rule)	Inspect arabnews.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
stc.com.sastc	Gulf/MENA	Usable responseHTTP 200	Allowed	None fully blocked	Absent / non-manifest	Inspect stc.com.sa: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
my.gov.saSaudi National Portal	Gulf/MENA	Unresolved requestHTTP not recorded	Undetermined	Undetermined	Unverified	Inspect my.gov.sa: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
alrajhibank.com.saAl Rajhi Bank	Gulf/MENA	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect alrajhibank.com.sa: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
saudia.comSaudia	Gulf/MENA	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect saudia.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
almarai.comAlmarai	Gulf/MENA	Usable responseHTTP 200	Allowed	None fully blocked	Detected (strict MIME)	Inspect almarai.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
jarir.comJarir Bookstore	Gulf/MENA	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect jarir.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
aljazeera.comAl Jazeera	Gulf/MENA	Usable responseHTTP 200	Blocked	At least one blocked	Absent / non-manifest	Inspect aljazeera.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
qatarairways.comQatar Airways	Gulf/MENA	Unresolved requestHTTP not recorded	Undetermined	Undetermined	Unverified	Inspect qatarairways.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
qnb.comQNB	Gulf/MENA	Usable responseHTTP 200	Allowed	None fully blocked	Absent / non-manifest	Inspect qnb.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
ooredoo.comOoredoo	Gulf/MENA	Usable responseHTTP 200	Allowed	None fully blocked	Absent / non-manifest	Inspect ooredoo.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
youm7.comYoum7	Gulf/MENA	Usable responseHTTP 200	Allowed	None fully blocked	Detected (strict MIME)	Inspect youm7.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
masrawy.comMasrawy	Gulf/MENA	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect masrawy.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
ahram.org.egAl-Ahram	Gulf/MENA	Usable responseHTTP 200	Allowed	None fully blocked	Absent / non-manifest	Inspect ahram.org.eg: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
medium.comMedium	Content platforms & communities	Usable responseHTTP 200	Blocked	None fully blocked	Unverified	Inspect medium.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
substack.comSubstack	Content platforms & communities	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect substack.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
reddit.comReddit	Content platforms & communities	Usable responseHTTP 200	Blocked	At least one blocked	Absent / non-manifest	Inspect reddit.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
quora.comQuora	Content platforms & communities	Usable responseHTTP 200	Blocked	At least one blocked	Absent / non-manifest	Inspect quora.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
wikipedia.orgWikipedia	Content platforms & communities	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect wikipedia.org: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
stackoverflow.comStack Overflow	Content platforms & communities	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect stackoverflow.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
github.comGitHub	Content platforms & communities	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect github.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
wordpress.comWordPress.com	Content platforms & communities	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect wordpress.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
blogger.comBlogger	Content platforms & communities	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect blogger.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
tumblr.comTumblr	Content platforms & communities	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect tumblr.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
youtube.comYouTube	Content platforms & communities	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect youtube.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
vimeo.comVimeo	Content platforms & communities	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect vimeo.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
twitch.tvTwitch	Content platforms & communities	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect twitch.tv: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
discord.comDiscord	Content platforms & communities	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect discord.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
pinterest.comPinterest	Content platforms & communities	Usable responseHTTP 200	Blocked	At least one blocked	Absent / non-manifest	Inspect pinterest.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
facebook.comFacebook	Content platforms & communities	Usable responseHTTP 200	Blocked	At least one blocked	Absent / non-manifest	Inspect facebook.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
instagram.comInstagram	Content platforms & communities	Usable responseHTTP 200	Blocked	At least one blocked	Absent / non-manifest	Inspect instagram.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
x.comX	Content platforms & communities	Usable responseHTTP 200	Blocked	At least one blocked	Absent / non-manifest	Inspect x.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
linkedin.comLinkedIn	Content platforms & communities	Usable responseHTTP 200	Blocked	At least one blocked	Absent / non-manifest	Inspect linkedin.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
tiktok.comTikTok	Content platforms & communities	Usable responseHTTP 200	Blocked	At least one blocked	Absent / non-manifest	Inspect tiktok.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
snapchat.comSnapchat	Content platforms & communities	Usable responseHTTP 200	Blocked	None fully blocked	Absent / non-manifest	Inspect snapchat.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
patreon.comPatreon	Content platforms & communities	Usable responseHTTP 200	Partial	At least one blocked	Absent / non-manifest	Inspect patreon.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
kickstarter.comKickstarter	Content platforms & communities	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect kickstarter.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
goodreads.comGoodreads	Content platforms & communities	Usable responseHTTP 200	Blocked	None fully blocked	Absent / non-manifest	Inspect goodreads.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
soundcloud.comSoundCloud	Content platforms & communities	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect soundcloud.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
bandcamp.comBandcamp	Content platforms & communities	Usable responseHTTP 200	Blocked	None fully blocked	Absent / non-manifest	Inspect bandcamp.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
flickr.comFlickr	Content platforms & communities	Usable responseHTTP 200	Blocked	None fully blocked	Absent / non-manifest	Inspect flickr.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
deviantart.comDeviantArt	Content platforms & communities	Usable responseHTTP 200	Partial	None fully blocked	Unverified	Inspect deviantart.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
behance.netBehance	Content platforms & communities	Usable responseHTTP 200	Blocked	None fully blocked	Absent / non-manifest	Inspect behance.net: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
dribbble.comDribbble	Content platforms & communities	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect dribbble.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
stackexchange.comStack Exchange	Content platforms & communities	Usable responseHTTP 200	Partial	At least one blocked	Absent / non-manifest	Inspect stackexchange.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
archive.orgInternet Archive	Content platforms & communities	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect archive.org: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
fandom.comFandom	Content platforms & communities	Usable responseHTTP 200	Blocked	At least one blocked	Absent / non-manifest	Inspect fandom.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
wikihow.comwikiHow	Content platforms & communities	Usable responseHTTP 200	Blocked	At least one blocked	Absent / non-manifest	Inspect wikihow.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
instructables.comInstructables	Content platforms & communities	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect instructables.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
kaggle.comKaggle	Content platforms & communities	No file recordedHTTP 404	Allowed	None fully blocked	Absent / non-manifest	Inspect kaggle.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
itch.ioitch.io	Content platforms & communities	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect itch.io: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
letterboxd.comLetterboxd	Content platforms & communities	Usable responseHTTP 200	Blocked	None fully blocked	Absent / non-manifest	Inspect letterboxd.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
last.fmLast.fm	Content platforms & communities	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect last.fm: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
dev.toDEV Community	Content platforms & communities	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect dev.to: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
openai.comOpenAI	AI companies	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect openai.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
anthropic.comAnthropic	AI companies	Usable responseHTTP 200	Allowed	None fully blocked	Absent / non-manifest	Inspect anthropic.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
perplexity.aiPerplexity	AI companies	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect perplexity.ai: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
mistral.aiMistral AI	AI companies	Usable responseHTTP 200	Allowed	None fully blocked	Detected (strict MIME)	Inspect mistral.ai: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
huggingface.coHugging Face	AI companies	Usable responseHTTP 200	Allowed	None fully blocked	Absent / non-manifest	Inspect huggingface.co: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
cohere.comCohere	AI companies	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect cohere.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
deepmind.comGoogle DeepMind	AI companies	Usable responseHTTP 200	Allowed	None fully blocked	Detected (strict MIME)	Inspect deepmind.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
stability.aiStability AI	AI companies	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect stability.ai: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
midjourney.comMidjourney	AI companies	Usable responseHTTP 200	Blocked	None fully blocked	Absent / non-manifest	Inspect midjourney.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
runwayml.comRunway	AI companies	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect runwayml.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
character.aiCharacter.AI	AI companies	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect character.ai: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
x.aixAI	AI companies	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect x.ai: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
aleph-alpha.comAleph Alpha	AI companies	Usable responseHTTP 200	Allowed	None fully blocked	Absent / non-manifest	Inspect aleph-alpha.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
replicate.comReplicate	AI companies	Usable responseHTTP 200	Allowed	None fully blocked	Detected (strict MIME)	Inspect replicate.com: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
elevenlabs.ioElevenLabs	AI companies	Usable responseHTTP 200	Partial	None fully blocked	Detected (strict MIME)	Inspect elevenlabs.io: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
gov.ukGOV.UK	Government & public sector (global)	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect gov.uk: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
usa.govUSA.gov	Government & public sector (global)	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect usa.gov: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
canada.caGovernment of Canada	Government & public sector (global)	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect canada.ca: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
australia.gov.auAustralian Government	Government & public sector (global)	Invalid responseHTTP 200	Undetermined	Undetermined	Absent / non-manifest	Inspect australia.gov.au: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
europa.euEuropean Union	Government & public sector (global)	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect europa.eu: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
who.intWorld Health Organization	Government & public sector (global)	Usable responseHTTP 200	Allowed	None fully blocked	Absent / non-manifest	Inspect who.int: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
un.orgUnited Nations	Government & public sector (global)	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect un.org: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
nasa.govNASA	Government & public sector (global)	Usable responseHTTP 200	Allowed	None fully blocked	Absent / non-manifest	Inspect nasa.gov: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
data.govData.gov	Government & public sector (global)	Usable responseHTTP 200	Allowed	None fully blocked	Absent / non-manifest	Inspect data.gov: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
whitehouse.govThe White House	Government & public sector (global)	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect whitehouse.gov: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
service-public.frService-Public.fr	Government & public sector (global)	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect service-public.fr: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
india.gov.inNational Portal of India	Government & public sector (global)	Usable responseHTTP 200	Allowed	None fully blocked	Absent / non-manifest	Inspect india.gov.in: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
gov.brGov.br	Government & public sector (global)	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect gov.br: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
gov.zaSouth African Government	Government & public sector (global)	Usable responseHTTP 200	Partial	None fully blocked	Unverified	Inspect gov.za: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.
gov.sgSingapore Government	Government & public sector (global)	Usable responseHTTP 200	Partial	None fully blocked	Absent / non-manifest	Inspect gov.sg: all 22 tokens and evidence references Enable JavaScript for all 22 classifications and evidence details. The complete register is also supplied as CSV and JSON.

llms.txt: 108 strict detections, one MIME mismatch, one nonstandard-format document, 335 absent/non-manifest outcomes and 55 unverified. Both baseline and follow-up provenance are retained. An archived response can exist even when its content does not qualify as an llms.txt manifest.

Frequently asked questions

Does blocking GPTBot stop my website from appearing in ChatGPT Search?

Not automatically. OpenAI documents GPTBot as training-related and OAI-SearchBot as its search-related crawler. These controls can be configured independently. ChatGPT Search also has other pathways for navigational references, so robots.txt should not be described as a universal brand-mention switch.

Does `Disallow: /` block all AI systems?

It is a full root-path disallow directive for a cooperating user-agent group under the applicable robots rules. It does not enforce network access, retroactively erase training data, override every user-directed fetch, or guarantee removal of third-party mentions. Rules targeting specific user agents differ from wildcard rules, and Applebot has an important provider-specific fallback.

What does `Partial` mean in this dataset?

Partial means the root path was not blocked but at least one path restriction existed in the applicable group. This is not counted as a full-site AI block. It also does not guarantee that a commercially important product or research page is accessible, since that page might lie under the restricted path.

Why do the baseline and follow-up use different denominators?

All 500 websites were attempted. The frozen baseline classified 448 robots policies: 433 usable bodies and 15 no-file responses. Its other 52 outcomes were 44 unresolved requests and eight invalid bodies. The follow-up resolved eight of those cases, bringing the reconciled register to 456 determined policies and 44 unknowns. Baseline findings and Figures 1–5 use 448; follow-up percentages use 456. Each table identifies its evidence view.

Are the news publisher numbers representative of every news website?

No. The news bucket contains 59 determined sites from a deliberately curated list. Its 41/59 result describes only that sample and observation window. It is not a statistically justified estimate for the entire news industry.

Is `/llms.txt` required to rank in Google AI Overviews or AI Mode?

No. Google states it does not use /llms.txt for Google Search. Its absence does not prevent inclusion in Google’s generative search features. Publishing one may serve other consumers, but this study did not measure such usage or benefits.

Two under the final provider-aware parser: USA Today and Reddit. A generic exact-token parser would have counted four, but Apple’s documented Googlebot fallback changes the effective interpretation for X and du. The result is based on the archived files supplied with this release.

Can a site’s firewall change the real access outcome?

Yes. A site can permit a bot in robots.txt yet block or challenge a request at the CDN, WAF, rate limiter, or origin. This research did not probe verified vendor crawlers from their real network ranges. Treat policy parsing and actual fetch diagnostics as separate investigations.

Does a file containing many URLs qualify as a high-quality `/llms.txt` manifest?

Not necessarily. The audit distinguishes 90 files with strict link patterns from 99 containing any absolute HTTP(S) URL. It did not test destinations, freshness, usefulness, or whether answer engines consumed those documents.

Sources and evidence classification

Source-level empirical evidence: Archived robots.txt and /llms.txt responses, the 500-domain source frame, generated per-token verdict CSV, indexed sizes/checksums, and analysis scripts in the downloadable study package. These establish a reproducible snapshot of the archived material, not verified bot behavior or population-level effects.

Primary technical standards and operator documentation: RFC 9309; OpenAI’s bots; Anthropic’s crawler controls; Perplexity’s crawlers; Google common crawlers and Google-Extended; Google’s generative AI optimization guidance; Google AI features; Applebot and Applebot-Extended; Common Crawl CCBot; Cloudflare Bot Fight Mode; and the proposed /llms.txt convention.

Interpretation, not verified by this study: Reasons organizations publish particular policies, commercial effects of restrictions, AI answer-engine citation rates, crawler compliance, WAF challenges, content rendering behavior, and the effectiveness of /llms.txt.

About the author

Asfand Yar Junejo researches technical search visibility, crawler access, and how brands become discoverable and attributable in AI-assisted search. Further background: About Asfand Yar Junejo and Services.

Version and freshness: November 2026 edition · baseline v3.1.2 with v3.2 verification supplement. This edition preserves a frozen baseline and a separate follow-up view; capture dates are documented in the accompanying files. Current website policies may differ.

Suggested citation: Junejo, A. Y. (2026). AI-Crawler Access Audit 2026: What 500 Websites Reveal About AI Training, Search Crawlers, and robots.txt Policies (November 2026 edition, baseline v3.1.2 with v3.2 verification supplement). asfandjunejo.com. https://asfandjunejo.com/lab/ai-crawler-access-audit-2026/

Rights, reuse, and source provenance

Original research by Asfand Yar Junejo. Cite this edition and version; consult the evidence archive for capture provenance. Archived third-party texts remain attributable to their source publishers; this release grants no license to train on or commercially republish those texts. The archive is supplied for evidence inspection, subject to applicable rights and law. Contact asfand@asfandjunejo.com for corrections or rights concerns.

Hashes demonstrate consistency between stored files and their indexes. They do not prove an exact server response time. Early captures lack full HTTP headers, exact times, and complete redirect chains. Five usable robots responses ended on other hostnames; those redirects were retained with this limitation.