September 26, 2026 retrieval of the July 26 S&P 500 cohort. Published rules do not prove actual access or visibility. Search by company or policy; inspect JSON for exact rule lines and hashes.
500 / 500
| Company | Retrieval | OAI-SearchBotsearch | Claude-SearchBotsearch | PerplexityBotsearch | GPTBottraining | ClaudeBottraining | ChatGPT-Useruser-directed | Claude-Useruser-directed | Perplexity-Useruser-directed |
|---|---|---|---|---|---|---|---|---|---|
| 3Mrobots.txt ↗ | inaccessible | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| A. O. Smithrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Abbott Laboratoriesrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| AbbVierobots.txt ↗ | inaccessible | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Accenturerobots.txt ↗ | readable200 | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard |
| Adobe Inc.robots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Advanced Micro Devicesrobots.txt ↗ | inaccessible | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| AES Corporationrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Aflacrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Agilent Technologiesrobots.txt ↗ | readable200 | ambiguousspecific | ambiguouswildcard | ambiguousspecific | ambiguousspecific | ambiguousspecific | ambiguousspecific | ambiguouswildcard | ambiguouswildcard |
| Air Productsrobots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Airbnbrobots.txt ↗ | readable200 | path_restrictionsspecific | path_restrictionswildcard | path_restrictionsspecific | blanket_disallowspecific | blanket_disallowspecific | path_restrictionsspecific | path_restrictionswildcard | path_restrictionswildcard |
| Akamai Technologiesrobots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Albemarle Corporationrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Alexandria Real Estate Equitiesrobots.txt ↗ | ambiguous_content200 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Align Technologyrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Allegionrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Alliant Energyrobots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Allstaterobots.txt ↗ | inaccessible | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Alphabet Inc. (Class A)robots.txt ↗ | readable200 | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard |
| Altriarobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Amazonrobots.txt ↗ | readable200 | ambiguousspecific | ambiguousspecific | ambiguousspecific | ambiguousspecific | ambiguousspecific | ambiguousspecific | ambiguousspecific | ambiguousspecific |
| Amcorrobots.txt ↗ | readable200 | ambiguouswildcard | ambiguouswildcard | ambiguousspecific | ambiguousspecific | ambiguousspecific | ambiguousspecific | ambiguouswildcard | ambiguouswildcard |
| Amerenrobots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| American Electric Powerrobots.txt ↗ | missing_file404 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| American Expressrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| American International Grouprobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| American Towerrobots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| American Water Worksrobots.txt ↗ | readable200 | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguousspecific | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard |
| Ameriprise Financialrobots.txt ↗ | readable200 | no_published_restrictionnone | no_published_restrictionnone | no_published_restrictionnone | no_published_restrictionnone | no_published_restrictionnone | no_published_restrictionnone | no_published_restrictionnone | no_published_restrictionnone |
| Ametekrobots.txt ↗ | readable200 | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard |
| Amgenrobots.txt ↗ | readable200 | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard |
| Amphenolrobots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Analog Devicesrobots.txt ↗ | inaccessible | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Aon plcrobots.txt ↗ | readable200 | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard |
| APA Corporationrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Apollo Global Managementrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Apple Inc.robots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Applied Materialsrobots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| AppLovinrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Aptivrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Arch Capital Grouprobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Archer Daniels Midlandrobots.txt ↗ | readable200 | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard |
| Ares Managementrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Arista Networksrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Arthur J. Gallagher & Co.robots.txt ↗ | ambiguous_content200 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Assurantrobots.txt ↗ | readable200 | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard |
| AT&Trobots.txt ↗ | readable200 | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard |
| Atmos Energyrobots.txt ↗ | missing_file404 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Autodeskrobots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Automatic Data Processingrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| AutoZonerobots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| AvalonBay Communitiesrobots.txt ↗ | readable200 | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard |
| Avery Dennisonrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Axon Enterpriserobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Baker Hughesrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Ball Corporationrobots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Bank of Americarobots.txt ↗ | readable200 | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard |
| Baxter Internationalrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Becton Dickinsonrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Berkshire Hathawayrobots.txt ↗ | missing_file404 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Best Buyrobots.txt ↗ | readable200 | ambiguousspecific | ambiguouswildcard | ambiguousspecific | ambiguousspecific | ambiguousspecific | ambiguousspecific | ambiguousspecific | ambiguousspecific |
| Bio-Technerobots.txt ↗ | inaccessible | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Biogenrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| BlackRockrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Blackstone Inc.robots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Block, Inc.robots.txt ↗ | readable200 | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard |
| BNY Mellonrobots.txt ↗ | missing_file404 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Boeingrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Booking Holdingsrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Boston Scientificrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Bristol Myers Squibbrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Broadcomrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Broadridge Financial Solutionsrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Brown & Brownrobots.txt ↗ | missing_file404 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Brown–Formanrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Builders FirstSourcerobots.txt ↗ | readable200 | no_published_restrictionspecific | no_published_restrictionspecific | no_published_restrictionspecific | no_published_restrictionspecific | no_published_restrictionspecific | no_published_restrictionspecific | no_published_restrictionspecific | no_published_restrictionspecific |
| Bunge Globalrobots.txt ↗ | missing_file404 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| BXP, Inc.robots.txt ↗ | readable200 | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard |
| C.H. Robinsonrobots.txt ↗ | readable200 | ambiguousspecific | ambiguousspecific | ambiguousspecific | ambiguousspecific | ambiguousspecific | ambiguousspecific | ambiguousspecific | ambiguousspecific |
| Cadence Design Systemsrobots.txt ↗ | readable200 | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard |
| Camden Property Trustrobots.txt ↗ | readable200 | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard |
| Capital Onerobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Cardinal Healthrobots.txt ↗ | inaccessible | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Carnival Corporationrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Carrier Globalrobots.txt ↗ | readable200 | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard |
| Carvanarobots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Casey'srobots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Caterpillar Inc.robots.txt ↗ | inaccessible | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Cboe Global Marketsrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| CBRE Grouprobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| CDW Corporationrobots.txt ↗ | readable200 | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard |
| Cencorarobots.txt ↗ | readable200 | path_restrictionsspecific | path_restrictionsspecific | path_restrictionsspecific | path_restrictionsspecific | path_restrictionsspecific | path_restrictionsspecific | path_restrictionsspecific | path_restrictionsspecific |
| Centene Corporationrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| CenterPoint Energyrobots.txt ↗ | readable200 | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard |
| CF Industriesrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Charles River Laboratoriesrobots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Charles Schwab Corporationrobots.txt ↗ | ambiguous_content200 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Charter Communicationsrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Chevron Corporationrobots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Chipotle Mexican Grillrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Chubb Limitedrobots.txt ↗ | readable200 | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard |
| Church & Dwightrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Cienarobots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Cignarobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Cincinnati Financialrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Cintasrobots.txt ↗ | readable200 | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard |
| Ciscorobots.txt ↗ | readable200 | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguousspecific | ambiguousspecific | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard |
| Citigrouprobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Citizens Financial Grouprobots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Cloroxrobots.txt ↗ | readable200 | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard |
| CME Grouprobots.txt ↗ | inaccessible | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| CMS Energyrobots.txt ↗ | readable200 | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard |
| Coca-Cola Company (The)robots.txt ↗ | readable200 | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard |
| Cognizantrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Coherent Corp.robots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Coinbaserobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Colgate-Palmoliverobots.txt ↗ | ambiguous_content200 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Comcastrobots.txt ↗ | inaccessible | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Comfort Systems USArobots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| ConocoPhillipsrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Consolidated Edisonrobots.txt ↗ | readable200 | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard |
| Constellation Brandsrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Constellation Energyrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Cooper Companies (The)robots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Copartrobots.txt ↗ | ambiguous_content200 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Corning Inc.robots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Corpayrobots.txt ↗ | ambiguous_oversize200 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Cortevarobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| CoStar Grouprobots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Costcorobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| CRH plcrobots.txt ↗ | readable200 | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard |
| CrowdStrikerobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Crown Castlerobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| CSX Corporationrobots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Cumminsrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| CVS Healthrobots.txt ↗ | readable200 | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard |
| D. R. Hortonrobots.txt ↗ | readable200 | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard |
| Danaher Corporationrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Darden Restaurantsrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Datadogrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| DaVitarobots.txt ↗ | readable200 | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard |
| Deckers Brandsrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Deere & Companyrobots.txt ↗ | readable200 | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard |
| Dell Technologiesrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Delta Air Linesrobots.txt ↗ | readable200 | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard |
| Devon Energyrobots.txt ↗ | missing_file404 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Dexcomrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Diamondback Energyrobots.txt ↗ | inaccessible | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Digital Realtyrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Dollar Generalrobots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Dollar Treerobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Dominion Energyrobots.txt ↗ | readable200 | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard |
| Domino'srobots.txt ↗ | ambiguous_content200 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| DoorDashrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Dover Corporationrobots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Dow Inc.robots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| DTE Energyrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Duke Energyrobots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| DuPontrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Eaton Corporationrobots.txt ↗ | inaccessible | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| eBay Inc.robots.txt ↗ | readable200 | ambiguousspecific | ambiguousspecific | ambiguousspecific | ambiguousspecific | ambiguousspecific | ambiguousspecific | ambiguousspecific | ambiguouswildcard |
| EchoStarrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Ecolabrobots.txt ↗ | readable200 | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionspecific | no_published_restrictionspecific | no_published_restrictionspecific | no_published_restrictionspecific | no_published_restrictionwildcard | no_published_restrictionwildcard |
| Edison Internationalrobots.txt ↗ | readable200 | no_published_restrictionspecific | path_restrictionswildcard | no_published_restrictionspecific | no_published_restrictionspecific | no_published_restrictionspecific | no_published_restrictionspecific | path_restrictionswildcard | path_restrictionswildcard |
| Edwards Lifesciencesrobots.txt ↗ | ambiguous_content200 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Electronic Artsrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Elevance Healthrobots.txt ↗ | readable200 | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard |
| Emcorrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Emerson Electricrobots.txt ↗ | readable200 | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard |
| Entergyrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| EOG Resourcesrobots.txt ↗ | missing_file404 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| EQT Corporationrobots.txt ↗ | readable200 | ambiguousnone | ambiguousnone | ambiguousnone | ambiguousnone | ambiguousnone | ambiguousnone | ambiguousnone | ambiguousnone |
| Equifaxrobots.txt ↗ | readable200 | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard |
| Equinixrobots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Equity Residentialrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Erie Indemnityrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Essex Property Trustrobots.txt ↗ | inaccessible429 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Estée Lauder Companies (The)robots.txt ↗ | readable200 | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard |
| Everest Grouprobots.txt ↗ | readable200 | no_published_restrictionnone | no_published_restrictionnone | no_published_restrictionnone | no_published_restrictionnone | no_published_restrictionnone | no_published_restrictionnone | no_published_restrictionnone | no_published_restrictionnone |
| Evergyrobots.txt ↗ | ambiguous_content200 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Eversource Energyrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionsspecific | path_restrictionswildcard | path_restrictionsspecific | path_restrictionswildcard | path_restrictionswildcard |
| Exelonrobots.txt ↗ | readable200 | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard |
| Expand Energyrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Expedia Grouprobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Expeditors Internationalrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Extra Space Storagerobots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| ExxonMobilrobots.txt ↗ | readable200 | no_published_restrictionspecific | path_restrictionswildcard | no_published_restrictionspecific | no_published_restrictionspecific | no_published_restrictionspecific | no_published_restrictionspecific | path_restrictionswildcard | no_published_restrictionspecific |
| F5, Inc.robots.txt ↗ | readable200 | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard |
| FactSetrobots.txt ↗ | readable200 | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard |
| Fair Isaacrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Fastenalrobots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Federal Realty Investment Trustrobots.txt ↗ | readable200 | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard |
| FedExrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| FedEx Freightrobots.txt ↗ | readable200 | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard |
| Fidelity National Information Servicesrobots.txt ↗ | readable200 | ambiguousnone | ambiguousnone | ambiguousnone | ambiguousnone | ambiguousnone | ambiguousnone | ambiguousnone | ambiguousnone |
| Fifth Third Bancorprobots.txt ↗ | ambiguous_content200 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| First Solarrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| FirstEnergyrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Fiservrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Flex Ltd.robots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Ford Motor Companyrobots.txt ↗ | readable200 | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard |
| Fortinetrobots.txt ↗ | readable200 | path_restrictionsspecific | path_restrictionsspecific | path_restrictionsspecific | path_restrictionsspecific | blanket_disallowwildcard | path_restrictionsspecific | path_restrictionsspecific | blanket_disallowwildcard |
| Fortiverobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Fox Corporation (Class A)robots.txt ↗ | readable200 | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard |
| Franklin Resourcesrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Freeport-McMoRanrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Garminrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | blanket_disallowspecific | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Gartnerrobots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| GE Aerospacerobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| GE HealthCarerobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| GE Vernovarobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Gen Digitalrobots.txt ↗ | readable200 | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard |
| Generacrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| General Dynamicsrobots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| General Millsrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| General Motorsrobots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Genuine Parts Companyrobots.txt ↗ | missing_file404 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Gilead Sciencesrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Global Paymentsrobots.txt ↗ | readable200 | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard |
| Globe Liferobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| GoDaddyrobots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Goldman Sachsrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionsspecific | path_restrictionswildcard | path_restrictionsspecific | path_restrictionswildcard | path_restrictionswildcard |
| Halliburtonrobots.txt ↗ | readable200 | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard |
| Hartford (The)robots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Hasbrorobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| HCA Healthcarerobots.txt ↗ | ambiguous_content200 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Healthpeak Propertiesrobots.txt ↗ | readable200 | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard |
| Henry Scheinrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Hershey Company (The)robots.txt ↗ | missing_file404 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Hewlett Packard Enterpriserobots.txt ↗ | inaccessible | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Hilton Worldwiderobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Home Depot (The)robots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Honeywell Aerospacerobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Honeywell Technologiesrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Hormel Foodsrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Host Hotels & Resortsrobots.txt ↗ | ambiguous_content200 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Howmet Aerospacerobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| HP Inc.robots.txt ↗ | readable200 | ambiguousspecific | ambiguousspecific | ambiguousspecific | ambiguousspecific | ambiguousspecific | ambiguousspecific | ambiguousspecific | ambiguousspecific |
| Hubbell Incorporatedrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Humanarobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Huntington Bancsharesrobots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Huntington Ingalls Industriesrobots.txt ↗ | readable200 | no_published_restrictionspecific | path_restrictionswildcard | no_published_restrictionspecific | no_published_restrictionspecific | no_published_restrictionspecific | no_published_restrictionspecific | path_restrictionswildcard | path_restrictionswildcard |
| IBMrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| IDEX Corporationrobots.txt ↗ | readable200 | no_published_restrictionspecific | no_published_restrictionwildcard | no_published_restrictionspecific | no_published_restrictionspecific | no_published_restrictionspecific | no_published_restrictionspecific | no_published_restrictionwildcard | no_published_restrictionwildcard |
| Idexx Laboratoriesrobots.txt ↗ | readable200 | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard |
| Illinois Tool Worksrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Incyterobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Ingersoll Randrobots.txt ↗ | inaccessible | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Insulet Corporationrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Intelrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | no_published_restrictionspecific | no_published_restrictionspecific | no_published_restrictionspecific | no_published_restrictionspecific | path_restrictionswildcard | path_restrictionswildcard |
| Interactive Brokersrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Intercontinental Exchangerobots.txt ↗ | ambiguous_content200 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| International Flavors & Fragrancesrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| International Paperrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Intuitrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Intuitive Surgicalrobots.txt ↗ | ambiguous_content200 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Invescorobots.txt ↗ | readable200 | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard |
| Invitation Homesrobots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| IQVIArobots.txt ↗ | readable200 | no_published_restrictionnone | no_published_restrictionnone | no_published_restrictionnone | no_published_restrictionnone | no_published_restrictionnone | no_published_restrictionnone | no_published_restrictionnone | no_published_restrictionnone |
| Iron Mountainrobots.txt ↗ | inaccessible429 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| J.B. Huntrobots.txt ↗ | readable200 | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard |
| J.M. Smucker Company (The)robots.txt ↗ | readable200 | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard |
| Jabilrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Jack Henry & Associatesrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Jacobs Solutionsrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Johnson & Johnsonrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Johnson Controlsrobots.txt ↗ | readable200 | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionspecific | no_published_restrictionspecific | no_published_restrictionspecific | no_published_restrictionwildcard | no_published_restrictionwildcard |
| JPMorgan Chaserobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Kenvuerobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Keurig Dr Pepperrobots.txt ↗ | readable200 | no_published_restrictionnone | no_published_restrictionnone | no_published_restrictionnone | no_published_restrictionnone | no_published_restrictionnone | no_published_restrictionnone | no_published_restrictionnone | no_published_restrictionnone |
| KeyCorprobots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Keysight Technologiesrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Kimberly-Clarkrobots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Kimco Realtyrobots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Kinder Morganrobots.txt ↗ | readable200 | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard |
| KKR & Co.robots.txt ↗ | readable200 | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard |
| KLA Corporationrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Kraft Heinzrobots.txt ↗ | missing_file404 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Krogerrobots.txt ↗ | inaccessible | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| L3Harrisrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Labcorprobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Lam Researchrobots.txt ↗ | readable200 | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard |
| Las Vegas Sandsrobots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Leidosrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Lennarrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Lennox Internationalrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Lilly (Eli)robots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Linde plcrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Live Nation Entertainmentrobots.txt ↗ | readable200 | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard |
| Lockheed Martinrobots.txt ↗ | readable200 | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard |
| Loews Corporationrobots.txt ↗ | readable200 | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard |
| Lowe'srobots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Lululemon Athleticarobots.txt ↗ | inaccessible | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Lumentumrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| LyondellBasellrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| M&T Bankrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Marathon Petroleumrobots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Marriott Internationalrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Marsh McLennanrobots.txt ↗ | readable200 | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard |
| Martin Marietta Materialsrobots.txt ↗ | readable200 | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard |
| Marvell Technologyrobots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Mascorobots.txt ↗ | readable200 | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard |
| Mastercardrobots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| McCormick & Companyrobots.txt ↗ | ambiguous_content200 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| McDonald'srobots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| McKesson Corporationrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Medtronicrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Merck & Co.robots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Meta Platformsrobots.txt ↗ | readable200 | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard |
| MetLiferobots.txt ↗ | readable200 | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard |
| Mettler Toledorobots.txt ↗ | readable200 | ambiguousspecific | ambiguouswildcard | ambiguouswildcard | ambiguousspecific | ambiguouswildcard | ambiguousspecific | ambiguouswildcard | ambiguouswildcard |
| MGM Resortsrobots.txt ↗ | inaccessible | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Microchip Technologyrobots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Micron Technologyrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Microsoftrobots.txt ↗ | readable200 | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard |
| Mid-America Apartment Communitiesrobots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Modernarobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Molson Coors Beverage Companyrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Mondelez Internationalrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Monolithic Power Systemsrobots.txt ↗ | ambiguous_content200 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Monster Beveragerobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Moody's Corporationrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Morgan Stanleyrobots.txt ↗ | inaccessible | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Mosaic Company (The)robots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Motorola Solutionsrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| MSCI Inc.robots.txt ↗ | readable200 | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard |
| Nasdaq, Inc.robots.txt ↗ | inaccessible | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| NetApprobots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Netflixrobots.txt ↗ | readable200 | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard |
| Newmontrobots.txt ↗ | readable200 | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard |
| News Corp (Class A)robots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| NextEra Energyrobots.txt ↗ | readable200 | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard |
| Nike, Inc.robots.txt ↗ | readable200 | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard |
| NiSourcerobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Nordson Corporationrobots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Norfolk Southernrobots.txt ↗ | readable200 | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionspecific | no_published_restrictionspecific | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard |
| Northern Trustrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Northrop Grummanrobots.txt ↗ | readable200 | no_published_restrictionspecific | path_restrictionswildcard | no_published_restrictionspecific | blanket_disallowspecific | blanket_disallowspecific | no_published_restrictionspecific | path_restrictionswildcard | path_restrictionswildcard |
| Norwegian Cruise Line Holdingsrobots.txt ↗ | readable200 | ambiguousnone | ambiguousnone | ambiguousnone | ambiguousnone | ambiguousnone | ambiguousnone | ambiguousnone | ambiguousnone |
| NRG Energyrobots.txt ↗ | readable200 | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard |
| Nucorrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Nvidiarobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| NVR, Inc.robots.txt ↗ | inaccessible | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| NXP Semiconductorsrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| O’Reilly Automotiverobots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Occidental Petroleumrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Old Dominionrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Omnicom Grouprobots.txt ↗ | readable200 | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard |
| ON Semiconductorrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Oneokrobots.txt ↗ | missing_file404 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Oracle Corporationrobots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Otis Worldwiderobots.txt ↗ | readable200 | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard |
| Paccarrobots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Packaging Corporation of Americarobots.txt ↗ | readable200 | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard |
| Palantir Technologiesrobots.txt ↗ | readable200 | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard |
| Palo Alto Networksrobots.txt ↗ | readable200 | path_restrictionsspecific | path_restrictionsspecific | path_restrictionsspecific | path_restrictionswildcard | path_restrictionswildcard | path_restrictionsspecific | path_restrictionsspecific | path_restrictionswildcard |
| Paramount Skydance Corporationrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Parker Hannifinrobots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Paychexrobots.txt ↗ | readable200 | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard |
| PayPalrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Pentairrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| PepsiCorobots.txt ↗ | readable200 | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard |
| Pfizerrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| PG&E Corporationrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | blanket_disallowspecific | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Philip Morris Internationalrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Phillips 66robots.txt ↗ | readable200 | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard |
| Pinnacle West Capitalrobots.txt ↗ | readable200 | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard |
| PNC Financial Servicesrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| PPG Industriesrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| PPL Corporationrobots.txt ↗ | readable200 | no_published_restrictionnone | no_published_restrictionnone | no_published_restrictionnone | no_published_restrictionnone | no_published_restrictionnone | no_published_restrictionnone | no_published_restrictionnone | no_published_restrictionnone |
| Principal Financial Grouprobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Procter & Gamblerobots.txt ↗ | readable200 | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard |
| Progressive Corporationrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Prologisrobots.txt ↗ | readable200 | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard |
| Prudential Financialrobots.txt ↗ | readable200 | no_published_restrictionspecific | path_restrictionswildcard | no_published_restrictionspecific | blanket_disallowspecific | path_restrictionswildcard | no_published_restrictionspecific | path_restrictionswildcard | path_restrictionswildcard |
| PTC Inc.robots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Public Service Enterprise Grouprobots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Public Storagerobots.txt ↗ | readable200 | path_restrictionsspecific | path_restrictionswildcard | path_restrictionsspecific | path_restrictionsspecific | path_restrictionsspecific | path_restrictionsspecific | path_restrictionswildcard | path_restrictionswildcard |
| PulteGrouprobots.txt ↗ | readable200 | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard |
| Qnity Electronicsrobots.txt ↗ | missing_file404 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Qualcommrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionsspecific | path_restrictionsspecific | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Quanta Servicesrobots.txt ↗ | readable200 | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard |
| Quest Diagnosticsrobots.txt ↗ | readable200 | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard |
| Ralph Lauren Corporationrobots.txt ↗ | missing_file404 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Raymond James Financialrobots.txt ↗ | inaccessible | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Realty Incomerobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Regency Centersrobots.txt ↗ | inaccessible | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Regeneron Pharmaceuticalsrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionsspecific | path_restrictionsspecific | path_restrictionsspecific | path_restrictionsspecific | path_restrictionswildcard | path_restrictionsspecific |
| Regions Financial Corporationrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Republic Servicesrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| ResMedrobots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Revvityrobots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Robinhood Marketsrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Rockwell Automationrobots.txt ↗ | readable200 | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard |
| Rollins, Inc.robots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Roper Technologiesrobots.txt ↗ | inaccessible | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Ross Storesrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Royal Caribbean Grouprobots.txt ↗ | readable200 | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard |
| RTX Corporationrobots.txt ↗ | readable200 | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard |
| S&P Globalrobots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Salesforcerobots.txt ↗ | readable200 | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard |
| Sandiskrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| SBA Communicationsrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Schlumbergerrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Seagate Technologyrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Semprarobots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| ServiceNowrobots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Sherwin-Williamsrobots.txt ↗ | readable200 | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard |
| Simon Property Grouprobots.txt ↗ | readable200 | no_published_restrictionspecific | no_published_restrictionspecific | blanket_disallowspecific | blanket_disallowspecific | blanket_disallowspecific | no_published_restrictionspecific | no_published_restrictionspecific | blanket_disallowspecific |
| Skyworks Solutionsrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Smurfit Westrockrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Snap-onrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Solventumrobots.txt ↗ | readable200 | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard |
| Southern Companyrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Southwest Airlinesrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Stanley Black & Deckerrobots.txt ↗ | readable200 | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard |
| Starbucksrobots.txt ↗ | readable200 | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard |
| State Street Corporationrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Steel Dynamicsrobots.txt ↗ | inaccessible | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Sterisrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Stryker Corporationrobots.txt ↗ | readable200 | no_published_restrictionnone | no_published_restrictionnone | no_published_restrictionnone | no_published_restrictionnone | no_published_restrictionnone | no_published_restrictionnone | no_published_restrictionnone | no_published_restrictionnone |
| Supermicrorobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Synchrony Financialrobots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Synopsysrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Syscorobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| T-Mobile USrobots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| T. Rowe Pricerobots.txt ↗ | readable200 | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard |
| Take-Two Interactiverobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Tapestry, Inc.robots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Targa Resourcesrobots.txt ↗ | inaccessible | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Target Corporationrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| TE Connectivityrobots.txt ↗ | readable200 | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguousspecific | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard |
| Teledyne Technologiesrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Teradynerobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Tesla, Inc.robots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Texas Instrumentsrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Texas Pacific Land Corporationrobots.txt ↗ | missing_file404 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Textronrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Thermo Fisher Scientificrobots.txt ↗ | readable200 | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard |
| TJX Companiesrobots.txt ↗ | missing_file404 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| TKO Group Holdingsrobots.txt ↗ | readable200 | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard |
| Tractor Supplyrobots.txt ↗ | inaccessible | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Trade Desk (The)robots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Trane Technologiesrobots.txt ↗ | readable200 | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard |
| TransDigm Grouprobots.txt ↗ | readable200 | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard |
| Travelers Companies (The)robots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Trimble Inc.robots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Truist Financialrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Tyler Technologiesrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Tyson Foodsrobots.txt ↗ | missing_file404 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| U.S. Bancorprobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Uberrobots.txt ↗ | readable200 | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard |
| UDR, Inc.robots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Ulta Beautyrobots.txt ↗ | readable200 | ambiguousspecific | ambiguousspecific | ambiguousspecific | ambiguousspecific | ambiguousspecific | ambiguousspecific | ambiguousspecific | ambiguousnone |
| Union Pacific Corporationrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| United Airlines Holdingsrobots.txt ↗ | inaccessible | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| United Parcel Servicerobots.txt ↗ | ambiguous_content200 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| United Rentalsrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| UnitedHealth Grouprobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Universal Health Servicesrobots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Valero Energyrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Veeva Systemsrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Ventasrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Veraltorobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Verisignrobots.txt ↗ | readable200 | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard |
| Verisk Analyticsrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Verizonrobots.txt ↗ | readable200 | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard |
| Vertex Pharmaceuticalsrobots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Vertivrobots.txt ↗ | readable200 | no_published_restrictionspecific | no_published_restrictionspecific | no_published_restrictionspecific | no_published_restrictionspecific | no_published_restrictionspecific | no_published_restrictionspecific | path_restrictionswildcard | no_published_restrictionspecific |
| Viatrisrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Vici Propertiesrobots.txt ↗ | readable200 | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard |
| Visa Inc.robots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Vistra Corp .robots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Vulcan Materials Companyrobots.txt ↗ | readable200 | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard |
| W. R. Berkley Corporationrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| W. W. Graingerrobots.txt ↗ | ambiguous_content200 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Wabtecrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Walmartrobots.txt ↗ | readable200 | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard |
| Walt Disney Company (The)robots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Warner Bros. Discoveryrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Waste Managementrobots.txt ↗ | inaccessible403 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Waters Corporationrobots.txt ↗ | inaccessible | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| WEC Energy Grouprobots.txt ↗ | missing_file404 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Wells Fargorobots.txt ↗ | readable200 | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard |
| Welltowerrobots.txt ↗ | readable200 | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard |
| West Pharmaceutical Servicesrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Western Digitalrobots.txt ↗ | readable200 | path_restrictionsspecific | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Weyerhaeuserrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Williams Companiesrobots.txt ↗ | readable200 | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard |
| Williams-Sonoma, Inc.robots.txt ↗ | readable200 | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard |
| Willis Towers Watsonrobots.txt ↗ | inaccessible429 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Workday, Inc.robots.txt ↗ | readable200 | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard |
| Wynn Resortsrobots.txt ↗ | readable200 | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard | path_restrictionswildcard |
| Xcel Energyrobots.txt ↗ | missing_file404 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Xylem Inc.robots.txt ↗ | ambiguous_content200 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Yum! Brandsrobots.txt ↗ | inaccessible | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Zebra Technologiesrobots.txt ↗ | readable200 | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard | ambiguouswildcard |
| Zimmer Biometrobots.txt ↗ | ambiguous_content200 | unknown | unknown | unknown | unknown | unknown | unknown | unknown | unknown |
| Zoetisrobots.txt ↗ | readable200 | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard | no_published_restrictionwildcard |