AI & Machine Learning

Common Crawl

Public web crawl archives with page captures, metadata, and extracted text for large-scale research.

Resource type
Dataset
Pricing
Free
Access
No key
Formats
WARC · WAT · WET
License
Stated by original source

Access & usage conditions

Corpus access is free; processing and cloud compute can cost money. Crawled content retains its original rights and may contain personal data.

Free access does not necessarily allow unrestricted reuse. The original provider's current terms apply.

Open original source

https://commoncrawl.org/overview