AI & Machine Learning
Common Crawl
Public web crawl archives with page captures, metadata, and extracted text for large-scale research.
- Resource type
- Dataset
- Pricing
- Free
- Access
- No key
- Formats
- WARC · WAT · WET
- License
- Stated by original source
Access & usage conditions
Corpus access is free; processing and cloud compute can cost money. Crawled content retains its original rights and may contain personal data.
Free access does not necessarily allow unrestricted reuse. The original provider's current terms apply.
https://commoncrawl.org/overview