unclecode/crawl4ai
LLM-friendly web crawler and scraper
This repository provides an open-source web crawler and scraper that converts web content into clean, LLM-ready Markdown. It solves the problem of extracting web data for retrieval-augmented generation, agents, and data pipelines.
- Stars
- 84,200
- Stars gained in 7 days
- +400
- Contributors
- 92
- Forks
- 8,700
- Days trending in the last 30
- 2 days
- Mark
- Watch — 915 stars per contributor. An automatic mark on public metrics: above 500 is watch, above 2,000 possible hype (not a judgement of quality).