Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Parsel, BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.
项目介绍
Crawlee——一款用于构建可靠爬虫的Python网络抓取和浏览器自动化库。可提取数据以供AI、LLMs、RAG或GPT使用。从网站下载HTML、PDF、JPG、PNG等文件。支持与BeautifulSoup、Playwright以及原生HTTP配合使用。支持有界面和無界面两种模式。具备代理轮换功能。【此简介由AI生成】
Apache-2.0 Python1.7 K提交数apifyautomationbeautifulsoupcrawlercrawlinghacktoberfestheadlessheadless-chromeparselpipplaywrightpythonscraperscrapingweb-crawlerweb-crawlingweb-scraping
定制我的领域