Scrapy!

* Download http://scrapy.org/download
* Documentation http://scrapy.org/doc

Scrapy alternatives

  • ParseHub

  • ParseHub is a web scraping tool built to handle the modern web.

    tags: api data-mining no-coding relative-select web-scraping
  • Portia

  • Portia is an open source visual scraping tool, allows you to scrape websites without any programming knowledge required! Simply annotate pages you're interested in, and Portia will create a spider to extract data from similar pages.

    tags: web-scraping web-crawler web-crawling screen-scraping
  • import.io

  • import.io is a free web-based platform that puts the power of the machine readable web in your hands. Using our tools you can create an API or crawl an entire website in a fraction of the time of traditional methods, no coding required. Our highly efficient and scalable platform allows you to process 1,000s of queries at once and get real-time data in any format you choose. We also offer an easy to use client library to make exporting, integrating and using your data as simple as extracting it.

    tags: data-mining data-extraction extractor crawler data-export
  • Kantu Web Automation Browser

  • Kantu allows you to visually automate your task, and makes web automation fun again. It lets you create solutions for web automation, web scraping or web testing in minutes. It's like the ultimate robot stand-in for all of your web browser automation needs! Whatever you used to do manually, Kantu can do automatically -- stuff like filling out forms, clicking on links, performing inquiries, you name it, Kantu has it. Multiple page forms? Not a problem for Kantu.

    tags: automation browser-integration computer-vision file-downloading file-uploading
  • UiPath

  • Robotic Process Automation Software.Automate rule based business processes. Train and design robots that drive the UI like a human.

    tags: automated-tasks automation browser-enhancement business-process-automation macro-recorder
  • Diggernaut

  • Diggernaut is a cloud based service for web scraping, data extraction and other ETL tasks. Imagine spending hours a day manually collecting data from websites you need. It's very cumbersome and time consuming. With Diggernaut, you can speed up the data collection process a thousand times and save time to do more important tasks. Our tiny diggers can do web scraping on your behalf and get data from websites for you. Just leave it up to Diggernaut to get your job done.

    tags: big-data data-mining web-scraping etl web-scraper
  • Octoparse

  • Octoparse is a modern visual web data extraction software. Both experienced and inexperienced users would find it easy to use Octoparse to bulk extract information from websites, for most of scraping tasks no coding needed. Users can extract data from 98% of open websites using our tools. Octoparse with its point-and-click interface, makes web-scraping very easy to learn and understand. Use the data extracted to power your business intelligence, build up your customer database.

    tags: cloud-service crawler data-analytics data-extraction data-miner
  • Webhose.io

  • Webhose.io is an advanced DaaS (Data as a Service) platform.

    tags: social-media search-engine search-tool big-data news-feed
  • Apify

  • Apify is the world’s most advanced web automation platform. Web crawler that works on every website.

    tags: jquery-crawler web-crawler web-crawling web-scraper web-scraping
  • Instaparser

  • Give your users a lightning-fast browsing experience by using parsing tools from Instapaper to handle your text.

    tags: Discontinued data-mining web-scraping data-extraction
  • 80legs

  • 80legs offers powerful web crawling. Extract data from web pages, images, and any other online content. Start crawling websites now faster, easier, and with unlimited reach.

    tags: data-mining crawling spidering harvesting htmlscraping
  • ScrapeHero

  • Main features of ScrapeHero:

    tags: crawler crawling-as-service data-as-service data-extraction data-mining
  • Zennoposter

  • ZennoPoster 5 is intended for SEO-experts, webmasters and people engaged in vigorous activity on the Internet. The software allows to record human actions on websites, blogs, forums (filling in forms, clicks on links, post messages) and repeat them in multiple threads (Professional version). The program also provides anonymity through proxies, processed by powerful built-in proxychecker.

    tags: content-discovery marketing-automation seo-optimization seo-spider visual-programming
  • Scrapinghub

  • Scrapinghub is the most advanced platform for deploying and running web crawlers (also known as "spiders"). It allows your organization to build crawlers easily, deploy them instantly and scale them on demand, without having to manage servers, backups or cron jobs. Everything is stored in our highly available database and retrievable from our API.

    tags: web-based data-mining company web-scraper
  • Product API by Fetchee

  • Simple API to extract product data for any URL.

    tags: api data-mining retailer web-scraping web-crawler