Description

Web Scraping nowdays has become one of the hottest topics, there are plenty of paid tools out there in the market that don’t show you anything how things are done as you will be always limited to their functionalities as a consumer.

In this course you won’t be a consumer anymore, i’ll teach you how you can build your own scraping tool ( spider ) using Scrapy.

You will learn:

   The fundamentals of Web Scraping
   How to build a complete spider
   The fundamentals of XPath
   How to locate content/nodes from the DOM using XPath
   How to store the data in JSON, CSV… and even to an external database(MongoDb)
   How to write your own custom Pipeline
   Fundamentals of Splash
   How to scrape Javascript websites using Scrapy Splash
   The Crawling behavior
   How to build a CrawlSpider
   How to avoid getting banned while scraping websites
   How to build a custom Middleware
   Web Scraping best practices
   How to scrape APIs
   How to use Request Cookies
   How to scrape infinite scroll websites
   Host spiders in Heroku for free
   Run spiders periodically with a custom script
   Prevent storing duplicated data
   Deploy Splash to Heroku
   Write data to Excel files
   Login to websites using FormRequest
   Download Files & Images using Scrapy
   Use Proxies with Scrapy Spider
   Use Crawlera with Scrapy & Splash
   Use Proxies with CrawlSpider

What makes this course different from the others, and why you should enroll ?

   First, this is the most updated course. You will be using Python 3.6, Scrapy 1.5 and Splash 2.0

   You will have an in-depth step by step guide on how to become a professional web scraper.

   I’ll show you how other courses scrape Javascript websites using Selenium and why shouldn’t do it in their way.
   You will learn how to use Splash to scrape Javascript websites and i can assure you won’t find any tutorials out there that teaches how to really use Splash like i’ll be doing in this course.
   You will learn how to host spiders in Heroku as well as Splash(Exclusive).
   You will learn how to create a custom script so spiders can run periodically without any intervention from you.

   30 days money back guarantee by Udemy

So whether you are a data analyst who wants to add web scraping to his tool set or someone else who wants to learn how to extract unstructured data from unstructured HTML web pages and then store back that data in a structured way to apply some data analysis on it then you are welcome to join this course.
Who this course is for:

   Anyone who wants to scrape data from any website
   Anyone who wants to learn Scrapy
   Anyone who wants to automate the task of copying contents from websites
   Anyone who wants to learn how to scrape Javascript websites using Scrapy-Splash
   Anyone who wants to learn the basics of Xpath
   Anyone who want to learn Scrapy Splash

Requirements

   Basics of Python
   Basics of HTML
   Basics of Javascript
   Internet access

Last updated 5/2019