Skip to main content
Scrapy Playbook - Scrapy Logo

The
Python Scrapy
Playbook

Everything you need to know to become a Scrapy Pro!

Scrapy Community:
Scrapy Reddit Community
Scrapy Discord Community
Scrapy Twitter

Stay Up To Date

Get notified of the latest Scrapy news, guides, extensions, and spiders as soon as they are released. Also, suggest new guides and extensions for the Scrapy Playbook then signup here.

Scrapy For Beginners Series

Scrapy Playbook - Create Your First Scrapy Spider - Python Scrapy Beginner Series [Part 1]

Create Your First Scrapy Spider - Python Scrapy Beginner Series [Part 1]

In this video, we go through how to set up your first Python Scrapy spider and how to scrape data from a live website.

Scrapy Playbook - Cleaning & Post-Processing Scrapy Data - Python Scrapy Beginners Series [Part 2]

Cleaning & Post-Processing Scrapy Data - Python Scrapy Beginners Series [Part 2]

In this video, we go through how to clean & post-process our scraped data when we scrape messy data.

Scrapy Playbook - Saving Scraped Data - Python Scrapy Beginner Series [Part 3]

Saving Scraped Data - Python Scrapy Beginner Series [Part 3]

In this video, we go through how to save data you have scraped into a MySQL & Postgres database, CSV & JSON files, and Amazon S3 buckets.

Scrapy Playbook - Bypassing Restrictions  - Python Scrapy Beginner Series [Part 4]

Bypassing Restrictions - Python Scrapy Beginner Series [Part 4]

In this video, we look at how to bypass any blockers, anti-bots, and anything else that might be stopping us from scraping a website using Python Scrapy by using a mixture of User Agents and Proxies.

How To Scrape Python Scrapy Series

Scrapy Playbook - Scraping Walmart.com with Python Scrapy (2022)

Scraping Walmart.com with Python Scrapy (2022)

In this video, learn how to build a Python Scrapy spider that will crawl Walmart.com for products and scrape Walmart product pages.

Scrapy Playbook - Scraping Indeed.com with Python Scrapy (2022)

Scraping Indeed.com with Python Scrapy (2022)

In this video, learn how to build a Python Scrapy spider that will crawl Indeed.com for jobs and scrape individual job pages.

Scrapy Playbook - Scraping Amazon Products with Python Scrapy (2022)

Scraping Amazon Products with Python Scrapy (2022)

In this video, learn how you can scrape product data from Amazon using Python Scrapy. We look at how to search for a specific item, crawl through the search results and then parse the associated product data.

Scrapy Playbook - Scraping Amazon Reviews with Python Scrapy (2022)

Scraping Amazon Reviews with Python Scrapy (2022)

In this video, learn how you can scrape reviews from Amazon products using Python Scrapy. We look at how to search for a specific product, crawl through the reviews and then parse the associated review data.

Scrapy Playbook - Scraping LinkedIn Profiles with Python Scrapy (2022)

Scraping LinkedIn Profiles with Python Scrapy (2022)

In this video, learn how you can scrape LinkedIn Profiles using Python Scrapy!

Crawling & Navigating Sites

Scrapy Playbook - How to Login with Python Scrapy (2022)

How to Login with Python Scrapy (2022)

In this video, we show you how to build a Scrapy spider that can log into any website and scrape private data.

Storing Data With Feed Exporters & Pipelines

Scrapy Playbook - Saving Data To CSV Files With Scrapy Feed Exporters

Saving Data To CSV Files With Scrapy Feed Exporters

In this video, we show you how to use Scrapy to scrape data from a website and save it to CSV files using Scrapy Feed Exporters.

Dealing With Javascript Heavy Websites

Scrapy Playbook - Scrapy-Playwright: How To Scrape Dynamic JS Websites (2022)

Scrapy-Playwright: How To Scrape Dynamic JS Websites (2022)

In this video, we go through how to scrape data from javascript rendered websites using Scrapy Playwright.

Scrapy Playbook - Scrapy Selenium - Scraping Javascript Rendered Websites (2022)

Scrapy Selenium - Scraping Javascript Rendered Websites (2022)

In this video, we go through how to scrape data from javascript rendered websites using Scrapy Selenium.

Scrapy Playbook - Scrapy Splash: How To Scrape JS Rendered Websites

Scrapy Splash: How To Scrape JS Rendered Websites

In this project, we use Scrapy Splash to scrape javascript rendered websites.

Hosting & Scheduling Spiders

Scrapy Playbook - How To Schedule Scrapy Spiders With Scrapyd & ScrapeOps

How To Schedule Scrapy Spiders With Scrapyd & ScrapeOps

In this video, we go through how to set up & schedule Scrapy spiders to run on Scrapyd using ScrapeOps.

Scrapy Playbook - Schedule & Run Scrapy Spiders On Digital Ocean Droplet With ScrapeOps

Schedule & Run Scrapy Spiders On Digital Ocean Droplet With ScrapeOps

In this video, we show you how you can provision a Digital Ocean server and schedule a Scrapy spider for scraping using the ScrapeOps dashboard.

Scrapy Playbook - Deploy & Run Your Scrapy Spiders On AWS Server Using ScrapeOps

Deploy & Run Your Scrapy Spiders On AWS Server Using ScrapeOps

In this video, we show you how you can provision a AWS server and schedule a Scrapy spider for scraping using the ScrapeOps dashboard.

Scrapy Playbook - How Schedule & Run Scrapy Spiders With Scrapy Cloud

How Schedule & Run Scrapy Spiders With Scrapy Cloud

In this video, we go through how to schedule & run Scrapy spiders to run on Scrapy Cloud, created by Zyte (formerly Scrapinghub)

Need an easy way to monitor your scrapers?

Sign up for a free ScrapeOps account today.