How to Scrape a Static Website

A really quick tutorial

Prerequisites: Knowledge of React.js will be required for this tutorial.

Let’s say you want to pull data from the frontend of a website because there’s no API available. You inspect the page and see that the data is available in the HTML, so how do you gather that information to be used in your app? It’s rather simple, we’re going to install two libraries and write less than 50 lines of code to demonstrate the scraping of a website. To keep this tutorial simple, we’ll use https://pokedex.org/ as our example.

1. In terminal:

create-react-app scraping-demo
cd scraping-demo
npm i request-promise
npm i cheerio

2. We’re going to start by using request-promise to get the HTML from https://pokedex.org/ into a console log.

In App.js:

3. Sometimes you may come across a CORS error blocking you from fetching. For demonstration purposes, try fetching pokemon.com

rp("https://www.pokemon.com/us/pokedex/")