How to Scrape Amazon Stores for Generating Price Alerts?

Initially, you will need a file named Tracker_PRODUCTS.csv with the links for the products you wish to check. After executing run, the scraper will save the results in a different file known as “search_history_[date].xlsx. These are the files placed inside search_history folder.

For completing this task, we will require BeautifulSoup as our web scraping tool. If you need to install any of them, simple script that includes simple pip/conda install will do. There are various sources which will help, but usually Python package Index will have it.

The HEADERS variables need to pass along the get method. Once the file is passes, it might be a good time to get the CSV file known as TRACKER_PRODUCTS.csv from the repository and place it in the folder named “trackers.”

Then, we will execute this through BeautifulSoup. This will convert the HTML in to some more convenient file named soup.

If ever you will find soup.find, it will mean that we are in search of an element of the page that uses its HTML tag(such as div, or span, etc.) With soup.select we will use CSS selectors.

(De)constructing the soup

There is also script for getting prices in USD is added

Once the testing is completed, the proper script is to be written which will:

  • Fetch the URLs from a csv file.
  • Will use a while loop to scrape every product and save the information.
  • Save all the results that will include previous searches in an excel file.

You will also need a scraper, and we will call it Amazon_scraper.py.

Observations on the file TRACKER_PRODUCTS.csv It’s a simple file only with three columns (“link,” “code,” and “buy below”). That’s where you’ll enter the product URLs you’d like to track.

You may even put this file in a synced Dropbox folder (and then update the script with the updated file URLs) so that you really can update it from your phone at any time. If you execute the script on a server or on your personal laptop at home, it will pick up that new product link from the file in its next run.

The SEARCH HISTORY files are the same way. On the very first run, you must add an empty file to both the folder “search history” (which may be found in the repository). When establishing the last search variable on line 116 of the script above, we’re looking for the most recent file in the search history folder. As a result, you must additionally put your own directory here. Simply change the text with the name of the folder where you’ll be working on this project (in my case, “Amazon Scraper”).

Setting up a Scheduled Task for Running Script

Setting up an automated task for executing small scripts.

1. You will initiate it by opening “Task Scheduler”. Then select “Create task” and pick up the “triggers” tab.

2. Next, you will need to move to the Actions tab. Here, you will need to add an action and pick your Python folder location for the “Program/script” box.

3. In the arguments box, you will need to type the name of our file with the function

4. We will tell the system to start the command in the folder where our file Amazon_Scraper.py is.

After running this script, the task is ready to run.

For any queries related to scraping Amazon stores for generating price alerts, contact 3i Data Scraping.

Originally published at https://www.3idatascraping.com.

--

--

3i Data Scraping is an Experienced Web Scraping Service Provider in the USA. We offering a Complete Range of Data Extraction from Websites and Online Outsource.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Data Scraping Services and Data Extraction

3i Data Scraping is an Experienced Web Scraping Service Provider in the USA. We offering a Complete Range of Data Extraction from Websites and Online Outsource.