Skip to content

decooliveira/scrape

Repository files navigation

Scraper

Web scraping tool. Scraping the first 5 pages of reviews of McKaig Chevrolet Buick from Dealer Rater
Scraper Tool prints the 3 most positive reviews from the five pages of reviews.

How the most positive reviews are found?

A very simple logic was applied to determine the most positive reviews. First, only reviews with five stars rating are collected and then those reviews are sorted by the length of the endorsements texts. Satisfied customers with positive reviews tend to write more. Obviously, in real scenarios, those endorsements could be assessed by AI tools. So, for the sake of simplicity, the considered most positive reviews are those with 5 stars rating and more text

Requirements

Python version 3.9.0 - See https://www.python.org/downloads

pip3 - The Python Package Installer. Version 20.3.4 - See https://pip.pypa.io/en/stable/

Dependencies

Requests: $ pip3 install requests

BeautifulSoup: pip3 install beautifulsoup4

Usage

Run

python3 scraper.py

Test

python3 -m unnittest to run all tests

python3 -m unnittest 'test_file_name' to run a specific unit test

Logs

File logs.txt stores the logs of this application. In order to change the file path of logs file, the value of FILE_PATH variable needs to be changed in the Logger.py class**

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors

Languages