Python Test Driven Development Example

Copy Link LinkedIn Message Facebook Email X

If you've closely followed along through the previous lessons, you now have a test file called test_refactor.py with the following code in it:

import unittest
import rescrape

class TestRescrape(unittest.TestCase):

    # requests can establish a connection and receive a valid response
    def test_get_valid_html_response(self):
        BASE_URL = "https://codingnomads.github.io/recipes/"
        index_page = rescrape.get_page_content(BASE_URL)
        self.assertEqual(index_page.status_code, 200)

    # requests can establish a connection and receive a valid response

    # the response contains HTML code

    # the HTML can be successfully converted to a Beautiful Soup object

    # can identify all links from the index page

    # can identify the author of a recipe

    # can get the main recipe text
    
if __name__ == "__main__":
    unittest.main()

Example Rescrape File

You also changed the code in your rescrape.py file that you wrote earlier in this course. Your code will most likely be a bit different than what you'll see below, but it should have the same functionality:

import requests
from bs4 import BeautifulSoup

BASE_URL = "https://codingnomads.github.io/recipes/"

def get_page_content(url):
    """Gets the response from a HTTP call to the URL."""
    page = requests.get(url)
    return page


if __name__ == "__main__":
    index_html = get_html_content(BASE_URL)
    index_soup = BeautifulSoup(index_html, "html.parser")
    recipe_links = [link["href"] for link in soup.find_all("a")]

    for r_link in recipe_links:
        URL = f"{BASE_URL}/{r_link}"
        html = get_html_content(BASE_URL)
        soup = BeautifulSoup(html, "html.parser")
        author = soup.find("p", class_="author").text.strip("by ")
        recipe = soup.find("div", class_="md").text
        print(f"({author})\t[{recipe}]\n\n\n")

You've taken your first step in adding tests and refactoring the web scraper code. Nice work! Of course, there's always more to do :)

Think about other functionality that you could compartmentalize in functions in your script. Consult the pseudocode code comments that you wrote in your test_rescrape.py file.

Tasks

• Repeat the TDD process you just went through. Write another test, then refactor your scraper code to make sure the test passes.
• Rinse and repeat for yet another piece of functionality.
• Keep going until you have written out all your test ideas and you've refactored your production code so that as much of its functionality as possible is covered by tests.

In the coming lesson, you'll learn about additional features of the unittest module that allow you to cut down on some of that repetition.

Summary: Python Test-Driven Development Example

• Adding tests makes your code more secure
• Repeat this process to increase the stability of your code
• The TDD approach is an option to refactor your code

Info: You'll find a link to a possible solution at the end of this section. However, it's strongly advised that you attempt to write the test cases and refactor your script accordingly by yourself since learning the process of doing this is what will ultimately come in useful. And learning processes require your time, effort, and repetition!