REST API with a single /breweries endpoint that returns a list of breweries in the United States using Node + Express + TypeScript + Passport + Jest stack.
Using the above tech stack, write a REST API with a single /breweries endpoint that returns a list of breweries in the United States. The /breweries endpoint of your API should be tested with Jest and secured using Passport. In addition to fetching this data directly from the data source above, this endpoint should function as an ETL data pipeline where you process the data in the following ways:
- Step 1: Remove any attributes that are null from the data
- Step 2: Convert the keys of the objects in the response from snake case to camel case (e.g. “postal_code” -> “postalCode”)
- Step 3: Group the breweries together by state and then sort them by created_at so the most recent ones come first.
- Step 4: Add an attribute to each brewery called region that adds the correct region to each brewery based on this map: https://www.worldatlas.com/articles/the-regions-of-the-united-states.html (hint - take a look at the GPS coordinates for the United States and then use the longitude & latitude attributes for each brewery to compute this). If the brewery does not have a longitude & latitude then filter it out.
Don't use lodash/underscore/rambda or any other helper libraries.
Install the latest LTS Node.js version. If you are using nvm you can just run nvm install in the root folder.
git clone https://github.com/hpieroni/brewerydb-etl-exercise.gitcd brewerydb-etl-exercisenpm install
In order to start the API you need to configure some ENV variables first. You can do this through an .env file (check .env.example ).
| Variable | Default | Description |
|---|---|---|
| PORT | 3000 | Port where the server will listen |
| TOKEN | Authentication token |
Finally run npm start and check the console for a message similar to:
App is listening at http://localhost:3000 !
Now you should be able to call:
curl -i -H "Content-Type: application/json" \
-H "Authorization: Bearer test" \
http://localhost:3000/breweriesand get a response similar to:
{
"Texas": [],
"Colorado": [
{
"id": 9180,
"obdbId": "boulder-beer-co-boulder",
"name": "Boulder Beer Co",
"breweryType": "regional",
"street": "2880 Wilderness Pl",
"city": "Boulder",
"state": "Colorado",
"postalCode": "80301-5401",
"country": "United States",
"longitude": "-105.2480158",
"latitude": "40.026439",
"updatedAt": "2018-08-24T00:00:00.000Z",
"createdAt": "2018-07-24T00:00:00.000Z",
"region": "west"
}
],
"Florida": [],
"Iowa": [],
"Louisiana": [],
"California": [],
"Pennsylvania": [
{
"id": 11039,
"obdbId": "goose-island-philadelphia-philadelphia",
"name": "Goose Island Philadelphia",
"breweryType": "brewpub",
"street": "1002 Canal St",
"city": "Philadelphia",
"state": "Pennsylvania",
"postalCode": "19123",
"country": "United States",
"longitude": "-75.13506341",
"latitude": "39.9648491",
"updatedAt": "2018-08-24T00:00:00.000Z",
"createdAt": "2018-07-24T00:00:00.000Z",
"region": "northeast"
}
],
"Georgia": [],
"Michigan": [
{
"id": 11767,
"obdbId": "ironbark-brewery-jackson",
"name": "Ironbark Brewery",
"breweryType": "micro",
"street": "2610 Kibby Rd",
"city": "Jackson",
"state": "Michigan",
"postalCode": "49203-4908",
"country": "United States",
"longitude": "-84.43762976",
"latitude": "42.2188971",
"phone": "5177487988",
"updatedAt": "2018-08-24T00:00:00.000Z",
"createdAt": "2018-07-24T00:00:00.000Z",
"region": "midwest"
}
],
"Virginia": [],
"New Hampshire": [],
"Maryland": [],
"Ohio": []
}npm run testruns the test suitnpm run test:coveragedisplays a test coverage report
I decided to follow a functional approach because of the nature of the data pipeline. I created some utility functions on that regard.
The BreweryETL class has the methods (extract, transform and load). The transform method is implemented by composing functions. If you want to add a new step to the pipeline, you need to add it there.
You can also create other ETLs following the same patterns. In this example, I've only created a breweries etl.
I used passport-http-bearer but, to make it simple, I only check for the token configured in .env file.
In order to find the US region given a coordinate I used a ray casting algorithm where I manually created an aproximatino of the regions with polygons (sorry Alaska for leaving you out). You can see the jsfiddle https://jsfiddle.net/k785fL0c/1/ I created.
- Implement
curryfunction and use it in utility functions.
