This repository contains training material on becoming familiar with APIs with steps and exercises using it with Magento Business Intelligence's RESTful API:
-
Guides/videos on learning the terminal and API (GET, POST requests)
-
Learning about cURL and how to send/retrieve data
-
Using the terminal to interact with Python and RJMetrics
-
Exercises on exporting data from APIs and an RJMetrics dashboard
-
Exercises on importing data into RJMetrics using APIs
-
Creating cron jobs to automate python scripts
-
Generating updated raw data exports & requesting new data through python
-
Load the generated report (which is a .csv) back into python and transform into a json format
-
Parse specific dimensions/columns from reports while keeping the same json format
-
Forecast data on existing reports
This video provides basic knowledge on how to navigate through the terminal and how to use the command line. Once completed, try completing the following quizzes. You may need to use Google to find some of the answers.
What is an API and what can you do with it? Review the following videos and guides to get a better understanding.
Now that you've learned about APIs, find out how to transfer data in the terminal using a cURL command.
Here's a brief introduction to Python
Check out the Open Philly Indego Bikes API and get a sense of what it does.
Then try the following:
1. Make a single request to the API
Hint: Use the requests library
2. Save the output to a .txt file
3. Format the output in a pretty json format (You should be able to do this with one line of code)
Check out the Help Center article on export APIs in RJMetrics and then try the following.
1. Export a specific report (in RJMetrics); save the contents to a .txt file
2. Export a .csv through a raw data export (try doing this in both the terminal and python)
Note: You'll need to create a report and a raw data export to be able to do this. You cannot use a tabular report.
This will require some googling to understand. Try to understand what you're doing along the way, and not just copy and paste indiscriminately.
Good luck!
Now that you know how to export data, now try importing into RJMetrics. You can reference this python article.
Also, read this Developers Article as well as this Help Center Article for details on how to get authenticated with the Data Import API.
Task:
For this exercise, try importing only one line of data such as:
data = {
"keys": ["id"],
"id": 1,
"email": "[email protected]",
"status": "pending",
"created_at": "2012-08-01 14:22:32"
}You now know how to send one data point. How do you send more than that?
Task:
Send multiple records to the RJMetrics API using a for loop. For example, something like this:
data1 = [{
# "keys": ["id"],
"id": 1,
"email": "[email protected]",
"status": "pending",
"created_at": "2012-08-01 14:22:32"
},{
"id": 2,
"email": "[email protected]",
"status": "pending",
"created_at": "2012-08-03 23:12:30"
},{
"id": 1,
"email": "[email protected]",
"status": "complete",
"created_at": "2012-08-05 04:51:02"
}]Now that you're comfortable using a for loop to import multiple lines of data, it's time to request and import real data.
Task:
- Part One: Write a script which requests indegoBike data and post it to the RJMetrics API. The data contains several nests. We are only looking to import data contained in properties.
- Part Two: Import the data into RJMetrics as a new table
- Part Three: Amend the script to add the current timestamp to each record. e.g., "time": "2017-02-24 00:00:00"
This will take you more time than previous assignments
Hint: For the third bullet, you will need to set a primary key. Will any of the columns work?
What if you wanted to complete the task of Homework 4 at each hour of the day?
This is where you will need to set up a cron job
Task:
Find out more about cron jobs on Google and set one up through the terminal Have it run at each hour
Note: This will not work if you've been using a notebook style editor such as Jupyter Notebooks. You will need to execute python files (.py) through the terminal.
A raw data export is useful for clients as they can view data in RJMetrics as a .csv. The issue is that a raw export only takes a snapshot of the table based on when the export was created.
We've already learned how to export an exisiting raw data export using the export API. Can you figure out how to generate 'fresh exports' using the export API? Use the current Help Center article for clues.
Task:
- Part One: Create an initial data export in the front end
- Part Two: Send a POST request to the export API to generate an updated copy of the initial data export
- Part Three: Obtain the updated export's export_id
- Part Four Send a POST request containing the new export_id and download the zip file
Note: There may be a delay for the newly created export ID to generate. How do you account for this when trying to complete this assignment in one python file?
Your code should work seemlessly after entering the intial table id (i.e. not having to insert the export id each execution)
Now that you have downloaded the new export as a zip and extracted the csv, write a script that imports the csv into python and transform it into a JSON format.
The application here is streamlining a way for clients to extract data out of Magento BI into another integration they use.
Many integrations (RJMetrics included) place a cap on the maximum number of requests that can be sent at a time. Can you come up with a way to get around this by sending requests in batches?
You should be pretty comfortable now importing and exporting data to/from APIs. Now try exporting a revenue report and extract the date and revenue values.
The final format of the extracted data should be as a list of dictionaries.
Export the same revenue report but this time don't use a raw export. Directly export the report into python and then use either R or Python to forecast the data.
Then send it back to RJMetrics.