UniCrimeMap

Inspiration

Our inspiration was the UofT Campus Safety Activity Reports website was not set up well. Using a calendar format made it difficult to overview all the activities happening in the area. Our goal was to simplify these reports into a user-friendly visual design. Additionally, we expanded our reach to Toronto Metropolitan University as well.

What it does

UniCrimeMap takes in all incident reports dating back to the user's specified time. All these reports are compiled and displayed on a map with which the user can interact. Additionally, the user can add filters, such as time and category restrictions, to view the incidents that are a match. Essentially, UniCrimeMap gives its users a simple way to access and view all reports in the area.

How we built it

Front End We used JS, HTML and CSS with a React framework to build the front end of the project. In addition, the Google Maps API was used in order to embed a Google Maps element in our website, as well as manipulate it with our data taken from webscraping and make it interactive.

Back End We used python to build the back end of our project. The main functionality of the python files is to scrape information off the specified URLs and convert that information into the appropriate elements for the CSV file we agreed on. Additionally, we created a file that returns CrimeReport objects that match the corresponding filters.

Challenges we ran into

The biggest challenge we ran into was scraping websites with incident reports. Due to the dynamic nature of the websites, we could not access the information needed initially using the methods we already knew. The HTML data would not load until the user performed specific interactions, such as waiting for a calendar to load or clicking a button to view the next page. The solution was to use a package called selenium which allowed us to simulate opening pages and clicking on buttons to access different pages. The problem we ran into with selenium was that public methods were changed. This caused a lot of confusion as many of the tutorials and guides we found called methods that were no longer available with the same name. After a lot of struggle, the fix to this was reading the documentation for selenium and finding the methods we needed.

Accomplishments that we're proud of

Due to all of the challenges that arose, we are proud to figure out how to web scrape dynamic websites. Especially with the UofT website, there were many nuances due to the dozens of different buttons we needed to keep track of. However, after figuring it all out, it was refreshing to see our crimes.csv fill up with all the reports.

What we learned

Besides the progress we made with web scrapping, we also learned how to work with Google Maps API. We learned how to use Flask, React, and integrate a front end with the webscraping backend.

What's next for UniCrimeMap

Refractor all code to allow for ease of updates
Expand to other universities or cities
Move the filtering process to the back end. As we did not have time, we created a temporary solution in JS instead.
Access TMU incident descriptions via web scraping
Make UniCrimeMap more polished and easier to use for general users. In addition, more views outside the default map view, like databases and graphs would be helpful for analysis.