JalanKami

Cover page
layout
scan
generate

Inspiration

Getting to places in Jakarta without a vehicle is near impossible. While public transportation has greatly improved in recent years, the glaring lack of sidewalks have rendered many areas unsafe to walk through. Only 8.7% of Jakarta's existing 7,000~ km of roads are lined with sidewalks, and an even smaller percentage represent those that are accessible and pedestrian-friendly.

Governments have relied on manual surveys to determine which areas do have sidewalks, and furthermore, the accessibility and quality of such areas (whether they are uneven, lack tactile bricks, and so on). However, this can prove to be time-consuming and decrease government initiative to invest on better sidewalk systems.

By using images from Google Streetview coupled with community input, JalanKami aims to support government and community initiatives for sidewalk development, which also has further environmental benefits by reducing carbon footprint.

What it does

JalanKami has four main functionalities:

Interactive map that allows users to view detailed information of walkability and accessibility in particular areas using detections done previously.
Community issue forum: Inspired by a similar government web app, JalanKita, users can add issues in a forum-style interface for infrastructure reports and suggestions.
Area summarizer tool: Users can select a location and radius, and generate a summary of the area’s key issues as well as insights to improve accessibility and walkability.
Sidewalk detection tool: Users can select a point to automatically gauge its walkability by detecting sidewalks and common accessibility features such as tactile paths.

How we built it

Frontend: React + Vite, Google Maps API

Backend: Flask, Firebase, OpenAI API, YOLOWorld

The pipeline was to first get the street view images of that specific location, save it to Firebase Storage, and then pass in the image to the fine-tuned YOLOWorld model. If a sidewalk was detected, we pass in the image again and detect for faults and accessibility features. We then save the detection object to Firebase and render it on the map. The forum was simple and implemented using react-hook-form and Firebase. The area summary tool queried detections based on geohash and passed relevant docs to the OpenAI client.

Challenges we ran into

Lack of comprehensive documentation for YOLOWorld
Transitioning from a full-stack framework like Next.js to React, Vite, and Flask
Had to retrain the YOLOWorld model on a custom dataset for it to produce useful results.
I owe openai $5 now

What we learned

How to integrate and fine-tune production-ready models
36-hour Flask crash course
How to create a custom dataset
How to code in traffic

Accomplishments that we're proud of

Getting everything except the forum to work in time

What’s next?

Applications:

can be expanded to include various cities in Indonesia, most of which still lack sidewalk infrastructure and accommodations for disabled citizens.
would love to fully implement a camera scanner which can be used to educate people on what accessibility features are necessary and should be implemented for sidewalk systems.

Technical:

Given more time I would go for a more specialized model and a different data pipeline. Ideally the image can be passed through a segmentation model, and we can crop that area (and everything above that) to detect faults and obstacles in the sidewalk.
Implementing sidewalk connectivity in this app through using aerial view images in detection (which can detect connected sidewalk systems rather than just individual spots). This way, local governments are able to see the density of sidewalks in one given area and fill in gaps where needed most.
Making the website fully desktop-compatible.