Forecasting San Francisco 311 / 911 safety-related service demand at the neighborhood level using time-aware regression models.

• Overview
• Methodology
• Analytical Techniques and Tools
• Feature Engineering
• Modeling Approach
• Results
• Project Setup
• Repository Structure

City services routinely need to respond to safety events which range from disturbances, including noise, illegal parking, graffiti, and curb problems. This project will predict the number of different types of 311 events in San Francisco per neighborhood and for a given date, based on historical data. Accurate models will help ensure that enough customer service agents and resources are available and able to respond promptly to these requests.

Our dataset is based on San Francisco 311 and 911 service call data ranging from 2016 to 2020. We develop and evaluate time-aware regression models to forecast aggregated San Francisco 311/911 safety-related service demand at the neighborhood level. These models assess whether historical call data, combined with temporal, weather, holiday, and operational schedule features, can produce reliable short-horizon demand forecasts suitable for planning and resource allocation. Forecasting is performed at a fixed 6-hour interval across selected neighborhoods and service categories.

• Analyze available events data to identify trends, impact of seasonality, and consistency of data.
• Build machine learning models to predict the number of events of a given type for a particular date and location.
• Use external datasets to improve model performance.
• Make predictions more explainable by using model interpretability tools.
• Prevent undesired biases or discrimination when models are put into production.
• Evaluate model performance during city-wide lockdowns during the COVID-19 pandemic.

• Cleaning, filtering and pre-processing the data loaded from Parquet files
  • pandas
• Analysis and visualization of location-based and time-dependent data
  • pandas
  • matplotlib
• Machine Learning model training, evaluation, hyper-parameter tuning
  • scikit-learn
• Python package creation
  • setuptools
• Evaluating of bias risk and ethical considerations
  • deon

We have several different service categories for 311 / 911 city event records:

• Street and Sidewalk Cleaning
• Potentially Life-Threatening
• Graffiti
• Encampments
• Non Life-threatening

For each service category we:

1. Filter events by category
2. Aggregate counts by:
   1. Neighborhood
   2. Six-hour interval
3. Construct a complete neighborhood and time grid
4. Fill the missing time intervals with zeroes to ensure consistent time indexing and avoid missing data

• Year, Month, Day
• Day of Week
• Hour of Day

• Daily weather variables
  • NOAA station data
• Federal and California holiday indicators
• Street sweeping schedule intensity by neighborhood

• Create a baseline model

  • DummyRegressor (mean)

• Generate anad assess several candidate models

  • Ridge Regression
  • Poisson Regressor
  • HistGradientBoostingRegressor

• Validation Strategy

  • Time-ordered train/test split
  • Forward-chaining cross-validation
  • Hyperparameter tuning using GridSearchCV / RandomizedSearchCV

Performance varies by service category and neighborhood.

Evaluation includes:

• RMSE
• Mean Absolute Error (MAE)
• Maximum Error
• Mean Residual (Directional Bias)
• Neighborhood-level residual time series

The models created in this project were evaluated using RMSE, MAE, and neighborhood-level residual analysis, predicting the count of events in 6-hour bins per neighborhood. A historical-mean baseline (DummyRegressor) was used as a benchmark. Data was split in a non-shuffling manner so that training explained the past and testing evaluates the future. Short-horizon municipal service demand can be forecasted with some accuracy when time patterns and external signals are incorporated. Gradient boosting models consistently outperformed linear and baseline approaches, indicating that service demand exhibits nonlinear relationships with time, weather, and added operational features.

• Gradient boosting consistently outperformed baseline models.

The baseline model simplistically predicts the historical mean, producing high errors with negative or near-zero R² across categories. The tuned HGBR model substantially lowers RMSE and MAE on the test set and produces consistently positive R² values. For example, Street and Sidewalk Cleaning test RMSE drops from about 316 under the baseline to about 70 with HGBR, with R² improving from negative to 0.77. Similar improvements appear across all categories, indicating that the gradient boosting model captures meaningful temporal and contextual structure.

• Service demand generally followed predictable temporal cycles, particularly by hour of day and day of week. Service calls had their highest volume, regardless of category, around 9:00 AM and on Monday, following traditional business hours.

• Mobile/Open311 was the predominant source of events, with Phone or Web events being distant second (or third) in volume. Events such as COVID-19 and its associated lockdowns mean the city loses significant amounts of crowdsourced monitoring capacity. At the same time, lessened monitoring does not mean that the underlying facts on the ground reduced or increased in volume.

The "Unknown" nature of Potentially Life-Threatening and Non Life-Threatening points to systemic encoding issue; events of this nature imply threat to human life and limb, while the other categories seems based on non-urgent events.

• Forecast accuracy varies by neighborhood, highlighting the need for neighborhood-level, localized, model evaluations instead of generalization based on city-wide metrics. The heterogenous nature of our model predictions within different event categoires imply features unexplored, such as employment type, foot and vehicle traffic patterns, and property types, all of which could impact the volume and classification of reporting.
• Applied to 2020 data, models trained on earlier years exhibited concept drift. After COVID-19 lockdown events, predicted and observed counts for city events diverged in interesting ways. For example, in the SoMA district the predicted number of Street and Sidewalk Cleaning events steadily rose, while the actual number observed dropped significantly and flattened. Lockdown events combined with work-from-home company policies may have reduced the number of people available to observe and report city events for category.

This behavior contrasts with the Mission district, where the actual number of events reported exceeds model-predicted amounts. Lockdown directives altered working and living patterns, which may unintentionally have enabled heightened monitoring in this district due to increased food vendor traffic, due to workplace cafeterias being unavailable, and larger amounts of delivery drivers in this area.