EcoAesthetics
Transforming street photography into urban sustainability insights
Inspiration
Urban sustainability assessment is currently subjective, time-consuming, and expensive. City planners rely on manual surveys, citizens lack tools to evaluate their neighborhoods, and policy decisions are made without comprehensive data. We were inspired by the potential to democratize urban sustainability analysis - imagine if every smartphone could become a sustainability scanner, turning street photos into actionable environmental insights. The climate crisis demands data-driven solutions, and we saw an opportunity to bridge the gap between AI technology and real-world urban planning needs.
What it does
EcoAesthetics transforms any street photograph into a comprehensive sustainability report in seconds. Users simply capture or upload a street photo, and our AI ensemble analyzes it across six key categories: Green Coverage (trees, vegetation), Walkability (sidewalks, pedestrian infrastructure), Transit Access (public transport, bike lanes), Car Dependency (parking, traffic), Building Efficiency (solar panels, green architecture), and Infrastructure (street lighting, amenities). The system provides a 0-100 sustainability score, detailed category breakdowns, and specific actionable recommendations for improvement. It's designed for city planners, officials, citizens, and researchers who need objective, scalable urban sustainability assessment. The platform provides specific sustainability scores and actionable recommendations to help city planners, officials, and citizens understand and improve their urban environment.
How we built it
We architected a full-stack solution with a React frontend and Python FastAPI backend. The frontend uses Vite, Tailwind CSS, and Framer Motion for a smooth user experience with camera integration and real-time results display. The backend orchestrates an ensemble of AI models: DETR (Detection Transformer) for object detection, DeepLab v3 for semantic segmentation, multiple YOLOv8 variants for enhanced recognition, and AWS Rekognition for cloud-based analysis. We developed a specialized Green Detection Specialist that combines color analysis, NDVI-like vegetation indexing, and multi-strategy voting for accurate vegetation assessment. The system includes intelligent fallback mechanisms, ensuring reliability even when individual models fail. We implemented comprehensive logging, error handling, and automated setup scripts for easy deployment.
Challenges we ran into
Model Integration Complexity : Combining multiple AI models (DETR, DeepLab, YOLO, AWS Rekognition) into a cohesive ensemble required careful orchestration and fallback strategies. Vegetation Detection Accuracy : Standard object detection models struggled with diverse vegetation types, leading us to develop custom color-based analysis and multi-strategy voting systems. Real-time Performance : Balancing model accuracy with processing speed required optimization techniques and smart caching. AWS Integration : Managing credentials securely while maintaining ease of use for development and deployment. Cross-platform Compatibility : Ensuring camera functionality works across different browsers and devices. Model Size Management : Large AI models (2GB+) required efficient loading and memory management strategies.
Accomplishments that we're proud of
- Successfully integrated 5+ different AI models into a cohesive, reliable system
- Created a novel Green Detection Specialist with advanced vegetation analysis capabilities
- Built an intuitive, responsive interface that makes complex sustainability data accessible to everyone
- Implemented intelligent fallback systems ensuring 99%+ uptime even with external API dependencies
- Developed a scoring system that translates technical AI analysis into actionable urban planning insights
- Created a platform that bridges the gap between advanced computer vision technology and practical sustainability assessment
What we learned
- The power of ensemble learning in computer vision applications
- The importance of robust fallback systems in production AI applications
- How to translate complex technical metrics into user-friendly sustainability insights
- The challenges and rewards of integrating multiple cloud-based AI services
- The value of interdisciplinary collaboration between AI/ML, urban planning, and environmental science
- How modern web technologies can make sophisticated AI accessible to non-technical users
What's next for EcoAesthetics
- Mobile App Development: Native iOS/Android apps for seamless field data collection
- Historical Analysis: Time-series analysis to track urban sustainability changes over time
- Community Features: User-generated sustainability maps and crowd-sourced environmental monitoring
- Policy Integration: Tools for city planners to generate sustainability reports and policy recommendations
- Advanced Metrics: Expansion to include air quality prediction, noise pollution assessment, and climate resilience scoring
- Global Expansion: Adaptation of the model to work effectively across different urban environments and climates worldwide
- API Platform: Developer APIs to enable third-party applications and urban planning tools integration
Log in or sign up for Devpost to join the conversation.