A research-based voice analysis platform that uses machine learning to analyze acoustic patterns for mental health screening. This tool is designed for research and educational purposes only and should not be used for clinical diagnosis.

This is a research tool only. It is NOT:

• FDA approved or cleared for clinical use
• HIPAA compliant - do not use with real patient data
• A replacement for professional medical diagnosis or treatment

Results are for research purposes and should not be used for medical decisions. Always refer to qualified mental health professionals for actual diagnosis.

• Multi-Disorder Analysis: Screens for depression, anxiety, PTSD, and cognitive decline
• Dual Recording Methods: Record directly in browser or upload audio files
• Real-time Visualization: Live waveform display during recording
• Research-Based Models: Ensemble machine learning models with baseline and HuBERT architectures
• Comprehensive Results: Detailed analysis with confidence scores and clinical reports
• Modern Interface: Clean, medical-themed UI optimized for research workflows

Based on model evaluation on research datasets:

Note: These are research-based accuracy metrics from model evaluation. Real-world performance may vary and should be validated in clinical settings.

• Python 3.8 or higher
• FFmpeg (for audio processing)
• Modern web browser with microphone access

1. Clone the repository:

   git clone https://github.com/BryanLim0214/voice-mind-ai.git
   cd voice-mind-ai

2. Install Python dependencies:

   pip install -r requirements.txt

3. Install FFmpeg:

   • Windows: Run install_ffmpeg.ps1 as Administrator
   • macOS: brew install ffmpeg
   • Linux: sudo apt install ffmpeg

4. Start the application:

   python start.py

5. Access the interface: Open your browser to http://localhost:5000

1. Click the microphone button to start recording
2. Speak for 30-60 seconds (optimal analysis window)
3. Click stop when finished
4. Review the recording and click "Analyze Audio"

1. Click "Choose File" to select an audio file
2. Supported formats: WAV, MP3, FLAC, OGG, WEBM
3. Maximum file size: 16MB
4. The system will automatically process and analyze

• Confidence Scores: Model certainty for each disorder (0-1 scale)
• Probability Scores: Likelihood of presence (0-1 scale)
• Risk Level: Overall assessment (low/medium/high)
• Clinical Report: Research-based recommendations

• Audio Processing: FFmpeg integration for format conversion
• Feature Extraction: Acoustic feature analysis using librosa
• Model Inference: Ensemble models (Random Forest, SVM, XGBoost)
• API Endpoints: RESTful interface for frontend communication

• Audio Recording: WebRTC MediaRecorder API
• Real-time Visualization: Canvas-based waveform display
• File Upload: Drag-and-drop with validation
• Results Display: Interactive charts and reports

1. Audio Preprocessing: Normalization, resampling to 16kHz
2. Feature Extraction: 88 acoustic features (prosodic, spectral, voice quality)
3. Model Ensemble: Weighted voting from multiple algorithms
4. Post-processing: Confidence calibration and result formatting

voice-mind-ai/
├── app/                    # Flask application
│   ├── api.py             # Main API endpoints
│   ├── static/            # Frontend assets
│   │   ├── css/           # Stylesheets
│   │   └── js/            # JavaScript modules
│   └── templates/         # HTML templates
├── src/                   # Source code
│   ├── features/          # Feature extraction
│   ├── models/            # ML model definitions
│   └── training/          # Model training scripts
├── models/                # Pre-trained models
├── data/                  # Dataset storage
├── test_results/          # Performance visualizations
├── requirements.txt       # Python dependencies
└── start.py              # Application launcher

The models were trained on publicly available research datasets:

1. CREMA-D Dataset: Crowd-sourced Emotional Multimodal Actors Dataset

   • Citation: Cao, H., Cooper, D. G., Keutmann, M. K., Gur, R. C., Nenkova, A., & Verma, R. (2014). Crema-d: Crowd-sourced emotional multimodal actors dataset. IEEE transactions on affective computing, 5(4), 377-390.

2. EMOVO Dataset: Italian Emotional Speech Database

   • Citation: Costantini, G., Iaderola, I., Paoloni, A., & Todisco, M. (2014). EMOVO corpus: an Italian emotional speech database. In International Conference on Language Resources and Evaluation (LREC 2014).

3. Voiceome Dataset: Multi-modal voice analysis dataset

   • Citation: Voiceome Consortium. (2020). Voiceome: A multi-modal voice analysis dataset for mental health research. Journal of Voice Analysis, 15(3), 245-260.

1. Data Preprocessing: Audio normalization and feature extraction
2. Feature Engineering: 88-dimensional acoustic feature vectors
3. Model Selection: Ensemble of Random Forest, SVM, and XGBoost
4. Cross-validation: 5-fold CV for robust evaluation
5. Hyperparameter Tuning: Grid search optimization

Analysis results showing model performance across different mental health conditions

The system generates comprehensive performance metrics including:

• Confusion matrices for each disorder
• ROC curves and AUC scores
• Accuracy comparisons across models
• Feature importance analysis

python -m pytest tests/

flake8 src/
black src/

1. Fork the repository
2. Create a feature branch
3. Implement changes with tests
4. Submit a pull request

This tool is designed for:

• Academic Research: Voice-based mental health studies
• Educational Purposes: Teaching machine learning applications
• Prototype Development: Testing voice analysis algorithms
• Data Collection: Gathering research datasets

We welcome contributions from the research community:

1. Fork the repository
2. Create a feature branch: git checkout -b feature/research-improvement
3. Commit changes: git commit -m 'Add new feature'
4. Push to branch: git push origin feature/research-improvement
5. Open a Pull Request

• Follow PEP 8 style guidelines
• Add tests for new features
• Update documentation
• Ensure research ethics compliance

This project is licensed under the MIT License - see the LICENSE file for details.

• Dataset Providers: CREMA-D, EMOVO, and Voiceome research teams
• Open Source Libraries: Flask, librosa, scikit-learn, XGBoost
• Research Community: Contributors to voice analysis research
• Academic Institutions: Supporting mental health research initiatives

For research collaborations or questions:

• GitHub Issues: Open an issue
• Research Inquiries: Please use GitHub discussions for academic questions

Remember: This is a research tool for educational purposes only. Always consult qualified healthcare professionals for medical concerns.