An automated job aggregation and alert system that scrapes multiple job platforms, filters relevant opportunities, removes duplicates, and sends a digest email with Excel attachment.

Job listings are scattered across multiple platforms, making it difficult to track relevant opportunities efficiently.

This system aggregates jobs from different platforms, filters them, and delivers a single consolidated digest.

• 🔍 Multi-keyword job search (Python, Backend, AI, etc.)
• 🌐 Multi-platform scraping (Internshala, Indeed, Wellfound)
• 🧠 Smart filtering (removes irrelevant roles like marketing, HR, etc.)
• 🗂️ SQLite database for deduplication
• ⏱️ Scheduler for periodic execution
• 📧 Email alerts (digest format, not spam)
• 📊 Excel report attachment

• Python
• Requests + BeautifulSoup
• Playwright (for JS-heavy sites)
• SQLite
• smtplib (Email automation)
• Schedule (cron-like automation)

job-scraper/
├── core/
│   ├── storage.py        # DB + deduplication
│   └── notifier.py       # Email alerts
├── scrapers/
│   ├── internshala.py
│   ├── indeed.py
│   └── wellfound.py
├── main.py               # Main pipeline
├── scheduler.py          # Runs periodically
├── requirements.txt
└── jobs.db               # Local database (ignored)

1. Scrapes jobs from multiple platforms
2. Filters irrelevant and duplicate jobs
3. Stores new jobs in SQLite DB
4. Sends a single digest email with all new jobs

pip install -r requirements.txt
python main.py

python scheduler.py

• Enable 2-Step Verification in Gmail
• Generate App Password
• Add credentials in notifier.py

• Job seekers looking for automated alerts
• Developers learning web scraping + automation
• Building real-world data pipelines