This project demonstrates advanced web scraping techniques combined with modern DevOps practices. Built with Python and Selenium, it intelligently extracts LinkedIn profile data from Google search results while implementing anti-detection measures and running autonomously in the cloud.

Perfect for:

• 🎯 Recruitment & HR professionals seeking candidate data
• 📊 Market researchers analyzing professional demographics
• 🔍 Lead generation and business development
• 💼 Career coaches building industry insights
• 🚀 Developers learning automation & web scraping

• 🔍 Intelligent Search - Automated Google search with customizable queries
• 🎭 Anti-Detection - Randomized user agents, delays, and stealth mode
• 📊 Data Export - Clean CSV output with timestamps
• 🔄 Fresh Data - Each run fetches new results, not cached data
• ⚡ Efficient Scraping - Optimized selectors and error handling

• ☁️ GitHub Actions Integration - Serverless execution in the cloud
• 🔄 Manual Trigger - On-demand workflow execution
• 📦 Artifact Storage - Automatic result archiving (30 days)
• 🐳 Docker Support - Containerized for easy deployment
• 🔒 Secure - No credentials stored, environment-based configuration

• 📝 Unique File Naming - Timestamped files for each run
• 🎯 Customizable Queries - Search any LinkedIn profile type
• 🛡️ Error Recovery - Graceful handling of failures
• 📈 Scalable Architecture - Easy to extend and modify
• 🔧 Production Ready - Robust error handling and logging

Actions → LinkedIn Scraper Bot → Run workflow
├── 🔍 Custom Search Query: "site:linkedin.com/in/ data scientist"
├── ⏰ Use Timestamp: true/false
└── ▶️ Run workflow

title,link,scraped_at
"John Doe - Senior Software Engineer",https://linkedin.com/in/johndoe,2026-01-19T14:30:25
"Jane Smith - Data Scientist at Google",https://linkedin.com/in/janesmith,2026-01-19T14:30:27
"Michael Johnson - Full Stack Developer",https://linkedin.com/in/michaelj,2026-01-19T14:30:29

linkedin-scraper-bot/
├── 🐍 bot.py                      # Main scraper with anti-detection
├── 📦 requirements.txt            # Python dependencies
├── 🐳 Dockerfile                  # Container configuration
├── 🔧 runtime.txt                 # Python version specification
├── 📄 README.md                   # This file
├── 🚫 .gitignore                  # Git ignore rules
└── 📂 .github/
    └── workflows/
        └── ⚙️ scraper.yml         # GitHub Actions workflow

• Python 3.11+
• Google Chrome or Chromium
• Git

1. Clone the repository

   git clone https://github.com/EHTISHAM-AI-ENTHUSIAST/linkedin-scraper-bot.git
   cd linkedin-scraper-bot

2. Install dependencies

   pip install -r requirements.txt

3. Run the bot

   # Default search query
   python bot.py
   
   # Custom search query
   set SEARCH_QUERY=site:linkedin.com/in/ python developer
   python bot.py
   
   # With visible browser (for debugging)
   set HEADLESS=false
   python bot.py

# Build the image
docker build -t linkedin-scraper .

# Run the container
docker run -e SEARCH_QUERY="site:linkedin.com/in/ AI engineer" linkedin-scraper

1. Navigate to Actions

   • Go to your repository on GitHub
   • Click the Actions tab
   • Select LinkedIn Scraper Bot workflow

2. Run Workflow

   • Click Run workflow button
   • Search Query: Enter your custom search (e.g., site:linkedin.com/in/ UX designer)
   • Use Timestamp: Choose true for unique files, false for overwriting
   • Click Run workflow

3. Download Results

   • Wait for workflow to complete (~2-5 minutes)
   • Scroll down to Artifacts section
   • Download linkedin-profiles-{run_number}.zip
   • Extract and open CSV file

# Set environment variables
export SEARCH_QUERY="site:linkedin.com/in/ machine learning engineer"
export HEADLESS=true
export USE_TIMESTAMP=true

# Run the scraper
python bot.py

# Find Python developers
SEARCH_QUERY="site:linkedin.com/in/ python developer"

# Find people at specific company
SEARCH_QUERY="site:linkedin.com/in/ Google software engineer"

# Find by location
SEARCH_QUERY="site:linkedin.com/in/ designer San Francisco"

# Find by title and skills
SEARCH_QUERY="site:linkedin.com/in/ DevOps AWS kubernetes"

graph LR
    A[Manual Trigger] --> B[GitHub Actions]
    B --> C[Setup Python 3.11]
    C --> D[Install Chrome]
    D --> E[Install Dependencies]
    E --> F[Run Scraper Bot]
    F --> G{Success?}
    G -->|Yes| H[Save CSV]
    G -->|No| H
    H --> I[Upload Artifact]
    I --> J[Commit to Repo]
    J --> K[Complete]

• ✅ No credentials required - Uses public Google search
• ✅ Rate limiting - Built-in delays to avoid detection
• ✅ Error handling - Graceful failures, no data corruption
• ✅ Anti-detection - Randomized user agents & human-like behavior
• ✅ GitHub secrets ready - Easy integration with private APIs
• ✅ GDPR compliant - Public data only

• 🔄 Proxy rotation support
• 📧 Email notifications on completion
• 💾 Database integration (PostgreSQL/MongoDB)
• 🌐 REST API wrapper
• 📱 Mobile app integration
• 🤖 AI-powered profile analysis
• 📊 Dashboard & analytics

Contributions are welcome! Please feel free to submit a Pull Request. For major changes, please open an issue first to discuss what you would like to change.

1. Fork the repository
2. Create your feature branch (git checkout -b feature/AmazingFeature)
3. Commit your changes (git commit -m 'Add some AmazingFeature')
4. Push to the branch (git push origin feature/AmazingFeature)
5. Open a Pull Request

This project is licensed under the MIT License - see the LICENSE file for details.

I offer professional web scraping and automation services:

• 🎯 Custom Scraper Development - Tailored solutions for your needs
• ☁️ Cloud Automation - GitHub Actions, AWS Lambda, Azure Functions
• 🔄 Data Pipeline Development - ETL processes and integrations
• 🤖 Bot Development - Telegram, Discord, WhatsApp bots
• 📊 Data Analysis - Python, Pandas, visualization

Available for freelance projects and consulting.

If this project helped you, please consider giving it a ⭐️ on GitHub!