A hybrid recommendation system for scientific papers that combines semantic embeddings, knowledge graphs, and user behavior analysis to provide personalized research paper recommendations.

• Hybrid Recommendation Engine: Combines content-based filtering using SciBERT embeddings with collaborative filtering
• Knowledge Graph Integration: Uses Neo4j to store and query relationships between papers, concepts, and authors
• Semantic Search: Leverages SciBERT embeddings for semantic similarity between research papers
• Ontology-based Recommendations: Integrates scientific concept hierarchies for better recommendations
• Modern Web Interface: Clean, responsive UI built with Flask and TailwindCSS
• Real-time Recommendations: Instant paper suggestions based on topics and user preferences

┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│   Flask Web App │────│  Recommendation │────│   Neo4j Graph   │
│                 │    │     Engine      │    │    Database     │
└─────────────────┘    └─────────────────┘    └─────────────────┘
         │                       │                       │
         │                       │                       │
    ┌─────────┐          ┌──────────────┐        ┌──────────────┐
    │   UI    │          │   SciBERT    │        │   OpenAlex   │
    │Templates│          │  Embeddings  │        │     API      │
    └─────────┘          └──────────────┘        └──────────────┘

• Python 3.8+
• Neo4j Database (Desktop or Cloud)
• Git
• 8GB+ RAM (for SciBERT embedding generation)

git clone https://github.com/anVSS1/Scientific-Article-Recommender.git
cd Scientific-Article-Recommender

# Create virtual environment
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

1. Download Neo4j Desktop: https://neo4j.com/download/
2. Create a new database
3. Set password (remember this!)
4. Start the database

# Copy environment template
cp .env.example .env

# Edit .env file with your settings:
NEO4J_URI=bolt://localhost:7687
NEO4J_USER=neo4j
NEO4J_PASSWORD=your_actual_password
FLASK_DEBUG=True
FLASK_PORT=5050

⚠️ This repository contains NO data files - you need to fetch and generate your own data.

1. Edit the notebook: existing_scripts/openalec-fetcher.ipynb
2. Add your email (required by OpenAlex API):

   EMAIL_FOR_OPENALEX = "[email protected]"  # Replace this!

3. Configure domains you want to fetch:

   DOMAINS = [
       ("Computer Science", "https://openalex.org/C41008148", 1400),
       ("Artificial Intelligence", "https://openalex.org/C154945302", 400),
       ("Physics", "https://openalex.org/C121332964", 200),
       # Add more domains as needed
   ]

4. Run the notebook to fetch articles and concepts

All scripts have hardcoded paths that you MUST change to match your setup:

# Change these paths to match your data location:
articles_file = "data/cleaned data/processed_articles.json"  # Update path
concepts_file = "data/cleaned data/processed_concepts.json"  # Update path
output_dir = "data/embeddings/"  # Update path

# Update these paths:
articles_file = "data/cleaned data/processed_articles.json"
concepts_file = "data/cleaned data/processed_concepts.json"

# Update these paths:
articles_file = "data/cleaned data/processed_articles.json"
concepts_file = "data/cleaned data/processed_concepts.json"
output_file = "data/fake_user_logs.csv"

# Update paths in the Flask app:
embeddings_file = "data/embeddings/embeddings_articles.csv"
concepts_embeddings_file = "data/embeddings/embeddings_concepts.csv"

# Generate SciBERT embeddings for your articles
python existing_scripts/generate_embeddings.py

Note: This process can take several hours depending on your dataset size and hardware.

# Import articles and concepts to Neo4j
python existing_scripts/import_data_to_neo4j.py

# Load embeddings to Neo4j
python existing_scripts/load_embeddings_to_neo4j.py

# Generate fake user data for testing
python existing_scripts/fake_user_generator.py

# Create user profiles in Neo4j
python existing_scripts/populate_user_profiles_neo4j.py

cd Website
python app.py

Navigate to http://localhost:5050

Scientific-Article-Recommender/
├── Website/                          # Flask web application
│   ├── app.py                       # Main Flask app
│   ├── backend/reco.py              # Recommendation engine
│   └── templates/                   # HTML templates
├── existing_scripts/                # Data processing scripts
│   ├── openalec-fetcher.ipynb      # Fetch data from OpenAlex
│   ├── generate_embeddings.py       # Generate SciBERT embeddings
│   ├── import_data_to_neo4j.py     # Import to Neo4j
│   ├── load_embeddings_to_neo4j.py # Load embeddings
│   ├── fake_user_generator.py      # Generate test users
│   └── populate_user_profiles_neo4j.py # Setup user profiles
├── data/                           # YOUR DATA GOES HERE
│   ├── fetched data/               # Raw OpenAlex data
│   ├── cleaned data/               # Processed articles/concepts
│   └── embeddings/                 # Generated embeddings
├── requirements.txt                # Python dependencies
├── .env.example                    # Environment template
└── README.md                      # This file

Edit openalec-fetcher.ipynb to fetch different scientific domains:

DOMAINS = [
    ("Your Domain", "OpenAlex_Concept_ID", target_count),
    ("Biology", "https://openalex.org/C86803240", 500),
    ("Medicine", "https://openalex.org/C71924100", 300),
]

Change the SciBERT model in generate_embeddings.py:

self.tokenizer = AutoTokenizer.from_pretrained("your-preferred-model")
self.model = AutoModel.from_pretrained("your-preferred-model")

Modify the hybrid recommendation weights in Website/backend/reco.py:

content_weight = 0.6  # Content-based filtering weight
ontology_weight = 0.4  # Ontology-based weight

1. Navigate to the main page
2. Select "Topic Search"
3. Enter: "Neural Networks", "Machine Learning", etc.
4. Get semantically similar papers

1. Select "Personalized" mode
2. Choose a user profile (generated by fake_user_generator.py)
3. Enter a search query
4. Receive recommendations based on user history

• Click "Explore Ontology" to browse concept hierarchies
• Search for concepts and explore relationships

"No module named 'neo4j'"

pip install neo4j

"Connection refused" to Neo4j

• Make sure Neo4j is running
• Check your .env file credentials
• Verify the URI (usually bolt://localhost:7687)

"File not found" errors

• Update all file paths in the scripts to match your setup
• Make sure you've run the data fetching steps

SciBERT model download fails

# Install transformers properly
pip install transformers torch

Embedding generation is slow

• Use GPU if available (install torch with CUDA)
• Reduce batch size in generate_embeddings.py
• Process smaller datasets first

• Use GPU: Install PyTorch with CUDA for faster embedding generation
• Increase RAM: 8GB+ recommended for large datasets
• SSD Storage: Faster I/O for large data processing
• Batch Processing: Adjust batch sizes based on your hardware

• Website: https://openalex.org/
• API Docs: https://docs.openalex.org/
• Rate Limits: Be respectful, use your email
• Data License: CC0 (public domain)

• Computer Science
• Artificial Intelligence
• Physics
• Biology
• Medicine
• Mathematics
• Engineering
• And many more...

1. Fork the repository
2. Create your feature branch (git checkout -b feature/AmazingFeature)
3. Update file paths in your modifications
4. Test with your own data
5. Commit changes (git commit -m 'Add AmazingFeature')
6. Push to branch (git push origin feature/AmazingFeature)
7. Open a Pull Request

This project is licensed under the MIT License - see the LICENSE file for details.

• OpenAlex for providing open scientific data
• SciBERT for scientific text embeddings
• Neo4j for graph database technology
• Flask for the web framework

Developer 1: anVSS1
Email: [email protected] LinkedIn: LinkedIn Profile
GitHub: @anVSS1

Developer 2: KAN
LinkedIn: LinkedIn Profile

Project Link: https://github.com/anVSS1/Scientific-Article-Recommender

1. No Data Included: This repository contains only code. You must fetch and generate your own data.

2. Update All Paths: Every script has hardcoded file paths that need to be updated for your system.

3. OpenAlex Email Required: You must add your email to the OpenAlex fetcher script.

4. Hardware Requirements: Embedding generation requires significant computational resources.

5. Neo4j Setup: You must install and configure Neo4j before running the application.

⭐ Star this repo if you find it helpful!