A serverless movie recommendation system built on AWS, featuring semantic search with ONNX models, collaborative filtering, and comprehensive user management capabilities.
- Semantic Search: Natural language movie search using ONNX-optimized embeddings stored in S3
- Collaborative Filtering: Personalized recommendations based on user ratings and behavior
- Content-Based Filtering: Recommendations based on movie similarity using weighted embeddings
- User Management: JWT authentication, favorites, reviews, and activity tracking
- Scalable Architecture: Single Lambda function with centralized routing and configuration
- ONNX Model Support: Optimized inference without heavy ML dependencies
The system uses a single Lambda function architecture with:
- Single Entry Point:
lambda_handler.pyroutes all API requests - Centralized Configuration:
utils/config.pymanages all environment variables - DynamoDB: 5 tables for users, movies, reviews, favorites, and activity
- S3 Storage: Movie embeddings (.npz format) and ONNX models
- HTTP API Gateway: AWS HTTP API (not REST) for endpoint management
- Lambda Layers: Optimized dependency management
- Layer 1:
PyJWT==2.8.0,bcrypt==4.1.2,onnxruntime,tokenizers - Layer 2:
numpy<1.27.0
-
Setup AWS Infrastructure
cd initial_setup python create_table.py -
Deploy Single Lambda Function
REM Package entire application powershell Compress-Archive -Path lambda_handler.py,lambda_functions,utils,requirements.txt -DestinationPath movie-recommender.zip aws lambda create-function --function-name Movie_recommender_system --runtime python3.9 --zip-file fileb://movie-recommender.zip
-
Configure Environment Variables
set JWT_SECRET=your-secret-key set EMBEDDINGS_BUCKET=movieembeddings set MODEL_BUCKET=movieembeddings
-
Setup API Gateway
cd initial_setup python api_gateway_setup.py
All endpoints are routed through a single Lambda function via HTTP API Gateway:
POST /auth/register- User registration with email/passwordPOST /auth/login- User login returning JWT tokenPOST /auth/refresh- Refresh JWT token
POST /search- Semantic search using ONNX model inferencePOST /content- Content-based recommendations from movie IDs with ratingsPOST /collaborative- Collaborative filtering (requires authentication)POST /similar- Find similar movies to a given movie
GET /user-data/favorites- Get user's favorite moviesPOST /user-data/favorites- Add movie to favoritesDELETE /user-data/favorites/{movieId}- Remove from favoritesGET /user-data/favorites/toggle/{movieId}- Check if movie is favoritedPOST /user-data/reviews- Add movie review with ratingGET /user-data/reviews- Get user's reviewsDELETE /user-data/reviews/{movieId}- Remove reviewGET /user-data/reviews/toggle/{movieId}- Check if movie is reviewedGET /user/activity- Get user activity historyDELETE /user/account- Delete user account
- Format:
.npz(NumPy compressed archive) - Structure: 385 columns (384 float embeddings + 1 string movie_id)
- Storage: S3 bucket
movieembeddings
- Model Files:
config.json,model.onnx,tokenizer.json,vocab.txt - Storage: S3 bucket
movieembeddings/model_onnx/ - Purpose: Optimized inference without sentence-transformers dependency
- Movies - Movie metadata (title, overview, genres, etc.)
- Reviews - User ratings and reviews (user_id, movie_id, rating, timestamp)
- MovieRecommender_Users - User authentication (email, username, password_hash, salt)
- MovieRecommender_Favorites - User favorites (user_id, movie_id, created_at)
- MovieRecommender_Activity - User activity logs (user_id, timestamp, action, data)## Development
For detailed setup instructions, see:
- DEPLOYMENT_GUIDE.md - Complete deployment walkthrough
- API Reference - OpenAPI 3.0 specification
- System Architecture - Detailed component overview
All environment variables are managed centrally in utils/config.py:
# Required
JWT_SECRET = "your-jwt-secret"
EMBEDDINGS_BUCKET = "movieembeddings"
# Optional (with defaults)
JWT_EXPIRY = 7200 # 2 hours
MAX_RESULTS = 100
DEFAULT_TOP_K = 10Download the following CSV files from Kaggle and place them in the initial_setup directory:
movies_metadata.csvcredits.csvratings_small.csv(orratings.csvfor full dataset)
Kaggle dataset: https://www.kaggle.com/datasets/rounakbanik/the-movies-dataset
The system uses Lambda Layers for optimized dependency management:
PyJWT==2.8.0
bcrypt==4.1.2
onnxruntime
tokenizers
numpy<1.27.0
PyJWT==2.8.0
bcrypt==4.1.2
numpy<1.27.0
onnxruntime
tokenizers
All the endopoints are tested on AWS lambda using JSON files in the tests directory. Each test file corresponds to a specific endpoint and includes sample requests and expected responses.
They are also tested using curl commands for remote testing. For example:
curl -X POST http://localhost:3000/auth/login -H "Content-Type: application/json" -d '{"email": "[email protected]", "password": "password"}'
curl -X POST http://localhost:3000/search -H "Content-Type: application/json" -d '{"query": "An action movie with a female superhero"}'
curl -X GET http://localhost:3000/user-data/favorites -H "Authorization: Bearer <token>"MIT License