PINTARTS is an innovative art recommendation platform that combines vector-based similarity search with social features to help users discover artwork tailored to their unique tastes. Built for the PINUS Hackathon, the system leverages AI embeddings and collaborative filtering to create personalized art experiences.
- Overview
- Key Features
- Technology Stack
- Architecture
- Core Features Explained
- Setup Instructions
- API Documentation
PINTARTS is a cross-platform mobile application that revolutionizes how users discover and collect artwork. By analyzing user preferences through AI-powered embeddings (CLIP model), the platform provides intelligent recommendations that evolve with user taste. The system supports social discovery through "taste blending" - allowing users to explore artwork that combines preferences from multiple users.
- Personalized Discovery: Use AI to understand and match user art preferences
- Social Exploration: Enable collaborative taste discovery through user blending
- Seamless Experience: Provide intuitive mobile-first interface built with React Native
- Scalable Architecture: Leverage vector databases for efficient similarity search
- Uses OpenAI's CLIP model to generate 512-dimensional embeddings
- Vector similarity search using pgvector for efficient matching
- Text-to-image semantic search capabilities
- Dynamic taste profile that evolves with user interactions
- Combine taste profiles from two users to discover hybrid recommendations
- Explore artwork that appeals to multiple aesthetic preferences
- Social discovery mechanism for finding like-minded art enthusiasts
- Save favorite artworks to personal wishlist
- Track artwork preferences across sessions
- Toggle wishlist status with real-time updates
- Natural language search for artwork descriptions
- "Find paintings with vibrant colors and abstract patterns"
- Text queries converted to embeddings for similarity matching
- Calculate taste similarity between users using cosine similarity
- Discover users with similar art preferences
- Social features based on taste compatibility
- Framework: FastAPI (Python)
- Database: PostgreSQL with pgvector extension
- AI/ML:
- OpenAI CLIP (clip-vit-base-patch32)
- PyTorch for model inference
- Transformers library
- ORM: SQLAlchemy with raw SQL for vector operations
- Authentication: Password hashing with bcrypt
- Container: Docker & Docker Compose
- Framework: React Native with Expo
- Navigation: React Navigation (Stack & Bottom Tabs)
- Language: TypeScript
- UI Components: Custom components with gesture handling
- State Management: React hooks
- API Client: Axios for HTTP requests
- Vector Database: PostgreSQL with pgvector (HNSW indexing)
- Containerization: Docker
- API Documentation: Swagger/OpenAPI (FastAPI auto-generated)
┌─────────────────────────────────────────────────────────────┐
│ FastAPI Application │
├─────────────────────────────────────────────────────────────┤
│ │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ Users │ │ Recommender │ │ Blend │ │
│ │ Router │ │ Router │ │ Router │ │
│ └──────┬───────┘ └──────┬───────┘ └──────┬───────┘ │
│ │ │ │ │
│ ┌──────────────┐ ┌──────────────┐ │
│ │ Wishlist │ │ CRUD │ │
│ │ Router │ │ Layer │ │
│ └──────┬───────┘ └──────┬───────┘ │
│ │ │ │
│ └──────────────────┴─────────────────┐ │
│ │ │
│ ┌─────────────────────────▼──────┐ │
│ │ Database Layer (SQLAlchemy) │ │
│ └─────────────────────────────────┘ │
└──────────────────────────────────┬───────────────────────────┘
│
┌──────────────▼──────────────┐
│ PostgreSQL + pgvector │
│ │
│ • Users (taste_vector) │
│ • Paintings (embedding) │
│ • Wishlists │
│ • HNSW Index for vectors │
└──────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ AI/ML Pipeline │
├─────────────────────────────────────────────────────────────┤
│ │
│ User Preferences ──► CLIP Text Encoder ──► 512D Vector │
│ │
│ Painting Images ──► CLIP Vision Encoder ──► 512D Vector │
│ │
│ Search Query ──► CLIP Text Encoder ──► Vector Search │
│ │
│ Blend = (Vector1 + Vector2) / 2 ──► Recommendations │
│ │
└─────────────────────────────────────────────────────────────┘
1. API Layer (FastAPI Routers)
- Users Router (
/users): User registration, login, profile management, taste profile updates - Recommenders Router (
/recommend): Personalized recommendations, semantic search - Blend Router (
/blend): Taste blending between users, collaborative discovery - Wishlist Router (
/wishlist): Add/remove artworks, view saved items
2. Business Logic Layer (CRUD)
- User management with password hashing
- Vector operations for similarity search
- Wishlist toggle operations
- Taste profile calculations
3. AI/ML Layer (recsys/utils.py)
- CLIP Model Integration: Text and image embedding generation
- Vector Operations: Cosine similarity for matching
- Embedding Pipeline:
- Text preferences → 512D taste vector
- Image URLs → 512D painting embeddings
- Search queries → vector search
4. Data Layer
- PostgreSQL with pgvector extension
- Vector similarity using
<=>operator (cosine distance) - HNSW indexing for efficient nearest neighbor search
┌─────────────────────────────────────────────────────────┐
│ React Native App │
├─────────────────────────────────────────────────────────┤
│ │
│ App.tsx (Root) │
│ │ │
│ └── NavigationContainer │
│ │ │
│ └── RootNavigator (Stack) │
│ │ │
│ ├── SplashScreen │
│ ├── LoginScreen │
│ ├── OnboardingScreen │
│ │ │
│ ├── MainApp (Tab Navigator) │
│ │ │ │
│ │ ├── DiscoverScreen (Feed) │
│ │ ├── WishlistScreen (Saved) │
│ │ ├── BlendScreen (Social) │
│ │ └── ProfileScreen (Settings) │
│ │ │
│ ├── ArtDetailScreen (Modal) │
│ └── ARPreviewScreen (Full) │
│ │
└─────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────┐
│ Component Layer │
├─────────────────────────────────────────────────────────┤
│ │
│ • LiquidGlassTabBar: Custom animated tab bar │
│ • SearchBottomSheet: Modal search interface │
│ • BlendResultsSheet: Show blended recommendations │
│ │
└─────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────┐
│ API Layer │
├─────────────────────────────────────────────────────────┤
│ │
│ src/api/ │
│ ├── config.ts (Base URL, Axios setup) │
│ ├── auth.ts (Login, Register) │
│ ├── users.ts (Profile, User list) │
│ ├── recommend.ts (Get recommendations, Search) │
│ └── wishlist.ts (Toggle, Get wishlist) │
│ │
└─────────────────────────────────────────────────────────┘
When a user signs up and selects preferences (e.g., "impressionist", "vibrant", "modern"):
- Preferences are combined into a text prompt
- CLIP text encoder generates a 512D embedding
- Vector is normalized and stored in PostgreSQL
- Used for all future recommendations
1. Retrieve user's taste_vector from database
2. Query paintings table with vector similarity:
ORDER BY embedding <=> user_taste_vector
3. Join with wishlists to show saved status
4. Return top N paintingsCombining two users' tastes:
blend_vector = (user1_vector + user2_vector) / 2
recommendations = query_paintings_by_vector(blend_vector)Text query to artwork:
1. User enters: "peaceful landscapes with water"
2. Generate embedding from query using CLIP
3. Search paintings by vector similarity
4. Return visually matching artworks- Python 3.8+
- Node.js 16+
- Docker & Docker Compose
- PostgreSQL (via Docker)
- Create virtual environment
python3 -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activate- Install dependencies
cd backend/app
pip install -r requirements.txt- Start PostgreSQL with pgvector
docker compose up -d- Populate database with paintings
python populate_paintings.py- Run the API server
cd ..
uvicorn app.main:app --host 0.0.0.0 --port 8000 --reload- Access Swagger documentation
http://127.0.0.1:8000/docs
- Install dependencies
cd frontend
npm install- Start Expo development server
npx expo start- Run on device
- Press
ifor iOS simulator - Press
afor Android emulator - Scan QR code with Expo Go app
Backend (backend/app/database.py):
DATABASE_URL = "postgresql://admin:password123@localhost:5432/art_recommender"Frontend (frontend/src/api/config.ts):
export const API_BASE_URL = "http://YOUR_IP:8000"POST /users - Register new user
{
"username": "artlover",
"email": "[email protected]",
"password": "secure123",
"preferences": ["impressionist", "vibrant", "landscape"]
}POST /users/login - User login
{
"username": "artlover",
"password": "secure123"
}GET /recommend/{user_id} - Get personalized recommendations
- Returns top 10 paintings based on taste vector
GET /recommend/search?query=peaceful landscapes - Semantic search
- Natural language search for artworks
GET /users?user_id={id} - Get all users with similarity scores
- Returns users sorted by taste compatibility
POST /blend?user_1_id={id1}&user_2_id={id2} - Blend two users
- Returns recommendations combining both taste profiles
POST /wishlist - Toggle painting in wishlist
{
"user_id": "uuid",
"painting_id": "uuid"
}GET /wishlist?user_id={id} - Get user's wishlist
- Returns all saved paintings
- AR Preview: View artwork in real space using device camera
- Social Feed: Share discoveries and collections
- Purchase Integration: Connect with art marketplaces
- Style Transfer: Apply artwork styles to user photos
- Collaborative Collections: Shared wishlists and galleries
Built for PINUS Hackathon 2026
This project is part of the PINUS Hackathon submission.