J.A.R.V.I.S

Just A Rather Very Intelligent System

AI-Powered OSINT, Profile Analysis & Intelligent Assistant Platform

J.A.R.V.I.S is a modular, full-stack Open Source Intelligence (OSINT) and AI assistant platform. It goes beyond simple profile searching — it is a comprehensive intelligence workstation that combines automated web reconnaissance, agentic AI conversations, multimodal vision analysis, biometric face matching, psychological profiling, predictive analytics, and real-time target monitoring into a single unified interface.

When a user submits a search query (a person's name), J.A.R.V.I.S orchestrates parallel scraping, API querying, and AI synthesis across 15+ social media platforms, developer ecosystems, corporate registries, data breach databases, and the open web. The result is a structured intelligence dossier complete with network graphs, geographic intelligence maps, social influence scores, sentiment analysis, and cross-validated findings.

Beyond search, J.A.R.V.I.S functions as an intelligent assistant:

• Agent Mode — An agentic AI that autonomously decides which OSINT tools to invoke during multi-turn conversations, reasoning through complex queries step by step.
• Vision Analysis — Multimodal image understanding powered by llama3.2-vision, capable of social media photo OSINT, screenshot OCR, and visual face comparison.
• Face Matching — Cross-platform biometric verification using DeepFace, comparing profile photos across discovered social accounts with confidence scoring.
• User Memory — A persistent context layer that remembers user preferences, facts, and interaction patterns across sessions, injected into every AI response for personalization.
• Watch System — Real-time monitoring of targets at configurable intervals (5 minutes to 24 hours), with automated change detection and diff reporting.
• System Control — Approval-gated command execution, application launching, and URL opening, with full audit trails.
• Plugin Architecture — Dynamic plugin discovery, toggle management, and manual execution for extending J.A.R.V.I.S capabilities.
• Health Tracking — Wellness telemetry with AI-powered suggestions and pattern detection.
• Export System — Generate classified-style PDF dossiers, structured JSON, or Maltego/i2-compatible CSV from any search result or saved profile.

The core philosophy of J.A.R.V.I.S centers on data privacy and local execution. All AI analysis runs entirely on your local machine using Ollama with dual models: qwen3:8b for text intelligence and llama3.2-vision for multimodal analysis. Queries, scraped contents, and results never leave your local network — data is persisted into a private SQLite database (or optionally PostgreSQL) under your full control.

J.A.R.V.I.S utilizes a decoupled frontend-backend architecture with 14 route groups, 22+ backend services, and a relational persistence layer.

graph TD;
    subgraph Frontend ["Next.js 15 + React 19 Frontend"]
        UI[Chat Interface] -->|Search Query + Depth| CHAT[Chat Terminal]
        UI -->|Agent Mode| AGENT[Agent Chat]
        UI -->|RAG Mode| RAG[RAG Panel]
        CHAT -->|Display| CARDS[Profile Card + Widgets]
        CARDS --> VIZ[NetworkGraph / GeoIntMap / Gauges]
        UI -->|Side Panels| PANELS[Memory / Watch / Plugins / System / Health]
    end

    subgraph Backend ["FastAPI Python Backend"]
        MW[Middleware: CORS + Rate Limit + Auth + Logging] --> ROUTES
        ROUTES[14 Route Groups / 55+ Endpoints]
        ROUTES --> ORCH[Search Orchestration]
        ORCH --> SS[Web Scraper]
        ORCH --> SC[Social Scraper]
        ORCH --> GH[GitHub API]
        ORCH --> BR[Breach Detection]
        ORCH --> DW[Dark Web Intel]
        ORCH --> CO[Company Records]
        ORCH --> GEO[GeoInt Service]
        ORCH --> FM[Face Matching]
        SS & SC & GH & BR -->|Context| AI["Ollama AI (qwen3:8b)"]
        AI -->|Structured JSON| ORCH
        ROUTES --> VIS["Vision Service (llama3.2-vision)"]
        ROUTES --> MEM[Memory Service]
        ROUTES --> WATCH[Watch Service]
        ROUTES --> SYS[System Control]
        ROUTES --> HEAL[Self-Healing Monitor]
        ROUTES --> PLG[Plugin Manager]
    end

    subgraph Persistence ["SQLite / PostgreSQL"]
        ORCH -->|SQLAlchemy ORM| DB[(profiles)]
        ORCH --> HIST[(search_history)]
        ORCH --> SNAP[(profile_snapshots)]
        MEM --> UMEM[(user_memories)]
    end

    Frontend <==>|"Axios HTTP + SSE Stream"| Backend

Exception Handling: Global handlers for RequestValidationError (422 with field-level detail) and unhandled exceptions (500 generic error) ensure consistent error response format across all 55+ endpoints.

Default: SQLite — Zero-configuration file-based database at data/jarvis.db. SQLAlchemy automatically creates all tables on first startup. No manual database setup required.

Optional: PostgreSQL 16 — For production deployments requiring concurrent access and advanced indexing. Configure via DATABASE_URL in .env. PostgreSQL is specifically useful for its native JSONB support, allowing indexed queries over the semi-structured LLM output fields (additional_info, similar_profiles, network_connections).

Both backends share identical SQLAlchemy models and are fully interchangeable by changing a single environment variable.

When a user initiates a search, J.A.R.V.I.S transitions through five pipeline stages. The depth and breadth of each stage is controlled by a user-selected Depth Configuration (1–10), which translates into concrete parameters via depth_config.py:

GitHub is the first point of contact, as many targets are software engineers or maintain public repositories.

1. Direct Lookup: Attempts an exact username query against https://api.github.com/users/{name}.
2. Fuzzy Fallback: On 404, the system queries /search/users?q={name}&per_page=1 and extracts the highest-confidence match.
3. Repository Interrogation: Once a valid user is found, /users/{username}/repos pulls the top repositories sorted by updated timestamp.
4. Contribution Analysis: Extracts language distributions, star counts, and activity patterns.
5. Context Formatting: Raw JSON is converted into structured plaintext injected directly into the LLM context window.

Direct scraping of social platforms results in HTTP 403 blocks. J.A.R.V.I.S circumvents this via search-engine proxying across 15+ platforms:

URL Unpacking: Yahoo obfuscates URLs behind redirect strings (/RU=https...). J.A.R.V.I.S decodes these payloads using urllib.parse.unquote and applies platform-specific regex with negative filters to ensure only base profile URLs are captured.

Additionally, the scraper detects mentions (not direct profiles) for platforms that resist scraping: Tinder, Bumble, and Discord.

This service extracts the biographical text required to generate a dossier.

1. Visual Authentication: Queries the Wikipedia API (en.wikipedia.org/w/api.php) with the target name. Applies Unicode normalization (unicodedata.normalize) and case-insensitive subset matching to prevent name collisions. Only on perfect overlap does it pull a profile thumbnail.
2. Multi-Vector Scraping: Creates num_query_variations distinct search queries (Name + Biography, Name + Education, Name + Career, etc.) based on depth config. Grabs URLs per query with deduplication.
3. Deep Document Parsing: Targets the top URLs (explicitly filtering out social media sites). Downloads raw DOM and recursively decomposes <script>, <style>, <header>, and <nav> tags.
4. Sanitization: Extracts semantic text (<p>, <h1>, <h2> tags), strips whitespace, and truncates to a dense character budget scaled by depth.

All formatted data from Stages 1–3 is compacted into a single prompt and sent to the local Ollama instance (qwen3:8b).

1. System Prompt: The AI is given a restrictive identity, commanded to produce structured analysis covering: biographical summary, professional trajectory, psychological profile, controversy assessment, influence network mapping, social engineering vectors, and future trajectory predictions.
2. Hallucination Prevention: The prompt explicitly instructs: "You MUST ONLY write about the exact requested person. If the search context is about a CLEARLY DIFFERENT person, you MUST IGNORE that context entirely."
3. Psychological Analysis: Generates personality trait vectors, communication style assessment, and social engineering vulnerability profiles via psychological_analysis_service.py.
4. Predictive Analysis: Produces behavioral forecasts across four time horizons (24h, 7d, 30d, 90d) with probability scores via predictive_analysis_service.py.
5. Cross-Validation: Compares data points across sources to flag inconsistencies and rate confidence levels.
6. JSON Structuring: A second lightweight AI pass restructures the raw dossier into a strictly formatted JSON payload for frontend ingestion.

After AI synthesis, parallel enrichment services activate based on depth configuration:

The Agent Mode transforms J.A.R.V.I.S from a search tool into an autonomous AI assistant. When enabled, the AI dynamically decides which OSINT tools to invoke based on the user's natural language query.

• Multi-Turn Conversations: Maintains conversation context across messages, allowing follow-up questions and iterative refinement.
• Tool Selection: The agent inspects available tools (GET /api/agent/tools) and selects the appropriate ones — search, scrape, analyze, compare — without manual intervention.
• Streaming Responses: Results stream token-by-token to the frontend via SSE, providing real-time feedback as the agent reasons and acts.

Four distinct vision modes powered by llama3.2-vision running locally via Ollama:

Cross-platform biometric verification using the DeepFace library:

• Compares profile photos discovered across social media accounts during the search pipeline.
• Returns confidence percentages and match/no-match verdicts.
• Supports multiple pose angles and lighting conditions.
• Results displayed in the FaceMatch.tsx widget with expandable comparison details.

Retrieval-Augmented Generation allows conversational Q&A over search results:

• After a search completes, users can ask follow-up questions about the discovered profile.
• The RAG system injects the full search context (dossier, social links, breach data) into the AI prompt.
• User memory context is automatically included for personalized responses.
• Responses stream in real-time via RagStreamingBubble.tsx.

A persistent context layer that makes J.A.R.V.I.S remember:

Memories are ranked by importance (1–10), support semantic keyword search, and are automatically injected into AI context for every response. Full CRUD operations available via the Memory Panel.

Real-time target monitoring with automated change detection:

• Configurable Intervals: Monitor targets every 5 minutes to 24 hours.
• Change Detection: Compares current scan results against previous snapshots, highlighting new social accounts, updated descriptions, or changed metrics.
• Bulk Management: Start, stop, or stop-all active watches from the Watch Panel.
• History: View all detected changes over time for any monitored target.

Extensible architecture for adding custom capabilities:

• Discovery: J.A.R.V.I.S scans the app/plugins/ directory on startup and registers all valid plugins.
• Toggle: Enable or disable plugins at runtime without restart.
• Execution: Manually trigger any enabled plugin from the Plugin Panel.

Approval-gated operating system interaction:

• Command Execution: Request shell commands — each enters a pending state requiring explicit user approval before execution.
• Application Launching: Open applications by name with the same approval workflow.
• URL Opening: Open URLs in the default browser, also approval-gated.
• Audit Trail: Full history of all requested, approved, denied, and executed actions.
• Security: Computer control is disabled by default (enable_computer_control = False). Must be explicitly enabled in .env.

Wellness telemetry integrated into the assistant:

• Categories: Predefined health categories (sleep, exercise, stress, nutrition, etc.).
• Recording: Log data points with timestamps from the Health Panel.
• AI Suggestions: Request AI-powered health recommendations based on recorded patterns.
• Pattern Detection: Automated analysis of health data trends over time.

Three export formats for every search result or saved profile:

Exports work on both saved profiles (by ID) and live search results (by POST payload with 1 MB body limit).

Profile evolution tracking across repeated searches:

• Snapshots: Every search creates an immutable snapshot of the discovered profile state.
• Change Reports: Compare the two most recent snapshots to see what changed — new social accounts, updated descriptions, modified metrics.
• Timeline View: Browse all historical snapshots for any previously searched person.
• Displayed via the VersionHistory.tsx widget with diff highlighting.

J.A.R.V.I.S uses 4 tables managed by SQLAlchemy ORM. All tables are auto-created on first startup — no manual schema initialization required for SQLite.

The primary data store for completed intelligence dossiers.

Tracks recent search queries with automatic 7-day expiration.

Immutable point-in-time captures for version history and change detection.

Persistent user context for AI personalization.

The Next.js frontend is a fully componentized single-page application with dynamic imports for performance optimization.

The HistorySidebar.tsx component provides a tabbed interface for managing all auxiliary features:

The project provides launcher scripts that handle everything automatically:

Windows:

start-jarvis.bat

macOS / Linux:

chmod +x start-jarvis.sh
./start-jarvis.sh

The launcher scripts perform these steps:

1. Verify Ollama is running (exits with error if not)
2. Create Python virtual environment if missing
3. Install backend dependencies (pip install -r requirements.txt)
4. Create .env from .env.example if missing
5. Create data/ directory for SQLite database
6. Install frontend dependencies (npm install)
7. Clear .next build cache
8. Kill existing processes on ports 8000 and 3000
9. Start FastAPI backend on port 8000
10. Start Next.js frontend on port 3000
11. Open browser to http://localhost:3000

A docker-compose.yml is provided for containerized deployment with three services:

docker-compose up -d

After containers are up, pull the required AI models:

docker exec -it <ollama-container-name> ollama pull qwen3:8b
docker exec -it <ollama-container-name> ollama pull llama3.2-vision

Data persists via Docker volumes: backend/data for SQLite and ollama_data for model weights.

Download from ollama.ai and pull the required models:

ollama pull qwen3:8b
ollama pull llama3.2-vision

cd backend
python -m venv venv

# Windows
venv\Scripts\activate
# macOS/Linux
source venv/bin/activate

pip install -r requirements.txt
cp .env.example .env    # Edit .env as needed
mkdir data              # SQLite storage directory
python -m app.main

cd frontend
npm install
npm run dev

Open http://localhost:3000 in your browser.

Frontend environment (.env.local):

The routes/ directory manages all external HTTP interfacing across 14 route groups and 55+ endpoints.

J.A.R.V.I.S implements a layered security architecture:

1. Scraper Flagging: Repeated requests via scraper_service.py against Yahoo SERPs in short intervals will temporarily IP-ban your network. Use moderate depth settings (4–6) for repeated searches, or increase delays between batch queries.
2. First-Load VRAM Transfer: Ollama unloads idle models from memory. The first query of any session experiences a lag while model weights transfer from SSD to GPU/CPU memory. Subsequent queries are fast.
3. Vision Model Memory: The llama3.2-vision model requires significant VRAM. Running both text and vision models simultaneously may exceed available GPU memory on systems with less than 8 GB VRAM.
4. Rate Limiting State: Rate limit counters are stored in-memory (dict[str, list[float]]). Server restarts reset all counters. Multi-instance deployments do not share rate limit state.
5. DeepFace First Load: The DeepFace library downloads face detection model files on first use, which may cause a one-time delay during the initial face matching request.

This project is open-source under the MIT License. It was developed strictly for OSINT, portfolio compilation, and development automation.

System operators are fully responsible for ensuring their usage of automated scraping scripts complies with all target platforms' robots.txt specifications and Terms of Service constraints.

Yigit Erdogan - System Architecture, Full-Stack Deployment, Model Tuning.

• LinkedIn: yigit-erdogan-ba7a64294
• Reddit: u/Yigtwx6
• GitHub: Yigtwxx