Specter is an advanced (legal focused) AI agent orchestration system built on the Elysia framework. It features intelligent routing, custom agent creation, and dynamic document management for legal-domain specific AI assistance.

• Get Started (Python from Github)
• Get started (App)
• Using the System
• Architecture
• Advanced Configuration
• Monitoring & Debugging
• Troubleshooting
• Best Practices

• Python 3.12 or higher
• Weaviate instance (local or cloud)
• OpenAI API key (or compatible LLM provider)

To get the latest development version, you can clone the github repo by running

git clone https://github.com/Siddharth-Khattar/specter-backend.git

move to the working directory via

cd specter-backend

Create a virtual environment with Python (version 3.12+)

python3.12 -m venv .venv
source .venv/bin/activate

and then install "Specter-version" of Elysia via pip

pip install -e .

Done! You can now use the the python package

Create a .env file in the project root:

# Weaviate Configuration
WCD_URL=your_weaviate_cluster_url
WCD_API_KEY=your_weaviate_api_key

# OpenAI Configuration
OPENAI_API_KEY=your_openai_api_key

# Model Configuration
BASE_MODEL=gpt-5o-mini
COMPLEX_MODEL=gpt-5

# Logging
LOGGING_LEVEL=INFO

Run the app via

elysia start

Then navigate to the settings page, add your required API keys, Weaviate cloud cluster details and specify your models.

Start frontend by opening specter-frontend repository and following instructions.

Documents can be uploaded to create agent-specific knowledge bases.

Form Data:

• file: The document file (PDF, TXT, DOCX, etc.)
• user_id: User identifier
• agent_name: (Optional) Associate document with specific agent
• collection_name: (Optional) Custom collection name

1. Documents are automatically chunked for optimal retrieval
2. Text is vectorized using OpenAI embeddings
3. Metadata is preserved (filename, upload date, agent association)
4. Documents are stored in agent-specific Weaviate collections

When you send a query through the chat interface, the system:

1. Analyzes the query to determine domain/intent
2. Routes to appropriate agent using the DomainRouter
3. Executes agent-specific tools with access to relevant knowledge base
4. Returns structured response with source citations

Response Stream:

{
  "type": "tree_update",
  "payload": {
    "from_node": "domain_routing",
    "to_node": "legal_advisor",
    "reasoning": "Query requires legal expertise and document analysis"
  }
}

{
  "type": "text",
  "payload": {
    "text": "Based on the service agreement, the key obligations include..."
  }
}

User Query
    ↓
DomainRouter (analyzes query intent)
    ↓
┌─────────────────────────────────┐
│  Specialized Agent Selection    │
├─────────────────────────────────┤
│ • Civil-law Agent               │
│ • Immigration-law Agent         │
│ • Work-law Agent                │
│ • General Agent (fallback)      │
└─────────────────────────────────┘
    ↓
Agent-Specific Knowledge Base
    ↓
Tool Execution (Query, Aggregate, etc.)
    ↓
Response with Citations

The system uses a branching decision tree that's visible in the UI:

1. Root Node: Domain routing evaluation
2. Branch Nodes: Agent selection and tool choice
3. Leaf Nodes: Tool execution (query, aggregate, text response)
4. Result: Comprehensive response with transparent reasoning

The CustomAgentRegistry class manages all custom agents:

from elysia.tools.domain.custom_agent_registry import CustomAgentRegistry

# Initialize registry
registry = CustomAgentRegistry(logger=logger)

# Load agents for a user
await registry.load_custom_agents(
    user_id="user123",
    client_manager=client_manager
)

# Get available agents
agent_names = registry.get_agent_names()
agent_descriptions = registry.get_agent_descriptions()

Agents can be fine-tuned through several parameters:

tree.add_tool(
    CustomAgentTool,
    branch_id="domain_routing",
    agent_config={
        "temperature": 0.7,
        "max_tokens": 2000,
        "use_reasoning": True
    }
)

Each agent can have multiple associated collections:

# Create collection for agent
await create_agent_collection(
    user_id="user123",
    agent_name="legal_advisor",
    collection_name="contracts_2024"
)

# Associate with agent
await update_agent_collections(
    agent_id="agent_uuid",
    collections=["contracts_2024", "legal_precedents"]
)

The DomainRouter can be configured to adjust sensitivity:

domain_router = DomainRouter(
    confidence_threshold=0.75,  # Minimum confidence for agent selection
    fallback_to_general=True,   # Use general agent if confidence low
    logger=logger
)

View the decision tree structure:

tree.view()  # Prints formatted tree structure

Agent not being selected:

• Check agent description matches query domain
• Verify DomainRouter confidence threshold
• Review agent system prompt for clarity

Documents not being retrieved:

• Confirm document upload completed successfully
• Check collection names match agent configuration
• Verify Weaviate connection and indexing

Routing loops:

• Review hidden_environment state (cleared on soft_reset)
• Check for circular tool dependencies
• Adjust recursion_limit if needed

1. Agent Specialization: Create focused agents for specific domains rather than general-purpose agents
2. System Prompts: Write clear, detailed system prompts that define agent expertise and limitations
3. Document Organization: Group related documents in dedicated collections per agent
4. Regular Updates: Keep agent knowledge bases current by periodically updating documents
5. Monitor Performance: Track agent selection accuracy and adjust confidence thresholds