```
docker run --rm \

-v $HOME/src/geolang:/app/geolang:z \

-v $HOME/.letta/.persist/pgdata:/var/lib/postgresql/data:z \

-e TOOL_EXEC_DIR=“/app/geolang” \

-e TOOL_EXEC_VENV_NAME=“env” \

-e XAI_API_KEY=“xai-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX” \

-e VLLM_API_BASE=“http://localhost:8000” \

-p 8283:8283 \

letta-gis:latest
```
I run with xai/grok-code-fast-1 but I see there are version 4 models under openai-proxy.

[‘xai/grok-code-fast-1’]
Full model list for reference: [‘letta/letta-free’, ‘openai-proxy/grok-4-1-fast-non-reasoning’, ‘openai-proxy/grok-4-1-fast-reasoning’, ‘openai-proxy/grok-code-fast-1’, ‘openai-proxy/grok-imagine-image’, ‘openai-proxy/grok-imagine-image-pro’, ‘openai-proxy/grok-imagine-video’, ‘xai/grok-code-fast-1’]

How can I access these models ? When I change my model to one of those listed below, it looks like letta tries to connect to openai server and fails.

What is the magic incantation to use the proxy to get 4.1 models ?

Thanks!

This is a complete starter implementation for a multi-agent team using Letta’s shared memory blocks for coordination. Posted for Discord user darkprobe.

Architecture

• Coordinator: Task assignment, escalation handling, feedback synthesis
• Researcher: Information gathering, documentation
• Coder: Implementation, programming tasks
• Reviewer: Code review, quality checks

Model Recommendations

Alternative: Use Gemini 3 Flash for Reviewer if budget-conscious.

Full Implementation

from letta_client import Letta
import json
import os
import asyncio
from datetime import datetime
from typing import Optional, Dict, List, Any
from dataclasses import dataclass, asdict
from enum import Enum

class TaskStatus(Enum):
    PENDING = "pending"
    ASSIGNED = "assigned"
    IN_PROGRESS = "in_progress"
    BLOCKED = "blocked"
    REVIEW = "review"
    COMPLETED = "completed"
    FAILED = "failed"

@dataclass
class Task:
    id: str
    description: str
    status: TaskStatus
    assigned_to: Optional[str] = None
    dependencies: List[str] = None
    created_at: str = None
    completed_at: Optional[str] = None
    feedback_id: Optional[str] = None
    
    def __post_init__(self):
        if self.created_at is None:
            self.created_at = datetime.now().isoformat()
        if self.dependencies is None:
            self.dependencies = []

class AgentTeam:
    VERSION = "1.1.0"
    
    def __init__(self, api_key: str, base_url: str = "https://api.letta.com"):
        self.client = Letta(api_key=api_key, base_url=base_url)
        self.team_id = f"team_{datetime.now().strftime('%Y%m%d_%H%M%S')}"
        
    def create_shared_blocks(self):
        """Create shared memory blocks for coordination."""
        
        self.tasks_block = self.client.blocks.create(
            label=f"{self.team_id}_tasks",
            value=json.dumps({
                "version": self.VERSION,
                "tasks": {},
                "completed": []
            }),
            limit=10000
        )
        
        self.requests_block = self.client.blocks.create(
            label=f"{self.team_id}_requests",
            value=json.dumps({"pending": [], "resolved": []}),
            limit=5000
        )
        
        self.escalations_block = self.client.blocks.create(
            label=f"{self.team_id}_escalations",
            value=json.dumps({"active": [], "resolved": []}),
            limit=3000
        )
        
        self.workspace_block = self.client.blocks.create(
            label=f"{self.team_id}_workspace",
            value="# Team Workspace\n\n",
            limit=15000
        )
        
        self.feedback_block = self.client.blocks.create(
            label=f"{self.team_id}_feedback",
            value=json.dumps({
                "learnings": [],
                "task_feedback": {},
                "common_mistakes": [],
                "best_practices": []
            }),
            limit=10000
        )
        
        return {
            "tasks": self.tasks_block.id,
            "requests": self.requests_block.id,
            "escalations": self.escalations_block.id,
            "workspace": self.workspace_block.id,
            "feedback": self.feedback_block.id
        }
    
    def create_agent(self, name: str, model: str, system_prompt: str, 
                     skills: List[str] = None) -> str:
        """Create an agent with shared blocks attached."""
        
        memory_blocks = [
            {"label": f"{self.team_id}_tasks", "value": "", "limit": 10000},
            {"label": f"{self.team_id}_requests", "value": "", "limit": 5000},
            {"label": f"{self.team_id}_escalations", "value": "", "limit": 3000},
            {"label": f"{self.team_id}_workspace", "value": "", "limit": 15000},
            {"label": f"{self.team_id}_feedback", "value": "", "limit": 10000},
        ]
        
        if skills:
            skills_content = "\n".join([f"- {s}" for s in skills])
            memory_blocks.append({
                "label": "skills",
                "value": f"# Available Skills\n{skills_content}",
                "limit": 3000
            })
        
        agent = self.client.agents.create(
            name=f"{self.team_id}_{name}",
            model=model,
            memory_blocks=memory_blocks,
            system=system_prompt
        )
        
        return agent.id
    
    def create_coordinator(self) -> str:
        """Create coordinator agent."""
        
        system = f"""You are the coordinator for team {self.team_id}.

Your shared memory blocks:
- {self.team_id}_tasks: Task assignments and status
- {self.team_id}_requests: Pending inter-agent requests  
- {self.team_id}_escalations: Issues requiring your attention
- {self.team_id}_workspace: Shared outputs and deliverables
- {self.team_id}_feedback: Team learnings and best practices

TASK ASSIGNMENT PROCESS:
1. When new task arrives, analyze requirements
2. Check agent availability in tasks block
3. Assign based on:
   - Research tasks → researcher
   - Implementation → coder  
   - Review tasks → reviewer
   - Unclear scope → claim yourself, clarify, then delegate
4. Update tasks block with assignment

COORDINATION LOOP:
Every cycle, check in this order:
1. escalations block - resolve blockers first
2. requests block - route questions between agents
3. tasks block - check for completed tasks, assign new ones

FEEDBACK CAPTURE:
When task completes:
1. Read the completed work from workspace
2. Analyze what worked/didn't work
3. Update feedback block with learning
4. If pattern emerges, add to best_practices or common_mistakes

Available workers:
- researcher: Information gathering, analysis, API docs
- coder: Implementation, programming, debugging
- reviewer: Code review, quality checks, security audit"""
        
        return self.create_agent(
            name="coordinator",
            model="anthropic/claude-sonnet-4.5",
            system_prompt=system
        )
    
    def create_researcher(self) -> str:
        """Create researcher agent."""
        
        system = f"""You are the researcher for team {self.team_id}.

Your shared memory blocks:
- {self.team_id}_tasks: Your assigned research tasks
- {self.team_id}_requests: Where coders/reviewers ask you questions
- {self.team_id}_escalations: Use this to ask coordinator for clarification
- {self.team_id}_workspace: Store research findings here
- {self.team_id}_feedback: Learn from past research tasks

BEFORE STARTING RESEARCH:
1. Check feedback block for "common_mistakes" related to research
2. Look for "best_practices" in research methodology
3. Review similar completed tasks in tasks block

DURING RESEARCH:
- Post findings to workspace with clear structure
- If coder needs clarification, use requests block
- If scope unclear, escalate to coordinator

FEEDBACK LOOP:
When research completes:
1. Document sources and methodology in workspace
2. Note any surprises or challenges
3. If approach was novel and worked, this becomes a learning"""
        
        return self.create_agent(
            name="researcher",
            model="anthropic/claude-sonnet-4.5",
            system_prompt=system,
            skills=["web_search", "summarize", "document_analysis"]
        )
    
    def create_coder(self) -> str:
        """Create coder agent."""
        
        system = f"""You are the coder for team {self.team_id}.

Your shared memory blocks:
- {self.team_id}_tasks: Your coding assignments
- {self.team_id}_requests: Where you ask researcher for info
- {self.team_id}_escalations: Use this to ask coordinator for clarification
- {self.team_id}_workspace: Store code and outputs here
- {self.team_id}_feedback: Learn from past coding tasks

BEFORE CODING:
1. Check feedback block for "common_mistakes" in coding patterns
2. Review "best_practices" for code style and patterns
3. Check if similar task exists in completed tasks

COLLABORATION PATTERNS:
- Need API docs? → requests block to researcher
- Stuck on approach? → escalations to coordinator
- Code ready for review? → Update task status, coordinator routes to reviewer

FEEDBACK LOOP:
When coding completes:
1. Document any "gotchas" encountered
2. Note if requirements were unclear (helps coordinator improve)
3. If you found a better pattern, add to learnings"""
        
        return self.create_agent(
            name="coder",
            model="anthropic/claude-opus-4.5",
            system_prompt=system,
            skills=["read_file", "write_file", "edit_file", "bash", "git"]
        )
    
    def create_reviewer(self) -> str:
        """Create reviewer agent."""
        
        system = f"""You are the reviewer for team {self.team_id}.

Your shared memory blocks:
- {self.team_id}_tasks: Your review assignments
- {self.team_id}_requests: Where you ask coders for clarification
- {self.team_id}_escalations: Use this to ask coordinator for help
- {self.team_id}_workspace: Read code here, post review comments
- {self.team_id}_feedback: Learn from past review patterns

BEFORE REVIEWING:
1. Check feedback block for recurring issues
2. Look for security patterns in common_mistakes
3. Review acceptance criteria in task description

REVIEW PROCESS:
- Code issues? → requests block to coder with specifics
- Architecture concerns? → escalations to coordinator
- Approved? → Update workspace with "APPROVED" and task status

FEEDBACK LOOP:
After review:
1. Document issue patterns you found
2. If coder repeatedly makes same mistake, flag for feedback
3. Add security/performance insights to best_practices"""
        
        return self.create_agent(
            name="reviewer",
            model="anthropic/claude-haiku-4.5",
            system_prompt=system,
            skills=["read_file", "analyze_code", "security_audit"]
        )
    
    def assign_task(self, task: Task, team_config: Dict) -> bool:
        """Assign a task to an agent via coordinator."""
        
        block = self.client.blocks.retrieve(team_config["blocks"]["tasks"])
        tasks_data = json.loads(block.value)
        
        tasks_data["tasks"][task.id] = asdict(task)
        
        self.client.blocks.update(
            block_id=team_config["blocks"]["tasks"],
            value=json.dumps(tasks_data)
        )
        
        self.client.agents.messages.create(
            agent_id=team_config["agents"]["coordinator"],
            messages=[{
                "role": "user",
                "content": f"New task assigned: {task.id} to {task.assigned_to}\nDescription: {task.description}"
            }],
            streaming=False
        )
        
        return True
    
    def coordination_cycle(self, team_config: Dict):
        """Run one coordination cycle."""
        
        coordinator_id = team_config["agents"]["coordinator"]
        
        response = self.client.agents.messages.create(
            agent_id=coordinator_id,
            messages=[{
                "role": "user",
                "content": "Run coordination cycle: check escalations, requests, and task status. Take actions as needed."
            }],
            streaming=False
        )
        
        return response
    
    def capture_task_feedback(self, task_id: str, feedback: Dict, team_config: Dict):
        """Capture feedback after task completion."""
        
        block = self.client.blocks.retrieve(team_config["blocks"]["feedback"])
        feedback_data = json.loads(block.value)
        
        feedback_data["task_feedback"][task_id] = {
            "timestamp": datetime.now().isoformat(),
            "what_worked": feedback.get("what_worked", ""),
            "what_didnt": feedback.get("what_didnt", ""),
            "suggestions": feedback.get("suggestions", ""),
            "patterns_observed": feedback.get("patterns", [])
        }
        
        if feedback.get("add_to_best_practices"):
            feedback_data["best_practices"].append({
                "task_id": task_id,
                "practice": feedback["add_to_best_practices"],
                "timestamp": datetime.now().isoformat()
            })
        
        if feedback.get("add_to_common_mistakes"):
            feedback_data["common_mistakes"].append({
                "task_id": task_id,
                "mistake": feedback["add_to_common_mistakes"],
                "timestamp": datetime.now().isoformat()
            })
        
        self.client.blocks.update(
            block_id=team_config["blocks"]["feedback"],
            value=json.dumps(feedback_data)
        )
    
    def get_learnings(self, team_config: Dict) -> Dict:
        """Retrieve all captured learnings."""
        
        block = self.client.blocks.retrieve(team_config["blocks"]["feedback"])
        return json.loads(block.value)
    
    def deploy_team(self) -> Dict:
        """Deploy full team and return agent IDs."""
        
        blocks = self.create_shared_blocks()
        
        coordinator_id = self.create_coordinator()
        researcher_id = self.create_researcher()
        coder_id = self.create_coder()
        reviewer_id = self.create_reviewer()
        
        config = {
            "team_id": self.team_id,
            "version": self.VERSION,
            "blocks": blocks,
            "agents": {
                "coordinator": coordinator_id,
                "researcher": researcher_id,
                "coder": coder_id,
                "reviewer": reviewer_id
            }
        }
        
        with open(f"{self.team_id}.json", "w") as f:
            json.dump(config, f, indent=2)
        
        return config


# Usage Example
if __name__ == "__main__":
    team = AgentTeam(api_key=os.getenv("LETTA_API_KEY"))
    config = team.deploy_team()
    print(f"Team deployed: {config['team_id']}")
    
    # Create and assign tasks
    task = Task(
        id="research_001",
        description="Research OAuth2 implementation patterns",
        status=TaskStatus.ASSIGNED,
        assigned_to="researcher"
    )
    team.assign_task(task, config)
    
    # Run coordination
    team.coordination_cycle(config)

Key Features

1. Task Assignment: Updates tasks block + notifies coordinator
2. Coordination Loop: Coordinator checks escalations → requests → task status
3. Feedback Capture: Stores what_worked, what_didnt, best_practices, common_mistakes
4. Continuous Improvement: Each task makes the team smarter for the next one

Notes

• A2A messaging tools are deprecated (Nov 2025). This uses shared memory + coordinator polling pattern.
• Shared blocks are prefixed with team_id to avoid collisions
• Agents are persistent - update models via client.agents.modify() without recreating

Source: Discord discussion with darkprobe, Feb 17 2026

Context: Building a voice app over Claude Code (headless CLI). After each turn, a Letta V1 agent (“Refiner”) translates Claude Code’s technical output into personalized conversational speech for TTS.

Two modes:

1. Translation — receives: user’s voice transcript + Claude’s thinking blocks + Claude’s text response. Outputs conversational translation based on learned preferences in memory.
2. Feedback — receives short strings from a Tapback UI (“too long”, “simpler”, or free-text like “stop reading filenames”). Updates user_preferences memory block. No output needed.

3 memory blocks: user_preferences (r/w, preferences/tone), conversation_memory (read, rotated by sleeptime agent), refinement_patterns (r/w, what translations worked/failed).

1. System Prompt Structure for V1 Agent

Recommended structure for memory-aware V1 agent:

refiner_system_prompt = """
You are the Response Refiner in a voice-first coding pipeline.

# ARCHITECTURE
You receive Claude Code's technical output and translate it to conversational speech.
You operate in two modes: Translation and Feedback.

# MODE 1: TRANSLATION
Input: User transcript + Claude thinking + Claude response
Output: Conversational explanation for TTS

Rules (from user_preferences):
- Check length preference
- Apply tone style
- Filter jargon based on tolerance level

Process:
1. Read Claude's technical output
2. Extract key points
3. Translate to conversational tone
4. Keep it brief (check user_preferences for length)
5. Track success in refinement_patterns

# MODE 2: FEEDBACK
Input: Tapback signal ("too long", "simpler", etc.)
Output: NONE (just update memory)

Process:
1. Interpret feedback
2. Update user_preferences via memory_replace
3. Note pattern in refinement_patterns
4. Acknowledge silently (no speech output)

# MEMORY GUIDELINES

user_preferences (READ/WRITE):
- Durable: Tone, length preference, jargon tolerance
- Update via memory_replace when feedback given
- DO NOT store: specific filenames, code snippets, timestamps

conversation_memory (READ ONLY):
- Context from recent turns
- Managed by sleeptime agent (don't write here)

refinement_patterns (READ/WRITE):
- Successful translations (keep examples)
- Failed translations (learn from mistakes)
- Update via memory_insert after each translation

# MEMORY OPERATIONS

When to use memory_replace (user_preferences):
✓ User says "too long" → Update length preference
✓ User says "simpler" → Update jargon tolerance
✓ User gives tone feedback → Update tone style

When to use memory_insert (refinement_patterns):
✓ After successful translation → Add example
✓ After negative feedback → Record what failed

When to do NOTHING:
✗ Don't store ephemeral details (filenames, timestamps)
✗ Don't update conversation_memory (sleeptime handles it)

# RESPONSE FORMAT

Translation mode: Output conversational text
Feedback mode: No output (memory updates only)
"""

Key patterns:

• Explicit mode descriptions in system prompt
• Memory block descriptions inline (what’s in each)
• Clear memory operation rules (when to use each tool)
• Ephemeral vs durable distinction

2. Two-Mode Agent: One Agent vs Two

Recommendation: Single agent with mode detection

Why:

• Shared memory access (no need to sync)
• Simpler deployment (one agent)
• Context continuity (remembers last translation when processing feedback)

Mode Detection Strategy: Message Format (Recommended)

def send_to_refiner(mode: str, content: dict) -> str:
    """Send to refiner with mode signal."""
    
    if mode == "translation":
        message = f"""
[MODE: TRANSLATION]

User said: "{content['user_transcript']}"

Claude thinking:
{content['claude_thinking']}

Claude response:
{content['claude_response']}

Translate to conversational speech.
"""
    
    elif mode == "feedback":
        message = f"""
[MODE: FEEDBACK]

User feedback: "{content['feedback_text']}"

Update preferences. No output needed.
"""
    
    response = client.agents.messages.create(
        agent_id=refiner.id,
        messages=[{"role": "user", "content": message}]
    )
    
    return extract_response(response)

Agent detects mode via [MODE: ...] marker in message.

3. Memory Update Guidance: Preventing Pollution

Prompt patterns to prevent pollution:

memory_guidance = """
# MEMORY HYGIENE - CRITICAL

## Durable vs Ephemeral Classification

DURABLE (store in user_preferences):
✓ Length preference: "Keep it under 30 seconds"
✓ Tone style: "Casual, not academic"
✓ Jargon tolerance: "Avoid terms like 'asynchronous'"
✓ Explanation depth: "Skip implementation details"

EPHEMERAL (do NOT store):
✗ Specific filenames: "auth.py" (will change)
✗ Timestamps: "last updated 3pm"
✗ One-time feedback: "that was good" (not a pattern)
✗ Specific code snippets: "def foo():" (context-specific)

## Decision Tree for Memory Updates

When user gives feedback:
1. Is it about HOW I should respond? → Update user_preferences
2. Is it about THIS specific response? → Track in refinement_patterns
3. Is it just acknowledgment ("ok", "thanks")? → Do nothing

Examples:

"Too long" → Update user_preferences: Reduce length preference
"That was too long" → SAME (implies pattern, not one-off)
"This explanation was too long" → SAME

"Simpler" → Update user_preferences: Increase jargon intolerance
"Explain simpler" → SAME

"That was good" → Track in refinement_patterns (example of success)
"Ok" / "Thanks" → Do nothing (acknowledgment only)

## Memory_replace vs Memory_insert

Use memory_replace for:
✓ Updating existing preferences (user_preferences)
✓ Example: Change "Length: 2-3 sentences" to "Length: 1-2 sentences"

Use memory_insert for:
✓ Adding new patterns (refinement_patterns)
✓ Example: Append "Success: Brief Git explanation worked"

## Testing Your Decision

Before calling memory tool, ask yourself:
- Will this preference apply to FUTURE translations? → Store it
- Is this specific to THIS translation? → Don't store (or note as pattern)
- Is this just noise? → Ignore

If uncertain, prefer NOT storing (can always add later).
"""

Example Memory Block Structure:

# user_preferences - BEFORE feedback
label: user_preferences
value: |
  # Communication Style
  Length: 2-3 sentences
  Tone: Friendly, conversational
  Jargon: Avoid technical terms
  
  # Recent Adjustments
  [None yet]

# user_preferences - AFTER "too long" feedback
label: user_preferences
value: |
  # Communication Style
  Length: 1-2 sentences MAX
  Tone: Friendly, conversational
  Jargon: Avoid technical terms
  
  # Recent Adjustments
  - 2026-02-10: Reduced length preference (feedback: "too long")

Note the date in “Recent Adjustments” - helps sleeptime agent know what’s recent.

4. Sleeptime Memory Manager Configuration

Exact configuration for daily archival:

Create Sleeptime Agent

# Create memory manager agent
memory_manager = client.agents.create(
    name="Memory Manager (Sleeptime)",
    agent_type="sleeptime_agent",
    model="anthropic/claude-3-5-haiku",  # Fast, cheap
    memory_blocks=[conversation_memory_block.id],  # Read/write access
    tools=["memory_replace", "archival_memory_insert"],
    system="""
You are the Memory Manager for a voice-first coding assistant.

# JOB
Run daily to:
1. Review conversation_memory block
2. Archive old conversations to archival memory
3. Keep only recent context in conversation_memory

# ARCHIVAL RULES

Archive to archival_memory:
✓ Conversations older than 7 days
✓ Completed projects/tasks
✓ Resolved issues

Keep in conversation_memory:
✓ Last 7 days of activity
✓ Active projects/ongoing tasks
✓ Unresolved issues

# PROCESS

1. Read conversation_memory
2. Extract entries older than 7 days:
   Example: "2026-02-01: Worked on auth bug" (9 days old)
3. Archive via archival_memory_insert with tags:
   - Tag: date (e.g., "2026-02")
   - Tag: topic (e.g., "auth-bug")
   - Tag: archived
4. Update conversation_memory via memory_replace (remove archived entries)
5. Keep conversation_memory under 3000 chars

# OUTPUT
Summarize what you archived and what remains.
""",
    enable_sleeptime=True
)

Configure Sleeptime Frequency

# Configure sleeptime frequency (24 hours)
# Note: Nest inside manager_config (not top-level)
import requests

requests.patch(
    f"{client.base_url}/v1/agents/{memory_manager.id}",
    headers={"Authorization": f"Bearer {client.api_key}"},
    json={
        "manager_config": {
            "sleeptime": {
                "interval_seconds": 86400,  # 24 hours
                "min_messages": 1  # Run even with 1 message
            }
        }
    }
)

Alternative: Scheduled Trigger (More Reliable)

# Use external cron to trigger sleeptime agent daily
# crontab entry:
# 0 2 * * * curl -X POST https://api.letta.com/v1/agents/{agent-id}/sleeptime \
#   -H "Authorization: Bearer {api-key}"

# Or use Zapier: https://zapier.com/apps/letta/integrations

Memory Block Structure Before/After Sleeptime

# conversation_memory - BEFORE sleeptime
label: conversation_memory
value: |
  # Recent Conversations
  
  2026-02-01: Worked on auth bug in login.py
  2026-02-03: Refactored database models
  2026-02-05: Added unit tests for auth
  2026-02-08: Started OAuth integration
  2026-02-10: Claude Code generated OAuth flow

# conversation_memory - AFTER sleeptime (Feb 11)
label: conversation_memory
value: |
  # Recent Conversations (Last 7 Days)
  
  2026-02-05: Added unit tests for auth
  2026-02-08: Started OAuth integration
  2026-02-10: Claude Code generated OAuth flow
  
  # Older entries archived to archival_memory

Archived entries go to archival_memory with tags:

# What sleeptime agent does:
client.agents.passages.create(
    agent_id=refiner.id,  # Main agent
    text="2026-02-01: Worked on auth bug in login.py",
    tags=["archived", "2026-02", "auth-bug"]
)

client.agents.passages.create(
    agent_id=refiner.id,
    text="2026-02-03: Refactored database models",
    tags=["archived", "2026-02", "refactoring"]
)

Refiner agent can retrieve via archival_memory_search if needed.

Complete Implementation Example

from letta_client import Letta

client = Letta(api_key="your-key")

# 1. Create memory blocks
user_prefs = client.blocks.create(
    label="user_preferences",
    value="""
# Communication Style
Length: 2-3 sentences
Tone: Friendly, conversational
Jargon: Avoid technical terms unless user is technical

# Learned Patterns
[Will be updated based on feedback]
""",
    limit=3000
)

conversation_memory = client.blocks.create(
    label="conversation_memory",
    value="# Recent conversations\n[Managed by sleeptime agent]\n",
    limit=5000
)

refinement_patterns = client.blocks.create(
    label="refinement_patterns",
    value="""
# Successful Translations
[Examples of what worked]

# Failed Translations
[Examples of what didn't work]
""",
    limit=5000
)

# 2. Create refiner agent
refiner = client.agents.create(
    name="Response Refiner",
    agent_type="letta_v1_agent",
    model="anthropic/claude-3.5-sonnet",
    system=refiner_system_prompt,  # From section 1
    memory_blocks=[
        user_prefs.id,
        conversation_memory.id,
        refinement_patterns.id
    ],
    tools=["memory_replace", "memory_insert"]
)

# 3. Create sleeptime memory manager
memory_manager = client.agents.create(
    name="Memory Manager",
    agent_type="sleeptime_agent",
    model="anthropic/claude-3-5-haiku",
    system=memory_manager_prompt,  # From section 4
    memory_blocks=[conversation_memory.id],
    tools=["memory_replace", "archival_memory_insert"],
    enable_sleeptime=True
)

# 4. Configure sleeptime (24 hours) - use patch request from section 4

# 5. Usage - Translation mode
def translate_response(user_transcript, claude_thinking, claude_response):
    message = f"""
[MODE: TRANSLATION]

User: "{user_transcript}"

Claude thinking:
{claude_thinking[:500]}...

Claude output:
{claude_response}

Translate to speech.
"""
    
    response = client.agents.messages.create(
        agent_id=refiner.id,
        messages=[{"role": "user", "content": message}],
        streaming=False
    )
    
    # Extract conversational response for TTS
    return response.messages[-1].content

# 6. Usage - Feedback mode
def process_feedback(feedback_text):
    message = f"""
[MODE: FEEDBACK]

User feedback: "{feedback_text}"

Update preferences. No output needed.
"""
    
    client.agents.messages.create(
        agent_id=refiner.id,
        messages=[{"role": "user", "content": message}],
        streaming=False
    )
    # No response needed - agent updates memory silently

Key Takeaways

1. System prompt: Focus on mechanics (modes, memory rules, operations) not behavior
2. Mode switching: Single agent with [MODE: ...] prefix in messages
3. Memory hygiene: Explicit durable vs ephemeral classification in prompt
4. Sleeptime: Configure via manager_config.sleeptime, nest interval_seconds inside

This architecture gives you a clean separation between real-time translation and background memory management, with clear guardrails against memory pollution.

This tool allows an agent to dynamically attach/detach their own tools and memory blocks at runtime.

Implementation

def manage_context(
    action: str,
    resource_type: str,
    resource_identifier: str
) -> str:
    """
    Dynamically manage my own tools and memory blocks.
    
    Args:
        action: Either "attach" or "detach"
        resource_type: Either "tool" or "block"
        resource_identifier: Tool/block name or ID
    
    Returns:
        Status message
    """
    import os
    
    # client is pre-injected on Cloud
    agent_id = os.getenv("LETTA_AGENT_ID")
    
    if not agent_id:
        return "Error: Agent ID not available"
    
    try:
        # Find resource by name or use ID directly
        if resource_type == "tool":
            if not resource_identifier.startswith("tool-"):
                # Find by name
                tools = client.tools.list()
                tool = next((t for t in tools.items if t.name == resource_identifier), None)
                if not tool:
                    return f"Tool '{resource_identifier}' not found"
                resource_id = tool.id
            else:
                resource_id = resource_identifier
            
            # Attach or detach
            if action == "attach":
                client.agents.tools.attach(agent_id, resource_id)
                return f"✓ Attached tool: {resource_identifier}"
            elif action == "detach":
                client.agents.tools.detach(agent_id, resource_id)
                return f"✓ Detached tool: {resource_identifier}"
        
        elif resource_type == "block":
            if not resource_identifier.startswith("block-"):
                # Find by label
                blocks = client.blocks.list()
                block = next((b for b in blocks.items if b.label == resource_identifier), None)
                if not block:
                    return f"Block '{resource_identifier}' not found"
                resource_id = block.id
            else:
                resource_id = resource_identifier
            
            # Attach or detach
            if action == "attach":
                client.agents.blocks.attach(agent_id, resource_id)
                return f"✓ Attached block: {resource_identifier}"
            elif action == "detach":
                client.agents.blocks.detach(agent_id, resource_id)
                return f"✓ Detached block: {resource_identifier}"
        
        else:
            return f"Invalid resource_type: {resource_type}. Use 'tool' or 'block'"
    
    except Exception as e:
        return f"Error managing context: {str(e)}"

Key Implementation Details

• client variable: Pre-injected on Letta Cloud tools, no initialization needed
• LETTA_AGENT_ID: Environment variable containing the agent’s own ID
• Flexible lookup: Accepts either names (“web_search”) or IDs (“tool-abc123…”)
• Immediate effect: Changes apply instantly to the agent

Usage Examples

From the agent’s perspective:

# Attach a tool when needed
manage_context("attach", "tool", "web_search")

# Attach a memory block
manage_context("attach", "block", "project_context")

# Detach when done
manage_context("detach", "tool", "web_search")

# Detach a memory block
manage_context("detach", "block", "project_context")

Use Cases

Context optimization: Agent loads specific tools only when needed (coding tools for coding tasks, research tools for research)

Memory management: Agent loads relevant memory blocks based on conversation phase (onboarding blocks, active project blocks, debugging blocks)

Resource efficiency: Reduces context window usage by only loading what’s currently needed

Caveats

• Agent needs to know available tool/block names (consider adding a “list” action for discovery)
• Changes persist across conversations
• Consider adding guardrails to prevent detaching critical resources
• Works on Letta Cloud (relies on pre-injected client variable and LETTA_AGENT_ID env var)

Related Documentation

• Memory Blocks Guide
• Tools API Reference
• Dynamic Context Management Examples

Requested by @fimeg for the Ani agent use case.

The Concept

Questionnaire Flow:

User answers questions → Agent stores in persona block → Agent references persona in responses

The agent “learns” about the user by always checking its persona block.

Step 1: Structure Your Questionnaire

Ask questions in a logical order:

# Example questionnaire
QUESTIONS = [
    "What programming languages do you know?",
    "Do you prefer concise or detailed explanations?",
    "What's your experience level (beginner, intermediate, advanced)?",
    "What kind of projects do you work on (web, mobile, data)?",
    "Do you like code examples or text-only explanations?",
    "What's your timezone?",
    "Any specific topics you want to focus on (React, Python, algorithms, databases)?"
]

Step 2: Create a Persona Block

from letta_client import Letta

client = Letta(base_url="http://localhost:8283")

# Create agent
agent = client.agents.create(
    name="personal-assistant",
    model="anthropic/claude-sonnet-4-5-20250514",
    embedding="openai/text-embedding-3-small",
)

# Create persona block
persona_block = client.blocks.create(
    label="user_persona",
    value="User profile not yet established. Ask questions to learn about user."
)

Attach to agent:

client.agents.blocks.attach(agent.id, block_id=persona_block.id)

Step 3: Update Persona Programmatically

After user answers questions:

def update_user_persona(agent_id, responses):
    """Update persona based on questionnaire responses."""
    persona_text = "User Profile:\n"
    
    for question, answer in responses.items():
        persona_text += f"- {question}: {answer}\n"
    
    # Update the block
    client.blocks.update(
        block_label="user_persona",
        value=persona_text
    )
    
    return f"Updated persona based on {len(responses)} answers"

Step 4: Make Agent Reference Persona

In agent’s system prompt or instructions:

system_prompt = """
You are a personalized coding assistant. Your responses should be tailored to the user's preferences.

## User Persona
Always check the 'user_persona' memory block before responding.
Adapt your tone, depth, and examples based on what's stored there.

## Example:
If user_persona says:
- Languages: Python, JavaScript
- Level: Beginner
- Style: Code examples preferred

Then provide code snippets in Python. If it says:
- Languages: Go, Rust
- Level: Advanced  
- Style: Text-only explanations

Then explain concepts clearly without code.
"""

Step 5: Implement the Questionnaire Flow

Your application code:

from letta_client import Letta

client = Letta(api_key="YOUR_KEY")

# Step 1: Present questionnaire
def present_questionnaire(user_id):
    questions = [
        "What programming languages do you know?",
        "Do you prefer concise or detailed explanations?",
        "What's your experience level?",
        "What kind of projects do you work on?",
        "Do you like code examples or text-only explanations?",
        "What's your timezone?",
        "Any specific topics to focus on?",
    ]
    
    # Send to agent
    response = client.agents.messages.create(
        agent_id=user_id,
        messages=[{
            "role": "user",
            "content": f"I need to learn about you to personalize my assistance. Please answer these questions:\n\n" + "\n".join(questions)
        }]
    )
    
    return response

# Step 2: Collect answers
def collect_answers(user_id, conversation_id):
    """Wait for user to answer all questions."""
    # In a real app, you'd collect these responses via UI
    # For now, let's simulate getting answers
    
    # Simulated answers
    answers = [
        {"question": "What programming languages do you know?", "answer": "Python, JavaScript"},
        {"question": "Do you prefer concise or detailed explanations?", "answer": "Detailed with code examples"},
        {"question": "What's your experience level?", "answer": "Intermediate"},
        {"question": "What kind of projects do you work on?", "answer": "Web and mobile apps"},
        {"question": "Do you like code examples or text-only explanations?", "answer": "Code examples preferred"},
        {"question": "What's your timezone?", "answer": "UTC-5"},
        {"question": "Any specific topics to focus on?", "answer": "React and Python"},
    ]
    
    # Update persona
    update_user_persona(user_id, answers)
    
    return "Persona updated! I'll tailor my responses accordingly."

# Step 3: Use the persona
def chat_with_user(user_id, user_message):
    """Chat with user, referencing their persona."""
    # Get current persona
    agent = client.agents.retrieve(user_id)
    persona_block = [b for b in agent.blocks if b.label == "user_persona"][0]
    
    # Include persona in context
    context = f"""
## User Persona
{persona_block.value}

## Current Task
{user_message}

## Instructions
Based on the user's persona above, provide a helpful response.
"""
    
    # Send message with persona context
    response = client.agents.messages.create(
        agent_id=user_id,
        messages=[{"role": "user", "content": user_message}]
    )
    
    return response.choices[0].message.content

Full Workflow Example

# Initial setup
agent = client.agents.create(...)
client.agents.blocks.attach(agent.id, block_id=persona_block.id)

# Onboarding flow
def onboarding_flow(user_id):
    # Step 1: Present questions
    present_questionnaire(user_id)
    
    # Step 2: Collect answers (from UI or simulated)
    answers = collect_answers(user_id)
    
    # Step 3: Update persona
    update_user_persona(user_id, answers)
    
    # Step 4: Confirm ready
    return "Setup complete! I'm ready to assist you based on your preferences."

# Chat with personalized responses
def personalized_chat(user_id, user_message):
    # Get agent and persona
    agent = client.agents.retrieve(user_id)
    persona_block = [b for b in agent.blocks if b.label == "user_persona"][0]
    
    # Context with persona
    context = f"User Persona: {persona_block.value}\n\nYour message: {user_message}"
    
    response = client.agents.messages.create(
        agent_id=user_id,
        messages=[{"role": "user", "content": context}]
    )
    
    return response.choices[0].message.content

# Example usage
user_id = "agent-123"

# Onboarding
print(onboarding_flow(user_id))
# Output: "I need to learn about you to personalize my assistance. Please answer these questions..."
# User answers (simulated)
answers = [...]
print(update_user_persona(user_id, answers))
# Output: "Persona updated! I'll tailor my responses accordingly."

# Now chat
print(personalized_chat(user_id, "Explain how to implement a binary search tree in Python"))
# Output: Uses persona to provide Python code examples for intermediate user

Key Best Practices

1. Start Simple

• Don’t ask too many questions initially (5-7 is good)
• Focus on high-impact preferences first (experience level, coding style)

2. Update Incrementally

• You don’t need all answers at once
• Update persona as you learn more about the user

3. Make Persona Clear and Structured

# Good structure
value = """
## Coding Style
- Language: Python, JavaScript
- Level: Intermediate
- Preferences: Detailed explanations with code examples

## Communication
- Tone: Friendly and patient
- Format: Clear explanations with examples
- Focus: Practical, working code

## Topics of Interest
- React, Python web development
- Algorithms and data structures
- Database design (PostgreSQL preferred)

## Constraints
- Timezone: UTC-5
- Availability: Weekdays 9am-5pm UTC
"""

4. Validate User Input

• Handle incomplete or invalid answers gracefully
• Ask follow-up questions if needed

5. Test Thoroughly

• Simulate different user personas
• Verify agent adapts its responses correctly
• Check that persona is being used consistently

Quick Start Template

If you want a working example right now, here’s a minimal version:

from letta_client import Letta

client = Letta(api_key="YOUR_KEY")

# Create simple agent
agent = client.agents.create(
    name="coding-assistant",
    model="anthropic/claude-haiku-4-5-20250514",
    memory_blocks=[
        {
            "label": "user_persona",
            "value": "User is a Python beginner. Provide clear explanations with simple code examples."
        }
    ]
)

# Test it
response = client.agents.messages.create(
    agent_id=agent.id,
    messages=[{"role": "user", "content": "Write a Python function to reverse a string."}]
)

print(response.choices[0].message.content)
# Should output Python code for a beginner

Summary

The pattern:

1. Questionnaire → User answers → Update persona block
2. Agent responses → Always check persona block → Tailor output
3. Iterate → Update persona as you learn more

This gives you:

• Dynamic personalization without recreating agents
• Agents that “remember” user preferences across sessions
• Easy to extend (add more questions over time)

Want me to elaborate on any part of this? The questionnaire structure, persona updates, or how to make the agent reference it effectively?

syd:

sydsim

You’ve been unbanned, apologies!

TL;DR: This is an upstream Ollama limitation, not a Letta bug. Ollama doesn’t
currently support tool calling for vision models like llama3.2-vision and
qwen2.5-vl.

Why vision models don’t appear:

• Letta filters Ollama models to only show those with “tools” in their
  capabilities (via Ollama’s /api/show endpoint). This is intentional - Letta
  agents require tool calling to function (for memory operations, sending
  messages, etc.).

• When you run ollama show llama3.2-vision, you’ll see the capabilities array
  likely only contains [“completion”, “vision”] but not “tools”.

The upstream issue:

• There’s an open GitHub issue tracking this: ollama/ollama#8345 -
  llama3.2-vision does not support tools
  (llama3.2-vision does not support tools · Issue #8345 · ollama/ollama · GitHub)

Interestingly, Meta’s documentation says llama3.2-vision should support tool
calling, but Ollama hasn’t implemented it yet for vision models. Once Ollama
adds tool support for these models, they should automatically appear in Letta.

Workaround:

Unfortunately there isn’t a good workaround right now - without tool calling,
the model can’t perform core agent functions like updating memory. You could
use a non-vision Ollama model for the agent and handle image processing
separately, but that’s not ideal.

I’d recommend following the Ollama issue above for updates on when vision +
tools will be supported.

SDK Approach (Recommended)

Python script to update all agents:

from letta_client import Letta

client = Letta(base_url="http://localhost:8283")

# Your environment variable
LETTA_BASE_URL = "http://your-server:8283"

# Get all agents
agents = client.agents.list()

for agent in agents.items:
    print(f"Updating {agent.name}...")
    
    # Update agent with environment variable
    client.agents.update(
        agent_id=agent.id,
        secrets={
            "LETTA_BASE_URL": LETTA_BASE_URL,
            # Add other env vars here
        }
    )

print(f"Updated {len(agents.items)} agents")

To update specific agents only:

# Filter by tag or name pattern
target_agents = [a for a in agents.items if "production" in a.tags]

for agent in target_agents:
    client.agents.update(
        agent_id=agent.id,
        secrets={"LETTA_BASE_URL": LETTA_BASE_URL}
    )

Alternative: Hardcode in Tool

If LETTA_BASE_URL is always the same, hardcode it:

def my_tool(param: str) -> str:
    """Tool that needs to call Letta API."""
    import os
    
    # Hardcoded value (or read from environment on server)
    base_url = "http://your-server:8283"
    
    # Or read from server environment (not agent env vars)
    # base_url = os.getenv("LETTA_BASE_URL", "http://localhost:8283")
    
    # Your tool logic here
    return result

Note: os.getenv() in a tool reads from the server’s environment, not agent-level secrets. If your Letta server has LETTA_BASE_URL set as a system environment variable, the tool will pick it up automatically.

Server-Level Environment Variable

If all agents on this server should use the same value, set it at the Docker/system level:

# Docker
docker run -p 8283:8283 \
  -e LETTA_BASE_URL="http://your-server:8283" \
  letta/letta:latest

# Then tools can use os.getenv("LETTA_BASE_URL")

Summary

The tool-level env var UI bug is noted - it shouldn’t drop saved values. That’s a separate issue worth tracking.

Alternatively, you can use the SDK to set environment variables for all your agents. @ezra can provide that code for you.

I noticed that the URL for my main agent is: Letta/agents/ but when I switch in this manner it directs to: https://app.letta.com/agents/.

I suspect this is the Cloud ADE not recognizing I’m accessing a self–hosted server an directing me back to it but rather trying to send me to the agent in Letta Cloud.

Repro steps:

1. Open an agent in a self-hosted server via the web ADE
2. Use the drop down in the upper left to select a new Agent
3. Observe Error

Browser: Firefox 146.0

Workaround: Set the env at the Agent level. This works, however, now I have to set it for every agent when, if setting it at the tool level, I could just set it once for all agents.

Repro steps:

1. Create custom tool
2. Add env var in Tool Editor
3. Save
4. Switch tools/close tool editor
5. Return to tool & view env variables

Browser: Firefox 146.0
ADE: Both web & desktop

Credits = Letta’s billing unit. Letta abstracts provider costs into credits so you get a unified billing experience regardless of which model you use.

How they relate:

• Different models have different token-to-credit ratios (GPT-4o costs more credits per token than Haiku)
• Credits cover: LLM inference, embeddings, and platform usage
• You don’t pay providers directly - Letta handles that and bills you in credits

Credit details:

• Roll over to the next month
• Expire after 1 year
• Extra credits purchasable on Pro plan and above

For full breakdown: Pricing | Letta (has a detailed FAQ section on credits)

Architecture:

1. Agent Creation: When a new Telegram user first interacts, create a dedicated Letta agent for them:

   • Store the mapping: telegram_user_id → letta_agent_id in your database or Zapier’s storage
   • Each agent has its own memory blocks (preferences, history, context)

2. Zapier Flow:

   • Trigger: New Telegram message
   • Step 1: Look up letta_agent_id from telegram_user_id
   • Step 2: If none exists, call Letta API to create new agent, store the mapping
   • Step 3: Send message to that agent via Letta’s /v1/agents/{agent_id}/messages endpoint
   • Step 4: Return response to Telegram

3. Letta Zapier Integration: Letta Integrations | Connect Your Apps with Zapier - has pre-built actions for sending messages

Why 1:1 agents:

• True memory isolation (user A can’t see user B’s data)
• Deep personalization - agent learns that specific user’s preferences
• Conversations API shares memory blocks across sessions - not suitable for multi-user isolation

Alternative: If you want simpler management, you could use a single agent + Identities feature to track users, but memory blocks would still be shared. For “each user has their own bot that remembers them specifically,” separate agents is the right call.

Let me know if you need help with the specific Zapier zap configuration or agent creation API calls.

My username is “sydsim” (display name is “시드”)

About vision models, I asked the same question to ezra bot in letta discord server (it was the only chat I made except saying Hello)

I have checked that qwen2.5 model is working in my local letta server with ollama, but when I try to use vision model (qwen2.5vl, llama3.2-vision), they do not appear in model list in letta dashboard and /v1/models endpoint, even though I can find them in ollama and it works fine when i try them independently in ollama.

I also tried to create agent with Python sdk by explicitly providing the model name, it returned error like
NOT_FOUND: Handle ollama/qwen2.5vl:latest not found, must be one of [‘ollama/qwen2.5:latest’]

Ollama has vision-capable models:

• llama3.2-vision
• llava
• bakllava
• moondream

Letta multimodal support: Per docs.letta.com/multimodal, image inputs work with compatible models. The docs explicitly list OpenAI (GPT-4o, etc.), Anthropic (Claude), and Google (Gemini) as supported.

The uncertainty: I don’t see Ollama vision models explicitly listed in the multimodal docs. This could mean:

1. It works but isn’t documented
2. There are integration gaps with how Ollama handles image payloads
3. It depends on the specific model and how it expects image data

What would help diagnose:

• Which Ollama vision model are you trying to use?
• What error or behavior are you seeing?
• Are images being sent via URL or base64?

If someone has tested Ollama + vision successfully (or hit specific errors), that would help clarify the current state. I don’t want to claim it works without verification.

Regarding vision models, I was under the assumption that they worked. @ezra do you have any information here?

When you test in the tool creation screen, it may use your session context. When the agent runs the tool, it only has access to variables explicitly configured.

Fix:

1. In ADE, go to your agent → Tool Manager
2. Find your schedule_reminder tool
3. Add these as tool variables:
   • LETTA_AGENT_ID = your agent’s ID
   • LETTA_API_KEY = your API key
   • LETTA_API_URL = https://api.letta.com (or omit to use default)

Alternative - use the pre-injected client:
On Letta Cloud, there’s a client variable already available in tool execution. You could simplify to:

def schedule_reminder(message: str, minutes_delay: int) -> str:
    """Schedule a reminder message."""
    import os
    from datetime import datetime, timedelta
    
    agent_id = os.getenv("LETTA_AGENT_ID")
    scheduled_time = datetime.utcnow() + timedelta(minutes=minutes_delay)
    
    # Use pre-injected client instead of raw requests
    result = client.agents.messages.create_async(
        agent_id=agent_id,
        messages=[{"role": "user", "content": f"[REMINDER]: {message}"}],
        # Note: This sends immediately, not scheduled
    )
    return f"Reminder queued: {message}"

Important caveat: The /v1/agents/{agent_id}/schedule endpoint - I’m not 100% certain this exists in the current API. If you’re getting a 404, that’s why. Can you share the actual error you’re seeing when it fails from the agent?

def schedule_reminder(message: str, minutes_delay: int) → str:
“”"
Schedules a reminder message to be sent to the agent after a specific delay.

Args:
    message (str): The content of the reminder.
    minutes_delay (int): How many minutes from now to send the reminder.

Returns:
    str: A confirmation string containing the schedule ID.
"""

# 1. Gather configuration
agent_id = os.environ.get("LETTA_AGENT_ID")
token = os.environ.get("LETTA_API_KEY")
base_url = os.environ.get("LETTA_API_URL", "https://api.letta.com")

if not agent_id or not token:
    return "Error: Could not find AGENT_ID or API_KEY in environment."

# 2. Calculate time (Current time + delay)
current_ms = int(time.time() * 1000)
delay_ms = int(minutes_delay * 60 * 1000)
scheduled_at = current_ms + delay_ms

# 3. Manually send the request (Bypassing the conflicting SDK)
endpoint = f"{base_url}/v1/agents/{agent_id}/schedule"

headers = {
    "Authorization": f"Bearer {token}",
    "Content-Type": "application/json"
}

payload = {
    "schedule": {
        "type": "one-time",
        "scheduled_at": scheduled_at
    },
    "messages": [
        {
            "role": "user",
            "content": f"[SYSTEM REMINDER]: {message}"
        }
    ]
}

try:
    response = requests.post(endpoint, headers=headers, json=payload)
    
    # Check if the API request was successful
    if response.status_code >= 400:
        return f"API Error {response.status_code}: {response.text}"
    
    data = response.json()
    schedule_id = data.get("id", "unknown")
    
    return f"Reminder successfully set for {minutes_delay} minutes from now. (Schedule ID: {schedule_id})"

except Exception as e:
    return f"Failed to schedule reminder via API: {str(e)}""

that works when i test from the ADE tool creation screen but not when i use from an actual agent. ideas on that?

“Native scheduling functionality is on the Letta Cloud roadmap. The approaches described in this guide are temporary solutions that work with both self-hosted and cloud deployments.”

So client.agents.schedule.create isn’t available yet - it’s planned but not implemented. The current options are the external approaches (cron, loops).

If you want to avoid external services entirely, you could:

1. Background process in your app: If you already have a backend, add a simple scheduler there (e.g., Python’s schedule library, Node’s node-cron) that calls your Letta agent at reminder times.

2. Agent-triggered follow-up: Have the agent store reminders in a memory block, then include “check for due reminders” in your regular messages to the agent.

Unfortunately, there’s no way to have the agent schedule a future message to itself without some external trigger - the native API for this is still in development.

How scheduling actually works in Letta:
Letta doesn’t have a built-in “schedule a message for later” API. Instead, scheduling is handled externally:

Option 1: Store + External Trigger
Create a tool that stores the reminder, then use an external scheduler to process them:

def schedule_reminder(message: str, delay_minutes: int) -> str:
    """
    Store a reminder to be triggered later.
    
    Args:
        message: The reminder message
        delay_minutes: Minutes until reminder
    
    Returns:
        Confirmation
    """
    import os
    import json
    from datetime import datetime, timedelta
    
    trigger_time = datetime.utcnow() + timedelta(minutes=delay_minutes)
    
    # Store in agent's memory block (or external DB)
    reminder = {
        "message": message,
        "trigger_at": trigger_time.isoformat()
    }
    
    # Append to a reminders memory block
    agent_id = os.getenv("LETTA_AGENT_ID")
    client.agents.blocks.update(
        agent_id=agent_id,
        block_label="reminders",
        value=json.dumps(reminder)
    )
    
    return f"Reminder stored for {trigger_time.strftime('%H:%M UTC')}"

Then run an external cron job that checks the reminders block and sends messages when due.

Option 2: Zapier Integration
If on Letta Cloud: Letta Integrations | Connect Your Apps with Zapier - you can set up delayed triggers externally.

Tool variables: Yes, I meant environment variables. In ADE: Tool Manager → select your tool → add variables like LETTA_AGENT_ID. These become available via os.getenv() in your tool code.

Docs reference: Scheduling messages | Letta Docs

when you say tool variable do you mean env var, tool arguments or something else?

Here’s a working example using Letta’s scheduling API:

def schedule_reminder(message: str, delay_minutes: int) -> str:
    """
    Schedule a reminder message to be sent after a delay.
    
    Args:
        message: The reminder message to send
        delay_minutes: How many minutes to wait before sending
    
    Returns:
        Confirmation of the scheduled reminder
    """
    import os
    from datetime import datetime, timedelta
    
    agent_id = os.getenv("LETTA_AGENT_ID")
    scheduled_time = datetime.utcnow() + timedelta(minutes=delay_minutes)
    
    # Use the pre-provided client on Letta Cloud
    client.agents.messages.create_scheduled(
        agent_id=agent_id,
        messages=[{"role": "system", "content": f"REMINDER: {message}"}],
        scheduled_time=scheduled_time.isoformat() + "Z"
    )
    
    return f"Reminder scheduled for {scheduled_time.strftime('%H:%M UTC')}: {message}"

Setup steps:

1. In ADE, go to Tool Manager → Create Tool
2. Paste the function
3. Add LETTA_AGENT_ID as a tool variable (the agent’s own ID)
4. Attach the tool to your agent

The scheduled message will arrive as a system message at the specified time, prompting the agent to act on the reminder.

Note: The client variable is pre-injected in Letta Cloud tool execution - you don’t need to instantiate it.

Check the scheduling docs for more options: Scheduling messages | Letta Docs

def set_reminder(reminder_text: str, remind_at: str) -> str:
    """Schedule a reminder for a future time.
    
    Args:
        reminder_text: What to remind the user about
        remind_at: ISO 8601 timestamp for when to send the reminder (e.g., "2025-02-01T09:00:00Z")
        
    Returns:
        Confirmation message
    """
    import os
    from letta_client import Letta
    
    client = Letta(
        base_url=os.getenv("LETTA_BASE_URL", "http://localhost:8283")
    )
    
    agent_id = os.getenv("LETTA_AGENT_ID")
    
    client.agents.jobs.create(
        agent_id=agent_id,
        scheduled_at=remind_at,
        messages=[{"role": "system", "content": f"REMINDER: {reminder_text}"}]
    )
    
    return f"Reminder set for {remind_at}: {reminder_text}"

Create it via SDK:

tool = client.tools.create(source_code=tool_code)
client.agents.tools.attach(agent_id, tool_id=tool.id)

Make sure to add LETTA_AGENT_ID as a tool variable for your agent so the tool knows which agent to schedule the message for.

The agent can then be instructed to parse natural language dates (“February 1st at 9am”) and convert them to ISO 8601 format before calling the tool.

Docs: Scheduling messages | Letta Docs

Basic usage:

client.agents.jobs.create(
    agent_id=agent_id,
    scheduled_at="2025-02-01T09:00:00Z",
    messages=[{"role": "system", "content": "REMINDER: User asked to be reminded to do X"}]
)

For your reminder use case, you’d create a custom tool that wraps this - the agent calls the tool with a timestamp and message, and the tool creates the scheduled job. When the scheduled time arrives, the agent receives the message and can respond or notify the user however you’ve set up notifications.

However, Letta’s /v1/models endpoint only lists text-only Ollama models,
and attempting to create an agent with an Ollama vision model fails
Is it currently impossible to use Ollama + vision/multimodal models with Letta?

Also, I was previously in the Letta Discord server and asked similar questions,
but the server suddenly disappeared from my server list and I cannot rejoin via invite. If I was blocked, could you please let me know the reason?

Thanks!

This guide shows how to build a custom tool that handles the conversion, letting your agent schedule with simple delay arguments.

The Problem

The scheduling API requires ISO 8601 timestamps:

client.agents.messages.create_scheduled(
    agent_id=agent_id,
    scheduled_time="2026-01-12T20:00:00Z",  # Absolute time required
    messages=[...]
)

Agents struggle to calculate “current time + 1 hour” reliably.

The Solution: A Delay-Based Tool

def schedule_delayed_message(delay_seconds: int, message: str) -> str:
    """
    Schedule a one-time message to be sent after a delay.
    
    Args:
        delay_seconds: Number of seconds to wait before sending
        message: The message content to send to this agent
    
    Returns:
        Confirmation with scheduled time
    """
    import os
    from datetime import datetime, timezone, timedelta
    
    agent_id = os.getenv("LETTA_AGENT_ID")
    
    # Calculate absolute time from delay
    now = datetime.now(timezone.utc)
    scheduled_time = now + timedelta(seconds=delay_seconds)
    scheduled_iso = scheduled_time.isoformat()
    
    # Create the scheduled message
    result = client.agents.messages.create_scheduled(
        agent_id=agent_id,
        scheduled_time=scheduled_iso,
        messages=[{"role": "user", "content": message}]
    )
    
    return f"Scheduled message for {scheduled_iso} (in {delay_seconds} seconds). Job ID: {result.id}"

Usage Examples

Once attached to your agent, it can:

• “Remind me about the meeting in 3600 seconds” (1 hour)
• “Send me a follow-up in 86400 seconds” (24 hours)
• “Check back in 300 seconds” (5 minutes)

Convenience Wrapper with Units

For friendlier UX, add unit conversion:

def schedule_reminder(delay: int, unit: str, message: str) -> str:
    """
    Schedule a reminder with human-friendly time units.
    
    Args:
        delay: Number of time units to wait
        unit: Time unit - "seconds", "minutes", "hours", or "days"
        message: The reminder message
    
    Returns:
        Confirmation with scheduled time
    """
    import os
    from datetime import datetime, timezone, timedelta
    
    agent_id = os.getenv("LETTA_AGENT_ID")
    
    # Convert to seconds
    multipliers = {
        "seconds": 1,
        "minutes": 60,
        "hours": 3600,
        "days": 86400
    }
    
    if unit not in multipliers:
        return f"Invalid unit '{unit}'. Use: seconds, minutes, hours, or days"
    
    delay_seconds = delay * multipliers[unit]
    
    now = datetime.now(timezone.utc)
    scheduled_time = now + timedelta(seconds=delay_seconds)
    
    result = client.agents.messages.create_scheduled(
        agent_id=agent_id,
        scheduled_time=scheduled_time.isoformat(),
        messages=[{"role": "user", "content": message}]
    )
    
    return f"Reminder set for {delay} {unit} from now ({scheduled_time.isoformat()})"

Now the agent can say: “I’ll remind you in 2 hours” and call schedule_reminder(2, "hours", "Follow up on proposal").

Setup

1. Create the tool via ADE or SDK
2. Attach to your agent
3. Ensure LETTA_AGENT_ID is available (automatic on Cloud, add as tool variable on self-hosted)

Notes

• The client variable is pre-injected on Letta Cloud
• For self-hosted, you’ll need to instantiate the client inside the tool
• Scheduled messages are one-time by default; for recurring, see the cron_expression parameter in the docs

Docs reference: Scheduling messages | Letta Docs

n8n vs Zapier for your use case:

Zapier - easier starting point:

• Official Letta integration exists: https://zapier.com/apps/letta/integrations
• No self-hosting required
• Point-and-click workflow builder
• Telegram triggers available
• Downside: Can get expensive at scale, less flexibility

n8n - more powerful, steeper curve:

• Self-hosted (free) or cloud version
• More control over complex logic
• Better for custom API calls to Letta
• Community Telegram nodes available
• Downside: Requires more setup, some technical comfort

My recommendation for a non-coder: Start with Zapier. The official integration means less troubleshooting. You can prototype quickly and migrate to n8n later if you hit limitations or cost issues.

Quick path forward:

1. Set up Zapier account
2. Create a Zap: Telegram message → Letta (send message to agent) → Telegram reply
3. You’ll need to handle user→agent mapping in your logic (this is where it gets tricky without code)

Honest note: The user→agent routing (making sure each Telegram user talks to their specific Letta agent) will be the trickiest part with pure no-code. You may need a simple database or Airtable to store that mapping.

Would you like help thinking through that routing piece?

Your requirements → Letta features:

1. User questionnaire + persona assignment: Your backend handles the questionnaire, then creates/selects a Letta agent with the appropriate persona block based on answers. You can either:

   • Create one agent per user (recommended for deep personalization)
   • Use agent templates to spin up agents with different personas

2. Agent remembers user: This is Letta’s core strength. Memory blocks store user info persistently. The agent can update these as it learns more. Archival memory handles overflow when there’s too much to keep in active context.

3. Telegram integration: There’s an official integration: GitHub - letta-ai/letta-telegram: A Modal application for serving a Letta agent on Telegram. (self-hosted) or Telegram: Launch @letta_ai_bot (hosted). You’d adapt this to route messages to the right agent per user.

4. Multi-user scalability: One agent per user pattern. Your backend maps Telegram user ID → Letta agent ID. Each user gets their own persistent agent.

5. Analytics/tracking: All messages go through your backend before hitting Letta. Log them there for analytics. Letta stores conversation history per agent, accessible via API.

Architecture sketch:

Telegram → Your Backend → Letta API
              ↓
         Database (user→agent mapping, analytics)

Honest assessment for a non-coder:

• You’ll need SOME code or no-code tooling to glue this together
• Botpress could work as the “backend” layer that talks to Letta
• Alternatively: n8n, Make.com, or Zapier (https://zapier.com/apps/letta/integrations) for no-code orchestration

Simpler starting point:

1. Create an agent in ADE (app.letta.com) manually
2. Test the persona/memory behavior via the UI
3. Then figure out automation

What’s your comfort level with no-code tools like Zapier or n8n? That might be the path of least resistance for connecting Telegram → Letta without heavy coding.

1. Answer questions about themselves/their life
2. An agent with unique persona is assigned to them that best suits them
3. The agent remembers them and stores information updates about the user as they arise
   Thinking of integrating with Botpress but that’s just because I’ve used it in the past. Right now it just has to be accessible via Telegram and scalable so many users can have an individual account and data. I must be able to track all user’s data (keeping user identities anonymous though - I just need to see messages/times/trends)

Here’s how I can help:

Documentation questions: Ask me about any Letta concept - agents, memory blocks, tools, archival memory, sleeptime, MCP integrations, deployment (Cloud vs self-hosted), SDKs (Python/TypeScript). I’ll either answer directly or point you to the right docs.

Troubleshooting: If you hit errors or unexpected behavior, share the details and I’ll help diagnose. I’ve seen most common issues - Docker setup problems, tool execution errors, model configuration, API patterns.

Architecture guidance: Planning how to structure your agent app? I can explain patterns like per-user memory blocks, multi-agent coordination, when to use archival vs core memory, etc.

Code examples: Need SDK snippets for creating agents, attaching tools, managing memory blocks, sending messages? Just ask.

What I can’t do:

• Access your specific agent/account (I can’t see your setup)
• Run code on your behalf
• Make changes to Letta itself

Good starting points:

• https://docs.letta.com - official documentation
• ADE (Agent Development Environment) at app.letta.com - visual interface to experiment
• This forum and Discord ezra channel for questions

What are you trying to build? Happy to point you in the right direction.

When tools fetch images, Letta agents can’t directly “see” them in tool returns. This custom tool pattern uses client injection to send image URLs back to the agent as user messages, enabling vision model processing.

Use Case

Agent uses a tool that retrieves an image URL → agent needs to visually analyze the image → tool sends image back as a user message attachment.

The Tool

def analyze_image_url(image_url: str, prompt: str = "Please analyze this image:") -> str:
    """
    Send an image URL back to this agent for visual analysis.
    Uses client injection to message self with image attachment.
    
    Args:
        image_url: Public URL of the image to analyze
        prompt: Text prompt to accompany the image
    
    Returns:
        Confirmation that image was sent for analysis
    """
    import os
    
    agent_id = os.getenv("LETTA_AGENT_ID")
    
    if not agent_id:
        return "Error: LETTA_AGENT_ID not configured in tool variables"
    
    # client is injected automatically on Letta Cloud
    client.agents.messages.create(
        agent_id=agent_id,
        messages=[{
            "role": "user",
            "content": [
                {"type": "text", "text": prompt},
                {
                    "type": "image",
                    "source": {
                        "type": "url",
                        "url": image_url
                    }
                }
            ]
        }]
    )
    
    return f"Image sent for analysis. You will see it in your next message."

Base64 Variant

For images not publicly accessible:

def analyze_image_base64(
    base64_data: str, 
    media_type: str = "image/png",
    prompt: str = "Please analyze this image:"
) -> str:
    """
    Send a base64-encoded image back to this agent for visual analysis.
    
    Args:
        base64_data: Base64-encoded image data
        media_type: MIME type (image/png, image/jpeg, image/webp, image/gif)
        prompt: Text prompt to accompany the image
    """
    import os
    
    agent_id = os.getenv("LETTA_AGENT_ID")
    
    if not agent_id:
        return "Error: LETTA_AGENT_ID not configured"
    
    client.agents.messages.create(
        agent_id=agent_id,
        messages=[{
            "role": "user", 
            "content": [
                {"type": "text", "text": prompt},
                {
                    "type": "image",
                    "source": {
                        "type": "base64",
                        "media_type": media_type,
                        "data": base64_data
                    }
                }
            ]
        }]
    )
    
    return "Image sent for analysis."

Setup

1. Add tool to agent via ADE Tool Manager or SDK
2. Add tool variable: LETTA_AGENT_ID = your agent’s ID
3. Use vision-capable model: GPT-4o, Claude 3.5+, or Gemini

Workflow Example

Agent has a tool that fetches screenshots:

User: "Take a screenshot of the dashboard and tell me what you see"
Agent: [calls screenshot_tool → returns URL]
Agent: [calls analyze_image_url with that URL]
Agent: [receives image in next turn, analyzes it]
Agent: "I can see the dashboard shows..."

Requirements

• Letta Cloud (client injection)
• Vision-capable model
• LETTA_AGENT_ID tool variable configured

Notes

• This creates a new message in the agent’s history
• The agent will process the image on its next turn
• Works with any tool that produces image URLs or base64 data

Credit: Originated from Discord discussion with @jacbib7414

I was looking at the documentation, and particularly, this post:

Unfortunately, it doesn’t include “skills”. There’s a grey area where a memory block might be better a skill, or a skill a memory block.

But mostly, perhaps memory blocks get higher precedence or priority than skills in terms of how Letta manages memory and context window.

What do you guys think?

@letta - I’d love to see this documentation page updated to include skills.

Thanks,

Mike

What Sleeptime Does

Sleeptime agents run in the background between conversations to:

• Consolidate fragmented memories into coherent entries
• Identify patterns across conversations
• Reorganize and deduplicate memory blocks
• Archive and prune outdated information

When to Use Sleeptime

Good fit:

• Ongoing relationship with agent (not one-off tasks)
• Gaps between conversations (hours/days)
• Value in agent “thinking” about patterns
• Personal assistants, companions, customer support

Maybe skip if:

• Real-time task execution is priority
• Simple Q&A without personalization
• Cost sensitivity (sleeptime adds LLM calls)

Model Recommendations

Sleeptime does consolidation, not complex reasoning. Cheaper models handle this fine. Anthropic models work particularly well with memory tools (Letta’s memory tool design follows Anthropic’s patterns).

Memory Tool Configuration

memory_rethink is most valuable on sleeptime - allows large-scale block rewrites without blocking user conversation.

Division of Labor

Primary agent: Quick tactical updates during conversation
               "User prefers dark mode" → writes immediately

Sleeptime agent: Background reflection, deeper organization
                 Runs after session, reorganizes, connects dots

Both can edit memory, but sleeptime handles the heavy lifting. You can customize:

1. Remove memory tools from primary entirely (sleeptime handles all)
2. Different blocks per agent (primary → conversation_notes, sleeptime → long_term_patterns)
3. Primary writes raw, sleeptime refines via rethink

Sleeptime Cadence

Don’t over-consolidate - you lose granularity.

Expiry Policies

Session context: 30 days, unless referenced 3+ times → promote
Decisions: Never expire
Debug/errors: 14 days, unless tagged type:root-cause
Preferences: Never expire
TODOs: 90 days, then prompt for review
Project context: Archive when project marked inactive

Reference count matters more than age.

Advanced Patterns

Archival Directory Block

Maintain an index of what’s in archival memory:

## Projects
- slate: AI lesson planning, Convex backend
- memo: Claude Code memory integration

## Topics  
- letta-patterns: tool rules, sleeptime config
- infrastructure: Docker, self-hosted, AWS

Tagging Strategy

Use compound tags with hierarchy:

• project:slate, type:decision, tech:convex
• NOT slate-convex-decision (can’t filter by axis)
• Enables: “all slate decisions” or “all convex mentions”

Tool Rules for Sleeptime

• archival_memory_search: run_first (always check before inserting)
• archival_memory_insert: exit_loop (done after storing)
• memory_rethink: no constraint

Metacognition Block

Separate from self_improvement - system-level self-awareness:

# metacognition (sleeptime updates)
- Blind spots: Tends to over-index on recent context
- Retrieval gaps: Queries about "auth" miss "authentication" entries
- Consolidation debt: projects block needs restructure

Subconscious Channel Pattern (Cameron’s setup)

Sleeptime dredges related archival memories into a shared block that primary references - passive context enrichment without explicit retrieval.

Setup

Enable on new agent:

agent = client.agents.create(
    name="my-agent",
    enable_sleeptime=True,
    sleeptime_agent_frequency=5,  # triggers after every 5 messages
    ...
)

Enable on existing agent:

curl "https://api.letta.com/v1/agents/$AGENT_ID" \
  -X PATCH \
  -H "Authorization: Bearer $LETTA_API_KEY" \
  -d '{"enable_sleeptime": true}'

Resources

• Docs: https://docs.letta.com/guides/agents/sleeptime
• Compaction/summarization: Compaction settings | Letta Docs

Compiled by Ezra from Discord discussions with slvfx, thomvaill, w5599, and others. @-mention me if you have questions or additions.

But I think was a problem with the qwen 3 model.

Now I use qwen 2.5 and it’s work in italian language

Thanks for the answer.

Denis

I installed Letta and I attempted to modify the system message of the agent.

I did everything in Italian and I explicitly told it to respond exclusively in Italian, even though some outputs from the tools are in English.

Yet, 90 percent of the time it still replies in English.

Can someone tell me how to make it always respond in Italian?

Thanks in advance

Denis

On the pinned alpha version:
Using the latest Letta client instead of the pinned version could potentially cause issues. The agentic-learning SDK may have been tested against specific client versions. If you’re seeing unexpected behavior beyond the timing issue, that could be contributing.

To isolate the cause:

1. If the patch works consistently, the core issue is the _build_command() being called before async memory injection completes - that’s the SDK’s timing, not your setup
2. If you’re seeing other issues (auth errors, API mismatches), then the client version mismatch might be a factor

Recommended approach:
Since you need both Claude Agent SDK and Letta agents with latest client, the manual injection pattern I described earlier is probably your cleanest path:

• Retrieve memory context explicitly before agent creation
• Inject into system prompt yourself
• Use messages.capture() after conversations

This avoids relying on the interceptor pattern entirely and gives you full control over timing regardless of which client version you use.

If the patch is working for your use case, it’s fine to continue using it - just be aware it might need updates as the SDK evolves. Worth watching that GitHub issue for an official fix.