Skip to content

SERVER.md

Latest commit

 

History

History
519 lines (435 loc) · 13 KB

SERVER.md

File metadata and controls

519 lines (435 loc) · 13 KB

Voiceflow CLI API Server

The Voiceflow CLI now includes an HTTP API server that exposes test execution functionality as REST endpoints with auto-generated OpenAPI/Swagger documentation and WebSocket support for real-time log streaming.

Features

  • HTTP API: Execute test suites via REST endpoints
  • WebSocket API: Execute, cancel, and monitor test suites over a persistent WebSocket connection with real-time log streaming
  • Real-time Logging: Capture and return test execution logs in API responses
  • OpenAPI/Swagger: Auto-generated API documentation at /swagger/index.html
  • Asynchronous Execution: Non-blocking test execution with status tracking
  • CORS Support: Enable cross-origin requests for web frontends
  • Health Checks: Built-in health check endpoints

Starting the Server

Basic Usage

# Start server on default port (8080)
voiceflow server

# Start server on custom port
voiceflow server --port 9090

# Start server with debug mode
voiceflow server --debug

# Start server with custom host
voiceflow server --host 127.0.0.1 --port 8080

Command Line Options

Flag Short Default Description
--port -p 8080 Port to run the server on
--host -H 0.0.0.0 Host to bind the server to
--debug -d false Enable debug mode
--cors true Enable CORS middleware
--swagger true Enable Swagger documentation endpoint

API Endpoints

Health Check

GET /health

Execute Test Suite

POST /api/v1/tests/execute
Content-Type: application/json

{
  "api_key": "your_api_key (optional)",
  "suite": {
    "name": "Example Suite",
    "description": "Suite used as an example",
    "environment_name": "production",
    "tests": [
      {
        "id": "test_1",
        "test": {
          "name": "Example test",
          "description": "These are some tests",
          "interactions": [
            {
              "id": "test_1_1",
              "user": {
                "type": "text",
                "text": "hi"
              },
              "agent": {
                "validate": [
                  {
                    "type": "contains",
                    "value": "hello"
                  }
                ]
              }
            }
          ]
        }
      }
    ]
  }
}

Response:

{
  "id": "123e4567-e89b-12d3-a456-426614174000",
  "status": "running",
  "started_at": "2023-01-01T00:00:00Z",
  "logs": ["Test execution started"]
}

Get Test Status

GET /api/v1/tests/status/{execution_id}

Response:

{
  "id": "123e4567-e89b-12d3-a456-426614174000",
  "status": "completed",
  "started_at": "2023-01-01T00:00:00Z",
  "completed_at": "2023-01-01T00:05:00Z",
  "logs": [
    "Starting test suite execution...",
    "Running Test ID: example_test",
    "Test suite execution completed successfully"
  ]
}

System Information

GET /api/v1/system/info

Response:

{
  "version": "1.0.0",
  "go_version": "go1.20.0",
  "os": "linux",
  "arch": "amd64"
}

OpenAPI/Swagger Documentation

Once the server is running, you can access the interactive API documentation at:

http://localhost:8080/swagger/index.html

WebSocket API

Connect to ws://localhost:8080/ws for a persistent connection with real-time log streaming. The WebSocket endpoint exposes the same functionality as the REST API but pushes log lines as they happen instead of requiring polling.

Protocol

All messages are JSON objects.

Client → Server:

Action Description Fields
execute Start a test suite { "action": "execute", "data": <TestExecutionRequest> }
cancel Cancel a running execution { "action": "cancel", "id": "<execution-id>" }
status Get execution status { "action": "status", "id": "<execution-id>" }

Server → Client:

Type Description
log Real-time log line: { "type": "log", "id": "...", "message": "..." }
status Status update: { "type": "status", "id": "...", "data": <TestStatusResponse> }
result Final result when execution finishes: { "type": "result", "id": "...", "data": <TestStatusResponse> }
error Error message: { "type": "error", "message": "..." }

WebSocket Examples

Using websocat (CLI)

# Install: brew install websocat
websocat ws://localhost:8080/ws

Then paste JSON messages:

{"action":"execute","data":{"suite":{"name":"Example Suite","description":"Test","environment_name":"production","tests":[{"id":"test_1","test":{"name":"Example test","description":"Test","interactions":[{"id":"t1","user":{"type":"text","text":"hi"},"agent":{"validate":[{"type":"contains","value":"hello"}]}}]}}]}}}

Cancel a running execution:

{"action":"cancel","id":"<execution-id-from-status-message>"}

Using JavaScript

const ws = new WebSocket('ws://localhost:8080/ws');

ws.onopen = () => {
  // Execute a test suite
  ws.send(JSON.stringify({
    action: 'execute',
    data: {
      api_key: 'your_api_key',
      suite: {
        name: 'Example Suite',
        description: 'Suite used as an example',
        environment_name: 'production',
        tests: [
          {
            id: 'test_1',
            test: {
              name: 'Example test',
              description: 'These are some tests',
              interactions: [
                {
                  id: 'test_1_1',
                  user: { type: 'text', text: 'hi' },
                  agent: { validate: [{ type: 'contains', value: 'hello' }] }
                }
              ]
            }
          }
        ]
      }
    }
  }));
};

ws.onmessage = (event) => {
  const msg = JSON.parse(event.data);
  switch (msg.type) {
    case 'log':
      console.log(`[LOG] ${msg.message}`);
      break;
    case 'status':
      console.log(`[STATUS] ${msg.id}: ${msg.message || msg.data.status}`);
      // Cancel example: ws.send(JSON.stringify({ action: 'cancel', id: msg.id }));
      break;
    case 'result':
      console.log('[RESULT]', msg.data);
      break;
    case 'error':
      console.error('[ERROR]', msg.message);
      break;
  }
};

Using Python

import asyncio
import json
import websockets

async def run_tests():
    async with websockets.connect('ws://localhost:8080/ws') as ws:
        # Execute a test suite
        await ws.send(json.dumps({
            'action': 'execute',
            'data': {
                'api_key': 'your_api_key',
                'suite': {
                    'name': 'Example Suite',
                    'description': 'Test',
                    'environment_name': 'production',
                    'tests': [{
                        'id': 'test_1',
                        'test': {
                            'name': 'Example test',
                            'description': 'Test',
                            'interactions': [{
                                'id': 't1',
                                'user': {'type': 'text', 'text': 'hi'},
                                'agent': {'validate': [{'type': 'contains', 'value': 'hello'}]}
                            }]
                        }
                    }]
                }
            }
        }))

        # Read messages until execution completes
        async for message in ws:
            msg = json.loads(message)
            if msg['type'] == 'log':
                print(f"[LOG] {msg['message']}")
            elif msg['type'] == 'result':
                print(f"[RESULT] {msg['data']}")
                break
            elif msg['type'] == 'error':
                print(f"[ERROR] {msg['message']}")
                break

asyncio.run(run_tests())

Usage Examples

Using curl

1. Start a test execution

curl -X POST http://localhost:8080/api/v1/tests/execute \
  -H "Content-Type: application/json" \
  -d '{
  "api_key": "your_api_key (optional)",
  "suite": {
    "name": "Example Suite",
    "description": "Suite used as an example",
    "environment_name": "production",
    "tests": [
      {
        "id": "test_1",
        "test": {
          "name": "Example test",
          "description": "These are some tests",
          "interactions": [
            {
              "id": "test_1_1",
              "user": {
                "type": "text",
                "text": "hi"
              },
              "agent": {
                "validate": [
                  {
                    "type": "contains",
                    "value": "hello"
                  }
                ]
              }
            }
          ]
        }
      }
    ]
  }
}'

2. Check test status

curl http://localhost:8080/api/v1/tests/status/YOUR_EXECUTION_ID

3. Health check

curl http://localhost:8080/health

Using JavaScript/fetch

// Execute a test suite
const response = await fetch('http://localhost:8080/api/v1/tests/execute', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    api_key: "your_api_key (optional)",
    suite: {
      name: "Example Suite",
      description: "Suite used as an example",
      environment_name: "production",
      tests: [
        {
          id: "test_1",
          test: {
            name: "Example test",
            description: "These are some tests",
            interactions: [
              {
                id: "test_1_1",
                user: {
                  type: "text",
                  text: "hi"
                },
                agent: {
                  validate: [
                    {
                      type: "contains",
                      value: "hello"
                    }
                  ]
                }
              }
            ]
          }
        }
      ]
    }
  })
});


const execution = await response.json();
console.log('Execution ID:', execution.id);

// Poll for status
const statusResponse = await fetch(`http://localhost:8080/api/v1/tests/status/${execution.id}`);
const status = await statusResponse.json();
console.log('Status:', status.status);
console.log('Logs:', status.logs);

Using Python requests

import requests
import time

# Execute a test suite
response = requests.post('http://localhost:8080/api/v1/tests/execute', json={
    'api_key': 'your_api_key (optional)',
    'suite': {
        'name': 'Example Suite',
        'description': 'Suite used as an example',
        'environment_name': 'production',
        'tests': [
            {
                'id': 'test_1',
                'test': {
                    'name': 'Example test',
                    'description': 'These are some tests',
                    'interactions': [
                        {
                            'id': 'test_1_1',
                            'user': {
                                'type': 'text',
                                'text': 'hi'
                            },
                            'agent': {
                                'validate': [
                                    {
                                        'type': 'contains',
                                        'value': 'hello'
                                    }
                                ]
                            }
                        }
                    ]
                }
            }
        ]
    }
})
execution = response.json()
print(f"Execution ID: {execution['id']}")

# Poll for completion
while True:
    status_response = requests.get(f"http://localhost:8080/api/v1/tests/status/{execution['id']}")
    status = status_response.json()
    
    print(f"Status: {status['status']}")
    
    if status['status'] in ['completed', 'failed']:
        print("Logs:")
        for log in status['logs']:
            print(f"  {log}")
        break
    
    time.sleep(1)

Configuration

Environment Variables

The server respects all existing Voiceflow CLI environment variables:

  • VF_API_KEY: Voiceflow API Key
  • OPENAI_API_KEY: OpenAI API Key (for similarity validations)

CORS Configuration

CORS is enabled by default. To disable CORS:

voiceflow server --cors=false

Debug Mode

Enable debug mode for detailed logging:

voiceflow server --debug

Security Considerations

  • The server runs on all interfaces (0.0.0.0) by default. In production, consider binding to specific interfaces.
  • There is no built-in authentication. Consider adding a reverse proxy with authentication if needed.
  • Test executions are stored in memory. Consider implementing persistent storage for production use.

Troubleshooting

Server Won't Start

  1. Check if the port is already in use:

    lsof -i :8080

  2. Try a different port:

    voiceflow server --port 9090

API Returns 404

Ensure you're using the correct base path /api/v1/ for API endpoints.

Logs Not Appearing

Enable debug mode to see more detailed logging:

voiceflow server --debug