Skip to content

testing.md

Latest commit

 

History

History
253 lines (196 loc) · 5.41 KB

testing.md

File metadata and controls

253 lines (196 loc) · 5.41 KB

Testing

Testing guide for IntellyWeave CLI.

Test Framework

IntellyWeave CLI uses:

  • AVA: Test runner
  • c8: Coverage reporting
  • TypeScript: Via @ava/typescript with tsx loader

Running Tests

# All tests
npm run test

# Specific test suites
npm run test:unit        # Unit tests
npm run test:commands    # Command tests
npm run test:shell       # Shell tests
npm run test:integration # Integration tests

# Other modes
npm run test:watch       # Watch mode
npm run test:coverage    # With coverage report

Test Organization

tests/
├── unit/              # Utility and helper tests
│   ├── execution.spec.ts
│   ├── config.spec.ts
│   └── weaviate-client.spec.ts
├── commands/          # Command-specific tests
│   ├── config.spec.ts
│   ├── weaviate.spec.ts
│   └── data.spec.ts
├── shell/             # Shell feature tests
│   ├── meta.spec.ts
│   ├── quick-commands.spec.ts
│   └── citations.spec.ts
└── integration/       # Full workflow tests

Writing Tests

Basic Test Pattern

import test from 'ava';
import { functionToTest } from '../src/lib/module.js';

test('function does expected thing', t => {
  const result = functionToTest('input');
  t.is(result, 'expected');
});

test('function handles edge case', t => {
  const result = functionToTest('');
  t.is(result, null);
});

Async Tests

test('async operation completes', async t => {
  const result = await asyncFunction();
  t.true(result.success);
});

Command Tests

import test from 'ava';
import { runCli } from '../helpers/cli-runner.js';

test('config show: displays configuration', async t => {
  const { stdout, exitCode } = await runCli(['config', 'show', '--quiet']);
  t.is(exitCode, 0);
  t.true(stdout.includes('Weaviate'));
});

test('weaviate collections: lists collections', async t => {
  const { stdout, exitCode } = await runCli(['weaviate', 'collections', '--json']);
  t.is(exitCode, 0);
  const data = JSON.parse(stdout);
  t.true(Array.isArray(data.collections));
});

Mutation Safety Tests

test('mutation shows DRY RUN without --live', async t => {
  const { stdout } = await runCli(['data', 'import']);
  t.true(stdout.includes('[DRY RUN]'));
  t.true(stdout.includes('--live flag required'));
});

test('mutation executes with --live flag', async t => {
  const { stdout, exitCode } = await runCli(['--live', 'data', 'import']);
  t.is(exitCode, 0);
  t.false(stdout.includes('[DRY RUN]'));
});

Manual Testing

Since the CLI interacts with Weaviate, manual testing is also required.

Read-Only Commands

npm run intellyweave -- config show
npm run intellyweave -- weaviate status
npm run intellyweave -- weaviate collections
npm run intellyweave -- weaviate stats

Dry-Run Mode

npm run intellyweave -- data import
npm run intellyweave -- weaviate delete TestCollection

Expected output:

[DRY RUN] Would import 150 objects to TestCollection
  Mutation blocked: --live flag required for execution

Live Mode

npm run intellyweave -- --live data import --collections TestCollection
npm run intellyweave -- --live weaviate delete TestCollection

Testing Checklist

Before submitting a PR:

  • Code compiles: npm run build
  • Linter passes: npm run check
  • Unit tests pass: npm run test:unit
  • Command tests pass: npm run test:commands
  • Mutation commands show dry-run without --live
  • Mutation commands execute with --live
  • Read-only commands work without --live
  • --json mode produces valid JSON
  • Error handling works (test invalid inputs)

Testing Philosophy

Fix the Code, Not the Test

When tests fail:

  1. Understand why the test exists
  2. Fix the underlying code issue
  3. Only modify the test if requirements changed

Tests Should Be Meaningful

// GOOD: Tests actual behavior
test('import returns count of imported objects', async t => {
  const result = await importData(testData);
  t.is(result.count, 5);
});

// BAD: Always passes
test('import exists', t => {
  t.true(typeof importData === 'function');
});

Test Edge Cases

test('search handles empty query', async t => {
  await t.throwsAsync(
    () => search(''),
    { message: /query cannot be empty/ }
  );
});

test('export handles empty collection', async t => {
  const result = await exportCollection('EmptyCollection');
  t.is(result.objects.length, 0);
});

Coverage

Generate coverage report:

npm run test:coverage

Output:

  • Console: Text summary
  • File: coverage/lcov.info

Coverage Settings

From package.json:

{
  "exclude": [
    "dist/**",
    "tests/**",
    "**/*.spec.ts",
    "src/banner.ts"
  ],
  "all": true,
  "reporter": ["lcov", "text"]
}

AVA Configuration

From package.json:

{
  "extensions": { "ts": "module" },
  "nodeArguments": ["--import=tsx", "--no-warnings"],
  "workerThreads": false,
  "serial": true,
  "environmentVariables": {
    "NODE_ENV": "test"
  }
}

Key settings:

  • serial: Tests run sequentially (avoid DB conflicts)
  • workerThreads: Disabled for Node.js compatibility
  • tsx: TypeScript execution

See Also