emx-testspec is a test specification engine for CLI E2E testing, inspired by Go's testscript. This document provides context for AI assistants working on this codebase.
-
Engine (
src/engine.rs)- Command registry and execution
- CommandResult abstraction
- Argument expansion utilities
-
Parser (
src/parser.rs)- Line-based script parsing
- Prefix detection (
!,?) - Condition parsing (
[unix],[!windows]) - Argument fragmentation (quotes, variables)
-
Runner (
src/runner.rs)- Test discovery and execution
- Working directory management
- Test result aggregation
- Txtar integration
-
State (
src/state.rs)- Execution context
- Environment variables
- Virtual files (stdout, stderr)
- File system operations
-
Commands (
src/commands.rs)- Built-in command implementations
- Process execution (exec)
- File operations (cmp, grep, cat, cp, mv, rm, mkdir)
- Flow control (cd, echo, env, sleep, stop, skip)
-
Conditions (
src/conditions.rs)- Platform detection (unix, windows, darwin, linux)
- Architecture detection (amd64, arm64)
- Executable detection (
[exec:program])
Scripts are parsed line-by-line:
- Simpler than full AST
- Error messages include line numbers
- Easier to debug test failures
Commands operate in a sandboxed working directory:
- Isolated from actual filesystem
stdoutandstderrcaptured as virtual files- Automatic cleanup (unless
--keepis set)
Pattern matching uses Rust's regex crate:
- Automatic multi-line mode (
(?m)) - DFA-based (1MB limit to prevent ReDoS)
- Escaped in arguments to prevent injection
Three types of expansion:
$VAR/${VAR}- Environment variables${/}- Path separator (OS-dependent)${:}- Path list separator (OS-dependent)
Commands can run in background:
exec server &
exec client
wait # Collect all background processes
Located in src/ modules:
- Parser tests for syntax edge cases
- Command tests for individual operations
- Condition tests for platform detection
Located in tests/integration.rs:
- Discovers and runs
.txtarfiles - Full end-to-end test execution
Run integration tests:
cargo test --test integrationTypical test structure:
# Test description
exec mytool input.txt
stdout 'expected output'
! exec mytool --invalid
stderr 'error message'
-- input.txt --
test data
-- expected.txt --
expected output
- Implement
Cmdtrait incommands.rs - Add to
default_commands()function - Add documentation in
--helpoutput - Add tests for command behavior
Example:
pub struct MyCmd;
impl Cmd for MyCmd {
fn usage(&self) -> (String, String) {
("mycmd".into(), "[args]".into())
}
fn run(&self, args: &[String], state: &mut State) -> Result<CmdResult> {
// Implementation
Ok(CmdResult::Success)
}
}- Implement
Conditiontrait inconditions.rs - Add to
default_conditions()function - Document in help text
When adding new syntax:
- Update
parse_line()inparser.rs - Add tests for new syntax
- Update grammar in README
Uses custom error types in src/error.rs:
ScriptError- Test script errorsErrorKind- Error categorization
Provides:
- Clear error messages
- Line numbers for syntax errors
- Context for command failures
- Tests run sequentially (no parallelism)
- Working directories created fresh for each test
- Regex compilation cached per test
- File operations are synchronous
- Use
anyhow::Resultfor error propagation - Prefer
CmdResult::Successover explicit success returns - Include example usage in doc comments
- Use
#[cfg(test)]for test-only code
When running tests:
cargo testExpected output: 23 tests passing
Integration tests:
TESTSCRIPT_VERBOSE=1 cargo test --test integration- No parallel execution - Tests run sequentially
- Windows path handling - Some edge cases with mixed
/and\ - Symlink support - Limited on Windows
- Resource limits - No CPU/memory limits
Potential improvements:
- Parallel test execution
- Test sharding
- Timeout support
- Retry mechanism
- Test profiling
- JUnit XML output
- Custom command registration via env var
emx-testspec tests/ -vShows:
- Each command being executed
- Exit codes
- Stdout/stderr content
emx-testspec tests/ --keepWorking directories are preserved in temp location for inspection.
TESTSCRIPT_VERBOSE=1 cargo testEnables verbose logging in integration tests.
- Sandboxed execution - All operations in working directory
- Regex limits - DFA size limited to prevent ReDoS
- Path validation - Txtar paths are validated
- No arbitrary code - Only predefined commands
- emx-txtar - Test fixture format
- Go testscript - Original Go version
- Testscript Grammar - Enhanced grammar