This guide provides detailed information about test coverage tracking, reporting, and best practices in Ampel.

• Overview
• Coverage Tools
• Running Coverage Locally
• CI/CD Integration
• Understanding Coverage Reports
• Improving Coverage
• Troubleshooting

Ampel uses comprehensive test coverage tracking to ensure code quality and catch regressions early. We aim for 80% overall coverage with higher standards for critical business logic.

┌─────────────────────────────────────────┐
│           Pull Request                  │
└─────────────────────────────────────────┘
                  │
                  ▼
┌─────────────────────────────────────────┐
│        GitHub Actions CI                │
│  ┌────────────┐      ┌──────────────┐   │
│  │  Backend   │      │   Frontend   │   │
│  │ (llvm-cov) │      │   (vitest)   │   │
│  └────────────┘      └──────────────┘   │
└─────────────────────────────────────────┘
                  │
                  ▼
┌─────────────────────────────────────────┐
│            Codecov                      │
│   - Aggregate reports                   │
│   - Check thresholds                    │
│   - Generate insights                   │
└─────────────────────────────────────────┘
                  │
                  ▼
┌─────────────────────────────────────────┐
│      PR Comment with Coverage           │
│      🟢 Backend: 82%                    │
│      🟢 Frontend: 78%                   │
│      🟢 Overall: 80%                    │
└─────────────────────────────────────────┘

cargo-llvm-cov is a fast, LLVM-based code coverage tool for Rust. It uses LLVM source-based code coverage, which is 5-10x faster than ptrace-based tools like tarpaulin.

Why cargo-llvm-cov?

• 5-10x faster than tarpaulin (compile-time instrumentation vs runtime ptrace)
• More accurate coverage data using LLVM's instrumentation
• Better integration with CI/CD pipelines
• Lower memory usage during coverage collection

Installation:

# Automatic via Makefile
make test-backend-coverage

# Manual installation
cargo install cargo-llvm-cov --locked
rustup component add llvm-tools-preview

Vitest uses v8 for native JavaScript coverage.

Configuration: See frontend/vitest.config.ts for coverage settings.

# Run all coverage (backend + frontend)
make test-coverage

# Backend only
make test-backend-coverage

# Frontend only
make test-frontend-coverage

# Run with default settings (HTML + LCOV reports)
cargo llvm-cov --all-features --workspace --html --output-dir coverage
cargo llvm-cov --all-features --workspace --lcov --output-path coverage/lcov.info

# Run with specific crate
cargo llvm-cov --package ampel-core --html

# Run with text output (quick summary)
cargo llvm-cov --all-features --workspace

# Generate Codecov-compatible JSON output
cargo llvm-cov --all-features --workspace --codecov --output-path coverage/codecov.json

# View report
open coverage/html/index.html  # macOS
xdg-open coverage/html/index.html  # Linux

cd frontend

# Run with default thresholds
pnpm test -- --run --coverage

# Run with specific file pattern
pnpm test src/components/dashboard -- --run --coverage

# Generate specific report formats
pnpm test -- --run --coverage --coverage.reporter=html,lcov,text

# View report
open coverage/index.html

Coverage is collected in three GitHub Actions jobs:

1. Backend Unit Tests (SQLite) - Fast unit test execution
2. Backend Integration Tests (PostgreSQL) - Full feature testing
3. Backend Coverage - Combines coverage from both test types
4. Frontend Test - Runs Vitest with coverage

backend-coverage:
  name: Backend Test Coverage
  needs: [backend-unit-test, backend-integration-test]
  runs-on: ubuntu-latest
  env:
    RUSTC_WRAPPER: '' # Disable sccache for coverage

  steps:
    - uses: actions/checkout@v6

    - name: Install Rust toolchain
      uses: dtolnay/rust-toolchain@master
      with:
        components: llvm-tools-preview

    - name: Install cargo-llvm-cov
      uses: taiki-e/install-action@cargo-llvm-cov

    - name: Generate coverage
      run: |
        cargo llvm-cov \
          --all-features \
          --workspace \
          --codecov \
          --output-path coverage/codecov.json

    - name: Upload to Codecov
      uses: codecov/codecov-action@v4
      with:
        files: ./coverage/codecov.json
        flags: backend

The coverage-pr-comment.yml workflow automatically posts coverage updates:

## 📊 Coverage Report

| Component   | Coverage   | Status |
| ----------- | ---------- | ------ |
| Backend     | 82.45%     | 🟢     |
| Frontend    | 78.30%     | 🟡     |
| **Overall** | **80.38%** | **🟢** |

### Coverage Thresholds

- 🟢 Green: ≥ 80% (target)
- 🟡 Yellow: 60-79% (acceptable)
- 🔴 Red: < 60% (needs improvement)

Visit codecov.io/gh/pacphi/ampel to see:

• Sunburst Chart: Visual representation of coverage by component
• File Browser: Line-by-line coverage for each file
• Trends: Coverage changes over time
• Flags: Separate backend/frontend tracking

HTML Report Colors:

• 🟢 Green: Line was executed
• 🔴 Red: Line was not executed
• ⚪ White: Not executable (comments, whitespace)

Key Sections:

1. Summary: Overall coverage percentage
2. File List: Coverage by file with percentages
3. Source View: Line-by-line coverage with execution counts

Navigate to frontend/coverage/index.html:

• Summary: Overall metrics (lines, functions, branches, statements)
• File Tree: Navigate by directory
• Source View: See covered (green) and uncovered (red) lines

# Backend: Generate report and view uncovered lines
make test-backend-coverage
open coverage/html/index.html

# Frontend: View coverage in terminal
cd frontend && pnpm test -- --run --coverage

Priority Areas (in order):

1. Authentication & Authorization (ampel-api)
2. Business Logic (ampel-core)
3. Database Operations (ampel-db)
4. Provider Integrations (ampel-providers)
5. Background Jobs (ampel-worker)
6. UI Components (frontend)

Backend Example:

#[cfg(test)]
mod tests {
    use super::*;

    #[tokio::test]
    async fn test_user_creation() {
        // Arrange
        let db = TestDb::new().await;
        let user = NewUser {
            email: "[email protected]".to_string(),
            password: "secure-password".to_string(),
        };

        // Act
        let result = create_user(&db.pool, user).await;

        // Assert
        assert!(result.is_ok());
        let created_user = result.unwrap();
        assert_eq!(created_user.email, "[email protected]");
    }
}

Frontend Example:

import { render, screen, waitFor } from '@testing-library/react';
import userEvent from '@testing-library/user-event';
import { LoginForm } from './LoginForm';

describe('LoginForm', () => {
  it('submits form with valid credentials', async () => {
    const onSubmit = vi.fn();
    render(<LoginForm onSubmit={onSubmit} />);

    await userEvent.type(screen.getByLabelText('Email'), '[email protected]');
    await userEvent.type(screen.getByLabelText('Password'), 'password123');
    await userEvent.click(screen.getByRole('button', { name: 'Login' }));

    await waitFor(() => {
      expect(onSubmit).toHaveBeenCalledWith({
        email: '[email protected]',
        password: 'password123',
      });
    });
  });
});

Don't just test the happy path:

#[tokio::test]
async fn test_user_creation_duplicate_email() {
    let db = TestDb::new().await;

    // Create first user
    let user = NewUser { email: "[email protected]", ... };
    create_user(&db.pool, user.clone()).await.unwrap();

    // Try to create duplicate - should fail
    let result = create_user(&db.pool, user).await;
    assert!(result.is_err());
}

# 1. Run current coverage
make test-backend-coverage

# 2. Identify gaps (red lines in HTML report)
open coverage/html/index.html

# 3. Write tests for uncovered code
vim crates/ampel-core/tests/integration_tests.rs

# 4. Re-run coverage
make test-backend-coverage

# 5. Verify improvement
# (Check coverage percentage increased)

# 6. Commit and push
git add .
git commit -m "test: improve coverage for user module"
git push

# Solution: Install the LLVM tools component
rustup component add llvm-tools-preview

# Solution: Ensure tests are actually running
cargo test --all-features  # Verify tests pass first

# Then run coverage with verbose output
cargo llvm-cov --all-features --workspace

# Solution: Clean build and re-run
cargo llvm-cov clean --workspace
make test-backend-coverage

# Check for global state conflicts (common with metrics/loggers)
# Ensure init functions are idempotent for parallel test execution
cargo test --all-features -- --test-threads=1

// frontend/vitest.config.ts
coverage: {
  lines: 80,
  functions: 75,
  branches: 75,
  statements: 80,
}

Check frontend/vitest.config.ts and ensure your code meets thresholds.

# Ensure coverage provider is installed
cd frontend
pnpm install @vitest/coverage-v8 --save-dev

# Run with coverage explicitly
pnpm test -- --run --coverage

Check .github/workflows/ci.yml:

- name: Upload coverage to Codecov
  uses: codecov/codecov-action@v4
  with:
    files: ./coverage/cobertura.xml
    flags: backend
    fail_ci_if_error: false # Don't fail CI if Codecov is down

1. Check workflow run logs in GitHub Actions
2. Verify GITHUB_TOKEN has pull-requests: write permission
3. Ensure PR is from same repository (not fork)

• Run coverage locally before pushing
• Write tests for new code before implementation (TDD)
• Test error paths not just happy paths
• Focus on business logic for highest coverage
• Use real data in integration tests
• Review coverage reports regularly

• Don't skip tests to "save time"
• Don't mock excessively - test real behavior
• Don't chase 100% coverage - focus on critical paths
• Don't test third-party code (shadcn/ui, etc.)
• Don't ignore failing coverage in CI

• cargo-llvm-cov Documentation
• Vitest Coverage Guide
• Codecov Documentation
• Backend Testing Guide
• Frontend Testing Guide
• CI/CD Guide

Last Updated: 2025-12-24