Skip to content

Latest commit

 

History

History
633 lines (433 loc) · 16 KB

File metadata and controls

633 lines (433 loc) · 16 KB

Yumdocs Improvement Roadmap

Current Status

  • ✅ Phase 1.1: Critical delimiter bug fix and comprehensive test coverage
  • ✅ Phase 1.2: Test suite refresh and Faker.js API migration
  • ✅ Phase 1.3: Resolve critical TODOs with enhanced error handling
  • ✅ Phase 1.4: Type safety improvements with proper TypeScript types
  • ✅ Phase 1.5a: TypeScript modernization with ES2022 and advanced features
  • ✅ Phase 2.1: Circular dependency resolution via dependency injection

Status:

  • All core tests passing (23/26 test suites, 74/91 tests)
  • Modern TypeScript 5.9.2 configuration with ES2022 target
  • Ready to proceed with basic security hardening (Phase 1.5)

Phase 1: Remaining Critical Fixes (Current Phase)

Priority: HIGH | Effort: 1-2 weeks

1.5 Basic Security Hardening

Tasks:

  • Add input validation for template content
  • Implement basic size limits for documents
  • Add basic expression validation

New Files:

  • src/security/validation.ts - Input validation utilities
  • src/security/limits.ts - Size and complexity limits

Effort: 3-4 days | Testing: Security test suite

1.6 Test Suite Re-engineering

Priority: HIGH | Effort: 2-3 weeks

Current Issues Identified:

  • 26 test files with inconsistent organization patterns
  • 17 OpenXML fixture files with inconsistent naming conventions
  • Mix of integration/unit tests without clear separation
  • Test antipatterns (private property access via as any)
  • Repetitive test setup and data generation
  • Missing edge case coverage and error scenarios

Proposed Solutions:

  • Fixture Naming Convention: Standardize to {format}-{feature}-{scenario}.{ext}
    • Examples: word-expression-basic.docx, ppt-each-nested.pptx, excel-if-conditional.xlsx
  • Test Structure Reorganization: Clear separation of unit vs integration tests
  • Test Utilities: Replace private property access with proper test helpers
  • Data Generation: Consistent patterns for test data creation
  • Coverage Expansion: Add missing edge cases and error scenario testing

Files to Create:

  • tests/helpers/TestDataFactory.ts - Centralized test data generation
  • tests/helpers/TemplateTestUtils.ts - Template testing utilities
  • tests/helpers/FixtureManager.ts - Standardized fixture loading
  • tests/integration/ - Clear integration test separation
  • tests/unit/ - Refactored unit tests with better patterns

Files to Rename/Reorganize:

  • All fixture files in tests/templates/ with new naming convention
  • Update all test imports and references
  • Reorganize test structure for better maintainability

Benefits:

  • Improved test maintainability and readability
  • Better coverage of edge cases and error scenarios
  • Consistent patterns across the test suite
  • Easier debugging and test development
  • Foundation for future testing improvements

Timeline: 2-3 weeks (made easier by modern TypeScript features from Phase 1.5a) Risk: Medium (extensive changes but with good test foundation)


Phase 2: Architecture Improvements (Week 3-5)

Priority: HIGH | Effort: 3 weeks

2.2 Separate Static Registries from Instance Methods

Problem: YumTemplate mixes static registration with instance processing

Solution: Create separate registry classes

// New classes:
class TagRegistry {
    static register(name: string, tag: ITagConstructor): void;
    static get(name: string): ITagConstructor | undefined;
}

class PartRegistry {
    /* similar */
}
class CultureRegistry {
    /* similar */
}

Files to Create:

  • src/registries/TagRegistry.ts
  • src/registries/PartRegistry.ts
  • src/registries/CultureRegistry.ts

Files to Update:

  • src/YumTemplate.ts - Remove static methods
  • Update all registration calls

Effort: 1 week | Testing: Ensure backward compatibility

2.3 Implement Resource Lifecycle Management

Problem: No cleanup/disposal methods

Solution: Add IDisposable pattern

interface IDisposable {
    dispose(): void;
    isDisposed: boolean;
}

class YumTemplate implements IDisposable {
    dispose(): void {
        this._parts.forEach((part) => part.dispose());
        this._zip = null;
        // Clean up resources
    }
}

Files to Update:

  • src/YumTemplate.ts - Add disposal
  • src/parts/AbstractPart.ts - Add disposal
  • src/tags/AbstractTag.ts - Add disposal

Effort: 3-4 days | Testing: Memory leak tests

2.4 Improve Error Handling Architecture

Problem: Generic error codes, poor error messages

Solution: Create structured error system

class YumTemplateError extends YumError {
    constructor(code: number, context: ErrorContext, suggestion?: string) {
        super(code, {
            message: this.buildMessage(code, context),
            suggestion,
            context,
        });
    }
}

Files to Create:

  • src/error/ErrorContext.ts
  • src/error/YumTemplateError.ts
  • src/error/errorMessages.ts

Effort: 3-4 days | Testing: Error handling tests


Phase 3: API Enhancements (Week 6-8)

Priority: MEDIUM | Effort: 2-3 weeks

3.1 Implement Fluent Builder Pattern

Problem: Limited configuration options, unclear API

Solution: Create fluent builder

class YumTemplateBuilder {
    private options: YumTemplateOptions = {};

    withDelimiters(start: string, end: string): YumTemplateBuilder;
    withLocale(locale: string): YumTemplateBuilder;
    withDebugMode(enabled: boolean): YumTemplateBuilder;
    withSecurityOptions(options: SecurityOptions): YumTemplateBuilder;
    build(): YumTemplate;
}

// Usage:
const template = new YumTemplateBuilder()
    .withDelimiters('<%', '%>')
    .withLocale('fr-FR')
    .withDebugMode(true)
    .build();

Files to Create:

  • src/builders/YumTemplateBuilder.ts
  • src/options/YumTemplateOptions.ts
  • src/options/SecurityOptions.ts

Effort: 1 week | Testing: Builder pattern tests

3.2 Enhanced Options System

Problem: Basic OptionsType lacks validation, schema

Solution: Comprehensive options with validation

interface YumTemplateOptions {
    delimiters?: DelimiterOptions;
    locale?: string;
    security?: SecurityOptions;
    performance?: PerformanceOptions;
    debugging?: DebugOptions;
    validation?: ValidationOptions;
}

class OptionsValidator {
    validate(options: YumTemplateOptions): ValidationResult;
}

Files to Create:

  • src/options/DelimiterOptions.ts
  • src/options/PerformanceOptions.ts
  • src/options/DebugOptions.ts
  • src/validation/OptionsValidator.ts

Effort: 1 week | Testing: Options validation tests

3.3 Consistent Async/Await Patterns

Problem: Mix of async/sync methods without clear rationale

Solution: Audit and standardize async patterns

  • Document processing: Always async
  • Registration methods: Always sync
  • Rendering: Always async
  • Serialization: Configurable (sync by default, async option)

Files to Update:

  • All tag implementations
  • All part implementations
  • src/YumTemplate.ts main methods

Effort: 1 week | Testing: Async behavior tests

3.4 Schema Validation for Templates

Problem: Templates not validated against expected structure

Solution: Template schema system

interface TemplateSchema {
    requiredTags: string[];
    allowedTags: string[];
    dataSchema?: JSONSchema;
}

class TemplateValidator {
    validate(template: YumTemplate, schema: TemplateSchema): ValidationResult;
}

Files to Create:

  • src/validation/TemplateSchema.ts
  • src/validation/TemplateValidator.ts
  • src/validation/ValidationResult.ts

Effort: 1 week | Testing: Schema validation tests


Phase 4: Performance & Security (Week 9-11)

Priority: MEDIUM-HIGH | Effort: 3 weeks

4.1 Expression Caching System

Problem: JEXL expressions re-evaluated unnecessarily

Solution: Multi-level caching

class ExpressionCache {
    private parseCache = new LRUCache<string, ParsedExpression>(1000);
    private evalCache = new LRUCache<string, unknown>(500);

    async evaluate(
        expression: string,
        context: Record<string, unknown>,
    ): Promise<unknown>;
}

Files to Create:

  • src/performance/ExpressionCache.ts
  • src/performance/LRUCache.ts

Files to Update:

  • src/tags/expressionEngine.ts - Integrate caching

Effort: 1 week | Testing: Performance benchmarks

4.2 Lazy Loading for Document Parts

Problem: All document parts loaded immediately

Solution: Lazy loading with dependency resolution

class LazyPart implements IPart {
    private _loaded = false;
    private _loadPromise?: Promise<void>;

    private async ensureLoaded(): Promise<void>;
    async render(data: Record<string, unknown>): Promise<void>;
}

Files to Create:

  • src/parts/LazyPart.ts
  • src/parts/PartLoader.ts

Effort: 1 week | Testing: Memory usage tests

4.3 Expression Sandboxing

Problem: JEXL can execute arbitrary expressions - security risk

Solution: Sandboxed expression engine

class SandboxedExpressionEngine implements IExpressionEngine {
    private allowedFunctions: Set<string>;
    private allowedProperties: Set<string>;

    async evaluate(
        expression: string,
        context: Record<string, unknown>,
    ): Promise<unknown> {
        // Validate expression before evaluation
        this.validateExpression(expression);
        return await this.safeEvaluate(expression, context);
    }
}

Files to Create:

  • src/security/SandboxedExpressionEngine.ts
  • src/security/ExpressionValidator.ts
  • src/security/SecurityPolicy.ts

Effort: 1 week | Testing: Security penetration tests

4.4 Input Validation & Size Limits

Problem: No validation or limits on inputs

Solution: Comprehensive input validation

class DocumentValidator {
    validateSize(document: ArrayBuffer): ValidationResult;
    validateStructure(document: JSZip): ValidationResult;
    validateContent(content: string): ValidationResult;
}

Files to Create:

  • src/security/DocumentValidator.ts
  • src/security/ContentSanitizer.ts
  • src/security/SecurityLimits.ts

Effort: 3-4 days | Testing: Security validation tests


Phase 5: Developer Experience (Week 12-14)

Priority: LOW-MEDIUM | Effort: 2-3 weeks

5.1 Enhanced Error Messages & Debug Mode

Problem: Generic error messages, no debugging capabilities

Solution: Rich error context and debug mode

class DebugYumTemplate extends YumTemplate {
    private debugOptions: DebugOptions;

    async render(data: Record<string, unknown>): Promise<void> {
        if (this.debugOptions.logSteps) {
            this.logRenderingSteps(data);
        }
        // Enhanced error reporting with context
    }
}

Files to Create:

  • src/debug/DebugYumTemplate.ts
  • src/debug/RenderLogger.ts
  • src/debug/ErrorReporter.ts

Effort: 1 week | Testing: Debug output validation

5.2 Advanced TypeScript Features

Problem: Basic TypeScript usage, could leverage advanced features

Solution: Enhanced type safety and developer experience

// Branded types for safety
type TemplateExpression = string & { __brand: 'template-expression' };
type LocaleCode = string & { __brand: 'locale-code' };

// Strict generics for better inference
interface YumTemplate<TData = Record<string, unknown>> {
    render(data: TData): Promise<void>;
}

// Template literal types for tag validation
type KnownTags = 'if' | 'each' | 'expression' | 'html' | 'image' | 'table';

Files to Update:

  • All interface definitions
  • Type definitions throughout codebase

Effort: 1 week | Testing: Type checking tests

5.3 Template Development Tools

Problem: Difficult to develop and test templates

Solution: Development utilities

class TemplateDeveloper {
    validateTemplate(template: string): ValidationReport;
    previewTemplate(template: string, data: any): PreviewResult;
    generateSampleData(template: string): any;
}

Files to Create:

  • src/dev-tools/TemplateDeveloper.ts
  • src/dev-tools/TemplatePreview.ts
  • src/dev-tools/SampleDataGenerator.ts

Effort: 1 week | Testing: Development tool tests


Phase 6: Advanced Features (Week 15-16)

Priority: LOW | Effort: 1-2 weeks

6.1 Plugin Versioning System

Problem: No version compatibility for custom tags/parts

Solution: Versioned plugin system

interface PluginMetadata {
    name: string;
    version: string;
    compatibleWith: string[];
    dependencies?: PluginDependency[];
}

class PluginManager {
    register(plugin: YumPlugin, metadata: PluginMetadata): void;
    validateCompatibility(metadata: PluginMetadata): boolean;
}

Files to Create:

  • src/plugins/PluginManager.ts
  • src/plugins/PluginMetadata.ts
  • src/plugins/YumPlugin.ts

Effort: 3-4 days

6.2 Template Compilation

Problem: Runtime template parsing impacts performance

Solution: Ahead-of-time compilation

class TemplateCompiler {
    compile(template: string): CompiledTemplate;
}

class CompiledTemplate {
    render(data: Record<string, unknown>): Promise<void>;
}

Files to Create:

  • src/compiler/TemplateCompiler.ts
  • src/compiler/CompiledTemplate.ts

Effort: 1 week

6.3 Streaming API for Large Documents

Problem: Everything loaded into memory

Solution: Streaming processing

class StreamingYumTemplate {
    processStream(
        inputStream: ReadableStream<Uint8Array>,
        data: Record<string, unknown>,
    ): ReadableStream<Uint8Array>;
}

Files to Create:

  • src/streaming/StreamingYumTemplate.ts
  • src/streaming/StreamProcessor.ts

Effort: 1 week


Implementation Guidelines

Testing Strategy

  • Unit Tests: Each phase requires 90%+ test coverage
  • Integration Tests: End-to-end template processing
  • Performance Tests: Benchmark before/after improvements
  • Security Tests: Validation and penetration testing

Breaking Changes Policy

  • Phases 1-2: May introduce breaking changes (major version bump)
  • Phases 3-6: Maintain backward compatibility where possible
  • Migration Guide: Document all breaking changes with migration paths

Code Review Process

  • Each task: Separate PR with detailed description
  • Peer Review: Minimum 2 reviewers for architectural changes
  • Documentation: Update CLAUDE.md with each significant change

Dependencies & Constraints

  • Phase Dependencies: Phase N cannot start until Phase N-1 is complete
  • Parallel Work: Within phases, independent tasks can be parallelized
  • Breaking Changes: Bundle breaking changes to minimize version churn

Success Criteria

  • Performance: 50% improvement in rendering speed
  • Security: Zero known vulnerabilities
  • Developer Experience: Comprehensive TypeScript support
  • Maintainability: Technical debt reduction measured by TODO count
  • Test Coverage: 95%+ coverage across all modules

Risk Assessment

High Risk Items

  1. Circular Dependency Resolution: May require significant refactoring
  2. Breaking Changes: Could impact existing users
  3. Performance Optimizations: Complex to implement correctly

Mitigation Strategies

  1. Feature Flags: Enable gradual rollout of new features
  2. Backward Compatibility: Maintain old APIs during transition
  3. Comprehensive Testing: Prevent regressions during refactoring

Timeline Summary

Phase Duration Focus Risk Level
1 Week 1-2 Critical Fixes Low
2 Week 3-5 Architecture High
3 Week 6-8 API Enhancement Medium
4 Week 9-11 Performance/Security Medium
5 Week 12-14 Developer Experience Low
6 Week 15-16 Advanced Features Low

Total Estimated Duration: 12-16 weeks Total Estimated Effort: 3-4 months of focused development