Provice a list of files to prompt and explain the content of the files.

flowchart TD
 subgraph Configuration["Configuration"]
        B["Config Struct\nInitialization"]
  end
 subgraph subGraph1["Directory Traversal"]
        H["Add to Queue\nfor Traversal"]
        G{"Ignored Folders\nand Files Check"}
        F["Filter Directories\nand Files"]
        E["Traverse Function"]
        I["Collect Files\nwith Matching Extensions"]
  end
 subgraph subGraph2["File Processing"]
        K["Process Content\nLimit Lines, Clean Input"]
        J["Collect Output\nFunction"]
  end
 subgraph subGraph3["Template Rendering"]
        N["Render with Tera\nEngine"]
        M{"Embedded or Custom\nTemplate File"}
        L["Render Template\nFunction"]
  end
 subgraph subGraph4["Output Handling"]
        P["Save to Output File"]
        O["Output to Stdout\nand/or File"]
        Q["Display on Console"]
  end
    A["Command Line Input\nArgs Parsing"] -- User Inputs --> B
    B -- Configuration Data --> C["Process Files\nFunction"]
    C -- File List or Directory Traversal --> D{"Traverse Directory\nor Use Provided Files"}
    D -- Directory Path --> E
    E -- Read Directory --> F
    F -- Ignore Rules --> G
    G -- Valid Directories --> H
    G -- Valid Files --> I
    H -- Continue Traversal --> E
    I -- File Paths --> J
    D -- Provided Files --> J
    J -- Read File Contents --> K
    K -- Format Content --> L
    L -- Select Template --> M
    M -- Template Content --> N
    N -- Formatted Output --> O
    O -- Write to File if Specified --> P
    O -- Display --> Q
    E --> F
    F --> G
    G --> H & I
    J --> K
    L --> M
    M --> N
    O --> P & Q
    subGraph1 --> n1["Untitled Node"]

    style B fill:#000,stroke:#fff,color:#fff
    style H fill:#000,stroke:#fff,color:#fff
    style G fill:#000,stroke:#fff,color:#fff
    style F fill:#000,stroke:#fff,color:#fff
    style E fill:#000,stroke:#fff,color:#fff
    style I fill:#000,stroke:#fff,color:#fff
    style K fill:#000,stroke:#fff,color:#fff
    style J fill:#000,stroke:#fff,color:#fff
    style N fill:#000,stroke:#fff,color:#fff
    style M fill:#000,stroke:#fff,color:#fff
    style L fill:#000,stroke:#fff,color:#fff
    style P fill:#000,stroke:#fff,color:#fff
    style O fill:#000,stroke:#fff,color:#fff
    style Q fill:#000,stroke:#fff,color:#fff
    style A fill:#000,stroke:#fff,color:#fff
    style C fill:#000,stroke:#fff,color:#fff
    style D fill:#000,stroke:#fff,color:#fff

Directory File Processor is a Rust-based command-line tool that recursively traverses directories, filters files based on specified extensions and criteria, processes their content, and renders the result using customizable templates.

• Recursive directory traversal
• File filtering based on extensions
• Configurable file and folder exclusion
• Content processing with regular expressions
• Output rendering using customizable templates
• Multi-threaded processing for improved performance
• Support for file-based and standard output

To install the Directory File Processor, follow these steps:

1. Clone the repository:

   git clone https://github.com/yourusername/directory-file-processor.git
   cd directory-file-processor

2. Build the project using Cargo:

   cargo build --release

3. The executable will be located in the target/release directory.

To perform a traversal of the current directory and process all .rs files:

./target/release/directory-file-processor

To specify a different starting directory and file extensions:

./target/release/directory-file-processor --dir_path /path/to/dir --file_patterns "rs,txt"

To save the output to a specific file:

./target/release/directory-file-processor --output_file output.txt

To limit the number of lines processed from each file:

./target/release/directory-file-processor --lines 50

To clean the input using regular expressions before processing:

./target/release/directory-file-processor --clean_input_enabled true

To ignore specific files and folders:

./target/release/directory-file-processor --ignore_files "file1.rs,file2.txt" --ignore_folders "folder1,folder2"

To use a custom template for rendering the output:

./target/release/directory-file-processor --template custom_template

To list all available templates:

./target/release/directory-file-processor --list_templates

You can configure Directory File Processor using the following command-line arguments:

• --dir_path: Starting directory for the traversal (default: .).
• --file_patterns: Comma-separated list of file extensions to process (default: txt,py,c,go,rs,java,js,html,ts).
• --output_file: Path to the output file (default: empty, meaning standard output).
• --lines: Limit the number of lines to process from each file (default: -1, meaning no limit).
• --clean_input_enabled: Enable cleaning of input using regular expressions (default: true).
• --files: Optional list of specific files to process.
• --ignore_files: Comma-separated list of file names to ignore.
• --ignore_folders: Comma-separated list of folder names to ignore.
• --template: The template file to render the output (default: default).
• --list_templates: List all available templates.

We welcome contributions to improve Directory File Processor! To contribute:

1. Fork the repository.
2. Create a new branch with a descriptive name.
3. Make your changes and test them thoroughly.
4. Commit your changes with a clear commit message.
5. Push your changes to your forked repository.
6. Submit a pull request to the main repository.

Please ensure your code adheres to the existing coding standards and passes all tests.

You can test Directory File Processor by running:

cargo test

This command will execute all unit tests and integration tests defined in the codebase.

./target/release/files_to_prompt -d $(pwd) -e "rs,json" -l 50

./target/release/files_to_prompt -d $(pwd) -e "rs,json" -l 50 | llm -s "Explain this sources code and show results in the table" -m gemini-1.5-pro-latest

./target/release/files-to-prompt -d mistral.rs/ -e "rs,json" -l 50

./target/release/files-to-prompt -f Cargo.toml -f README.md

cargo build --release

This project is licensed under the MIT License. See the LICENSE file for details.

This project uses the following libraries and tools:

• Clap for command-line argument parsing
• Rayon for data parallelism
• Regex for regular expression support
• Tera for template rendering
• Rust-embed for embedding assets

Special thanks to all the open-source contributors and the Rust community for their valuable work.