Skip to content

proflead/invisible-ai-chart-detector

Repository files navigation

Invisible AI Chart Detector

Detect • visualize • normalize invisible watermark characters and common “confusables” in text.

  • Watermarks (Unicode Cf): zero-widths (ZWSP/ZWNJ/ZWJ/WJ), BOM, bidi controls (LRE/RLE/LRO/RLO/PDF/isolates), etc.
  • Special spaces (optional): NBSP/NNBSP/Ideographic/En/Em/Thin/Hair/MMSP.
  • Confusables → ASCII: curly quotes, fancy dashes/minus, ellipsis, etc.

The extension can visualize problem characters as ⟦U+XXXX⟧, report exact locations, and normalize text by replacing where a safe ASCII alternative exists and removing the rest.


Why

Invisible controls and look-alike glyphs (e.g., U+2019 vs ', U+00A0 vs space) break search, parsing, and diffs, and are often used for watermarking or sneaky edits. This tool makes them obvious and fixes them safely.


Features

Visualize

  • Toggle to replace target characters with ⟦U+XXXX⟧ so you can see & search them.
  • Run the same command again to restore the original characters losslessly.

Normalize (Replace-or-Remove)

  • When a safe ASCII equivalent exists → replace (e.g., NBSP → space, curly quotes → '/").
  • Otherwise (true invisibles like ZWSP, bidi controls, BOM) → remove.

Report

  • Output channel report listing line:column and the Unicode code point for every target character found.

Workflows

  • In-place clean the current file.
  • Save a cleaned copy… to keep the original untouched.

Commands

  • Toggle Visualize (Replace ↔ Restore)
    Invisible AI Chart Detector: Toggle Visualize (Replace ↔ Restore)
    invisibleAiChartDetector.toggleVisualize

  • Normalize & Strip (replace if possible, remove otherwise)
    Invisible AI Chart Detector: Normalize & Strip (replace if possible, remove otherwise)
    invisibleAiChartDetector.removeAll

  • Scan & Report (Output)
    Invisible AI Chart Detector: Scan & Report (Output)
    invisibleAiChartDetector.scanReport

  • Clean In Place (replace if possible, remove otherwise)
    Invisible AI Chart Detector: Clean In Place (replace if possible, remove otherwise)
    invisibleAiChartDetector.cleanInPlace

  • Clean & Save Copy…
    Invisible AI Chart Detector: Clean & Save Copy…
    invisibleAiChartDetector.cleanSaveCopy

Note: the legacy commands Toggle (Diagnostics/Decorations) and Fix Double LFs were removed.


What gets replaced vs removed?

Replaced (→ ASCII)

Kind Unicode Replacement
NBSP family & special spaces U+00A0, U+202F, U+205F, U+3000, U+2000–U+200A space (U+0020)
Single quotes / primes U+2018, U+2019, U+201B, U+2032 '
Double quotes U+201C, U+201D, U+201F, U+2033 "
Dashes / minus U+2010, U+2011, U+2012, U+2013, U+2014, U+2015, U+2212 -
Ellipsis U+2026 ...
Soft hyphen U+00AD removed (acts like a layout hint)

Removed (no safe visible equivalent)

  • Unicode Cf format controls:
    ZWSP U+200B, ZWNJ U+200C, ZWJ U+200D, WJ U+2060, LRM/RLM/ALM U+200E/U+200F/U+061C, bidi format/overrides U+202A–U+202E/U+202C, isolates U+2066–U+2069, invisible math ops U+2061–U+2064, BOM U+FEFF, etc.
  • (Optional) Newlines if you enable that setting (off by default).

Normal ASCII space (U+0020), TAB (U+0009), and LF/CR are not targets unless you explicitly enable the newline options.


Settings

All settings are under Invisible AI Chart Detector:

  • invisibleAiChartDetector.watermark.includeUnicodeCf (default true)
    Treat all Unicode format controls (Cf) as targets.

  • invisibleAiChartDetector.watermark.includeSpaceFamily (default true)
    Treat NBSP/NNBSP/Ideographic/En/Em/Thin/Hair/MMSP as targets.

  • invisibleAiChartDetector.detect.includeNewlines (default false)
    Also flag LF/CR in diagnostics (usually keep false).

  • invisibleAiChartDetector.replace.format (default unicode)
    Visualization style. Regardless, we wrap as ⟦U+XXXX⟧ to restore safely.

  • invisibleAiChartDetector.replace.includeNewlines (default false)
    When visualizing, also turn LF/CR into visible tokens (not recommended).

  • invisibleAiChartDetector.remove.includeNewlines (default false)
    When cleaning, also delete LF/CR (dangerous; flattens files).

  • invisibleAiChartDetector.eolBadge (default false)
    Draw a small at each line end (visual aid only).

Example settings.json snippet:

{
  "invisibleAiChartDetector.watermark.includeUnicodeCf": true,
  "invisibleAiChartDetector.watermark.includeSpaceFamily": true,
  "invisibleAiChartDetector.detect.includeNewlines": false
}

Privacy

This extension performs all scanning and cleaning locally and does not transmit your data. It does not include telemetry or analytics. See PRIVACY.md for details.

License

MIT © Vladislav Guzey (https://proflead.dev)

About

Detect and visualize invisible/control characters in VS Code. Highlights LF (`U+000A`), CR (`U+000D`), other C0 controls/DEL, flags consecutive blank lines, and can draw a ⏎ badge at each end-of-line. Includes a quick fix to collapse multiple blank lines.

Resources

License

Stars

Watchers

Forks

Packages

 
 
 

Contributors