Vim/hyp 184 cli hs follow command for websocket streams#74
Vim/hyp 184 cli hs follow command for websocket streams#74vimmotions wants to merge 91 commits intomainfrom
Conversation
…ase 1) Core streaming MVP: connect to a deployed stack's WebSocket, subscribe to a view (e.g. OreRound/latest), and stream entity data as NDJSON to stdout. Supports --raw mode for raw frames and merged entity output (default). Resolves WebSocket URL from --url, --stack, or hyperstack.toml. Also exports parse_frame, parse_snapshot_entities, ClientMessage, and deep_merge_with_append from hyperstack-sdk for reuse.
…s stream (Phase 2) - Filter DSL via --where: =, !=, >, >=, <, <=, ~regex, !~regex, ?, !? with dot-path support for nested fields (e.g. --where "user.age>18") - --first exits after first entity matches filter criteria - --select projects specific fields (comma-separated dot paths) - --ops filters by operation type (upsert, patch, delete) - --no-dna outputs NO_DNA v1 agent-friendly envelope format with lifecycle events (connected, snapshot_complete, entity_update, disconnected) - --count shows running update count on stderr - 9 unit tests for filter parsing and evaluation
…hase 3) - --save <file> records all raw frames with timestamps to a JSON file - --duration <secs> auto-stops recording after N seconds - --load <file> replays a saved snapshot through the same merge/filter pipeline (no WebSocket connection needed) - Snapshot format includes metadata (view, url, captured_at, duration)
…ase 4) EntityStore tracks full entity state + per-entity history ring buffer (default 1000 entries). Supports: - --history: outputs full update history for --key entity as JSON array - --at N: shows entity state at specific history index (0 = latest) - --diff: shows field-level diff (added/changed/removed) between updates with raw patch data when available These flags provide non-interactive agent equivalents of the TUI time travel feature. 5 unit tests for store operations and diffing.
Behind --features tui flag, `hs stream --tui` launches a ratatui-based terminal UI with: - Split-pane layout: entity list (30%) + detail view (70%) - Entity navigation (j/k, arrows), detail focus (Enter/Esc) - Time travel through entity history (h/l, Home/End) - Diff view toggle (d) showing field-level changes - JSON syntax coloring in detail panel - Pause/resume live updates (p) - Save snapshot to file (s) - Entity key filtering (/) - Raw frame toggle (r) - Status bar with keybinding hints - Timeline bar showing history position Dependencies: ratatui 0.29, crossterm 0.28 (optional) Without the tui feature, --tui prints an error with install instructions.
The server sends subscribed acknowledgments as binary frames with a different shape (no `entity` field), causing parse_frame to fail. Now falls back to try_parse_subscribed_frame before warning, so real parse errors are still surfaced while subscribed frames are handled cleanly. Re-exports try_parse_subscribed_frame from hyperstack-sdk.
Previously, keys like r, d, s, h, l etc. were matched as TUI commands before checking if filter input was active, making it impossible to type those characters in the filter. Now checks filter_input_active first and routes all Char keys to the filter text input.
The / filter now recursively searches all string, number, and boolean values in each entity's JSON data. Typing "test" matches any entity where any field value contains "test" (case-insensitive), not just entities whose key contains the search term.
When typing a filter that reduces the entity list, the selection now clamps to stay within the filtered results. Navigation (j/k) also bounds against the filtered count instead of the full entity list.
Uses ratatui's ListState with the selected index so the list widget automatically scrolls to keep the highlighted entity in view when navigating past the visible area. Also shows filter text and filtered/total count in the title when a filter is active.
- gg: jump to top of list - G: jump to bottom of list - Ctrl+d / Ctrl+u: half-page down/up - n: jump to next filter match (wraps around) - Number prefixes: e.g. 10j moves down 10, 5k moves up 5, 3Ctrl+d moves 3 half-pages down - Esc clears any pending count/g prefix
ListState was being recreated fresh each frame, losing the scroll offset. Now stored in App and synced with selected_index on every action, so ratatui properly auto-scrolls the entity list in both directions — pressing k after scrolling past the bottom now scrolls back up as expected.
The timeline bar now shows two distinct pieces of info: - Row position: "Row 1/513" (your position in the entity list) - Entity version: "version 1/2" (update history for the selected entity) Previously only showed "update N/N" which was confusing because it looked like it referred to list position rather than entity history.
The entity count is already shown in the list panel title, no need to repeat it in the header bar.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Greptile SummaryThis PR adds the Key changes:
Issues found:
Confidence Score: 4/5
Important Files Changed
Sequence DiagramsequenceDiagram
participant User
participant CLI as hs stream (mod.rs)
participant Client as client.rs
participant WS as WebSocket Server
participant Store as EntityStore
participant Output as StdoutWriter
User->>CLI: hs stream View/mode --url wss://...
CLI->>CLI: resolve_url() / validate_ws_url()
CLI->>Client: stream(url, view, args)
Client->>WS: connect_async(url)
WS-->>Client: WebSocket handshake
Client->>WS: Subscribe(view, key, take, skip, after)
loop Frame loop (tokio::select!)
WS-->>Client: Binary/Text Frame
Client->>Client: parse_frame()
alt Snapshot frame
Client->>Store: upsert(key, data, "snapshot")
Client->>Client: filter.matches(data)?
Client->>Output: emit_entity / NO_DNA / raw
else Upsert/Create
Client->>Store: upsert(key, data, op)
Client->>Output: emit_entity
else Patch
Client->>Store: patch(key, data, append_paths)
Client->>Client: deep_merge_with_append()
Client->>Output: emit_entity (merged state)
else Delete
Client->>Store: delete(key)
Client->>Output: print_delete
end
Client->>Client: --first? → break
Client->>Client: --duration elapsed? → break
User-->>Client: Ctrl+C → break
end
Client->>Client: SnapshotRecorder.save() if --save
Client->>Client: output_history_if_requested() if --history/--at/--diff
|
Previously the deadline was checked at the top of the loop before entering select!, so on quiet streams the actual stop time could overshoot by up to the 30-second ping interval. Now uses a sleep future as a select! arm that fires exactly when the duration expires.
…lisions
Previously --select "a.id,b.id" would output {"id": <b's value>},
silently overwriting a.id. Now uses the full dot-path as the output
key: {"a.id": 1, "b.id": 2}. Single-segment paths are unchanged
(--select "name" still outputs {"name": ...}).
Adds test for the collision case.
raw_frames were only collected when show_raw was active, and were never read by any rendering code. Now: - Always collects raw frames (so toggling on shows recent data) - selected_entity_data() checks show_raw first and returns the most recent raw WebSocket frame for the selected entity key
If the TUI panics, disable_raw_mode and LeaveAlternateScreen never executed, leaving the user's terminal in raw mode (unusable until running 'reset'). Now installs a panic hook before entering raw mode that restores the terminal state before re-invoking the original hook.
entity_keys.contains() was O(n) per frame, degrading with thousands of entities. Now maintains a parallel HashSet<String> for O(1) membership checks. HashSet::insert returns false if already present, so it doubles as the contains check. Delete also removes from the set.
- Use strip_suffix instead of manual string slicing for ? and !? suffix - Use is_some_and/is_none_or instead of map_or for Option comparisons - Use direct == Some() comparison instead of map_or for equality check
The snapshot_complete detection and NO_DNA event emission only existed in the Message::Text branch. Since the server primarily sends binary frames, consumers relying on the NO_DNA snapshot_complete lifecycle event would never see it. Now mirrors the same tracking logic in the binary frame branch.
Byte-index slicing panics when the cut point lands in the middle of a multi-byte codepoint (emoji, CJK characters). Now uses char_indices to find safe byte boundaries for truncation.
…nfigured Previously fell through to the first stack with any URL, silently connecting to an unrelated stack. Now only auto-selects when there is exactly one stack with a URL (unambiguous), and prints which stack was chosen. With multiple stacks, requires explicit --url or --stack.
Previously --first only triggered when a --where filter was present, silently running forever without one. Now --first always exits after the first output: with --where it exits on first match, without --where it exits after the first frame (raw) or entity (merged).
Adds comments explaining that two-char operators are checked before single-char to avoid misparsing, and that the split is on the first operator occurrence so values may contain operator characters (e.g. --where "name=a=b" works correctly).
… in --ops - TUI snapshot duration_ms now computed from first/last frame ts instead of recorder start_time which was near-zero - --ops upsert now also matches create frames (semantically identical) - Document panic hook limitation and history() ordering convention
- Reset pending_g after GotoTop (vim gg required double-press again) - Delete retains entity history with tombstone for post-stream analysis - SnapshotRecorder capped at 100k frames with warning - entity_count updated after snapshot loop, not per-iteration
Ctrl+D, Ctrl+U etc were being inserted as literal characters in the filter text. Now Ctrl+U clears the filter line, Ctrl+W deletes the last word, and other control/alt combos are ignored.
…rite - --ops create now correctly matches Create frames (normalized to upsert) - entity_count updated per-entity in snapshot loop so NoDna events have accurate counts - record_with_ts respects MAX_FRAMES cap - --load conflicts_with tui at Clap level (cleaner error) - Snapshot writes use tmp+rename for atomicity
…r errors - TUI WS task handles subscribed frames via try_parse_subscribed_frame fallback (previously silently dropped) - Snapshot tmp file placed in same directory as target (avoids EXDEV) - build_state called before connect_async for fast-fail on bad args - Empty field name in --where (e.g. "=value") now returns clear error
…lisions - Float equality uses string match first, then epsilon comparison (avoids f64 rounding issues for decimals like 1.1) - visible_rows subtracts 5 not 6 (header+timeline+status+2 borders) - Remove unreachable --load/--tui runtime check (clap handles it) - Snapshot filenames include milliseconds to prevent same-second collisions
The warning fired on every call after MAX_FRAMES since len never exceeded the cap. Now uses a limit_warned flag to print once.
…ove premature log - Delete of entity before cursor now shifts selected_index back to preserve which entity the user was looking at - --raw and --no-dna with --tui now bail with clear messages - Removed premature "Subscribing to..." printed before connection
…source - Moved connected event from build_state to after connect_async succeeds, so failed connections don't emit a connected event with no matching disconnected - Replay connected event includes "source": "replay" so consumers can distinguish live vs replay - --load now conflicts_with --duration at clap level
- Bail on --where/--select/--ops/--first with --tui (previously ignored) - TUI detects WebSocket disconnect and shows DISCONNECTED in header - Float equality uses exact bitwise comparison after string match (relative epsilon was too loose for large numbers) - Duration expiry sends WebSocket close frame before breaking
- finalize_count() clears the overwriting \r count line before post- stream messages (prevents garbled terminal output) - Snapshot write removes existing destination before rename (Windows compatibility where fs::rename fails if target exists) - Document that NoDna snapshot entity_count is a running tally - Document silent delete filter drop for unseen entities
- Remove-before-rename only runs on Windows (POSIX rename overwrites) - Propagate remove_file errors instead of silently swallowing them - Clean up tmp file on rename failure before propagating error
…okups - Output functions now use a shared BufWriter<Stdout> held in StreamState instead of acquiring/releasing stdout lock per call - Text WebSocket frames parsed once directly to Frame instead of double- parsing (Value then Frame) - diff_at stores entry in local variable instead of 3 redundant gets - TUI channel buffer increased to 10k for large snapshot batches - Document that filter cache invalidation is per-tick not per-frame
…emove dead flush - Replay snapshot_complete now checks received_snapshot (consistent with live stream path) - Document --select flattening behavior in help text - Document compute_diff as shallow top-level only - Document colorize_json_line serde_json assumption - Confirm --first semantics with comment - Remove unused StdoutWriter::flush (Drop impl suffices)
…g_count cap - StdoutWriter flushes after each writeln (prevents delays on low- throughput streams) - JSON key coloring uses "\": " to avoid matching colons inside keys - Snapshot serialization streams to file via BufWriter instead of building full JSON string in memory - Document --ops snapshot as valid value - Cap pending_count at 99999 to prevent usize overflow
hs stream— Live WebSocket Stream CLI + Interactive TUISummary
Adds a new
hs streamcommand that connects to a deployed stack's WebSocket and streams live entity data to the terminal. This is the CLI equivalent of what the TypeScript SDK does programmatically — but with filtering, recording, time-travel, and an interactive TUI for exploration.This was the highest-leverage missing UX feature: users could deploy stacks with
hs upbut had no way to observe live stream data without writing code.What's new
Core streaming (
hs stream <View> --url <wss://...>)Entity/modesyntax (e.g.PumpfunToken/list)--rawmode outputs unmerged WebSocket frames directly| jq,| head,| grep, etc.--urlexplicit,--stackfromhyperstack.toml, or auto-match from config--key,--take,--skip,--no-snapshot,--afterFiltering & triggers
--whereDSL with 10 operators:=,!=,>,>=,<,<=,~regex,!~regex,?(exists),!?(not exists)--where "info.symbol=TRUMP"--whereflags are ANDed together--firstexits immediately after the first entity matches the filter--selectprojects specific fields:--select "info.name,info.symbol"--opsfilters by operation type:--ops upsert,patch--countshows a running update count on stderrAgent-friendly output
--no-dnaoutputs NO_DNA v1 envelope format with lifecycle events (connected,snapshot_complete,entity_update,disconnected)Recording & replay
--save snapshot.jsonrecords all frames with timestamps--duration 30auto-stops recording after N seconds--load snapshot.jsonreplays through the same merge/filter pipeline (no WebSocket needed)Entity history & time-travel
EntityStoretracks per-entity history with a ring buffer (default 1000 entries)--history --key <key>outputs full update history as JSON--at N --key <key>shows entity state at a specific point in history--diff --key <key>shows field-level changes between updatesInteractive TUI (
--tui)Behind
--features tui(ratatui + crossterm):/): filters entities by searching all values in the JSON tree, not just keysh/lstep through entity version history, with diff view toggle (d)j/k,G/gg,Ctrl+d/Ctrl+u,n(next match), number prefixes (10j)p): freeze the stream while explorings): dump current recording to JSON filer)SDK changes
deep_merge_with_appendmade public for reuseparse_frame,parse_snapshot_entities,try_parse_subscribed_frame,ClientMessage,SnapshotEntityexported fromhyperstack-sdkUsage examples
New files
New dependencies (cli)
hyperstack-sdk— reuse Frame types, subscription protocol, merge logictokio,futures-util,tokio-tungstenite— async WebSocket clientratatui,crossterm— TUI (optional, behindtuifeature flag)Tests
PumpfunToken/listview