Hydroscope

What you are looking at

DBLog bootstraps a consistent copy of a table with a snapshot that takes rows in small chunks, interleaved with the live CDC stream. Each chunk is bracketed by a pair of watermarks (LW for low watermark and HW for high watermark), written to and read back from the log itself. Any log event that arrives between LW and HW touching a pk the chunk selected wins over the snapshot: the log version is newer, the snapshot row is dropped from the refresh emit.

Hydroscope is a read-only observer that attaches to DBLog's educational tap endpoint (GET /api/v1/tap/stream) and shows the algorithm from the outside: every source-log event on the top-left, every post-reconciliation sink event on the bottom-left, and the live state of the active chunk's reconciler on the right.

Build

cd ops/tap-tui
cargo build --release --bin hydroscope

Output: target/release/hydroscope. A single binary, no runtime dependencies beyond libc.

Run: demos & live

Offline / demo mode

Three built-in scenarios replay scripted event streams without touching a database. Same wire format as the live tap, so the UI rendering path is identical:

target/release/hydroscope --scenario showcase      # infinite, rich
target/release/hydroscope --scenario chunk42         # finite, 1 chunk, teaching
target/release/hydroscope --scenario chunk42-standby # chunk42 + standby pair
target/release/hydroscope --demo                     # alias for chunk42

See Scenarios below for what each one walks through.

Live mode

Omit --scenario / --demo and hydroscope connects to the DBLog tap:

target/release/hydroscope    # default: 127.0.0.1:8085
target/release/hydroscope --url http://host:port/api/v1/tap/stream

Requires the tap to be enabled in the DBLog runtime (see docs/OPERATION.md § 5.1.2 and docs/CONTROL_PLANE.md § 5.4 in the main repo). 503 tap_not_enabled surfaces as a clear disconnect banner with reconnect backoff. Run-id changes (process restart on the DBLog side) reset UI state.

Configuration (flags)

Runtime keys

Pace state is surfaced in the banner: full, N ms/event, or STEP (space=next).

UI anatomy

The screen is split 72% / 28% horizontally. Left column stacks SOURCE EVENTS (top) and SINK STREAM (bottom). The right column is the RECONCILER sidebar. Six distinct areas, each shown below at its actual rendered size so you can read the content:

Title bar

row 1 · always present

Brand marker on the left, then run <id> and source <source_id>. A run id longer than 17 characters is shortened to its first 8 and last 8 characters, joined by an ellipsis. Until a run id has been received, the run shows as «waiting».

The window state is not part of the title bar. It is on the RECONCILER sidebar. There, the Low Watermark and High Watermark markers are hollow triangles (▽, △) while the watermark is still awaited on the change stream. They become filled triangles (▼, ▲) once it has been received.

Connection banner

row 2 · priority-ordered

Priority order: standby > error > parse error > disconnect > connection state. When healthy, shows ● connected with the stream source, the latest request's id + scope + state, and current pace.

⚠ STANDBY and ✗ ERROR / ⚠ DISCONNECTED are the attention states. STANDBY shows the queue capacity and the queue_full_ms value reported when the standby began. ERROR stays for about 30 seconds of stream time and then gives way to the regular banner. DISCONNECTED shows retrying and the last disconnect reason.

Source events

top-left pane · 50% of the left column

Tap events in the order they arrived. Columns: seq, LSN, op, table / pk, kind, note.

Event kinds: cdc, wm-written, wm-received, chunk-sel. Colour maps to semantics below.

checkpoint.advanced events are producer-side bookkeeping. The TUI accepts them on the wire but intentionally drops them, so the pane stays focused on what drives the reconciler.

A collision does not get a row of its own. The cdc row that caused it turns yellow and carries the note collision in chunk N. The pane does not list chunk.collision and chunk.completed events. The reconciler sidebar reports the number of replaced rows as replaced-by-cdc.

Sink stream

bottom-left pane · 50% of the left column

What downstream consumers see after reconciliation. Columns: seq, LSN, op, table / pk, origin, badge.

LOG = CDC passthrough · SELECT = snapshot refresh row emitted on HW. The badge names the chunk. A LOG row that landed inside a watermark window shows chunk N · IN-WINDOW. A LOG row that replaced a snapshot row of the chunk shows chunk N · LOG WON in yellow. A SELECT row shows chunk N. A LOG row outside any window has an empty badge.

Reconciler sidebar

right column · 28% · full height

Live state of the active chunk only. Flows top-to-bottom in algorithm time-order:

1. chunk / table / pk [lo … hi]: chunk id, table and pk range
2. ▼ Low Watermark: LSN and latency once received (hollow ▽ while awaited)
3. chunk buffer: pk range visualisation, excluded pks crossed out
4. in-window cdc: collisions listed (up to 3), passthrough count
5. ▲ High Watermark: LSN and latency once received (hollow △ while awaited)
6. replaced-by-cdc: number of snapshot rows replaced by log events
7. emit-ready: refresh row count waiting on HW

With no active chunk the sidebar shows no chunk active.

Fixed-width sidebar: doesn't collapse or resize when the chunk lifecycle advances, so the source/sink panels stay visually anchored.

Footer

row -1 · always present

Keyboard shortcuts cheatsheet. See Runtime keys for the full list.

Colour semantics

Colours carry consistent meaning across panels:

Scenarios

Showcase intro, step by step

1. Pre-request warmup: heartbeat + plain CDC + checkpoint
2. Request 42 submitted (scope TABLE, app.orders) → ACTIVE
3. Chunk 1: 15 rows, 0 collisions (baseline pass)
4. Plain-CDC gap (DELETE + INSERT outside any window)
5. Chunk 2: 15 rows, 3 collisions (one is a DELETE hit)
6. stream.standby + stream.resumed (backpressure interlude)
7. Chunk 3: 20 rows, 5 collisions (heavy contention)
8. Chunk 4: 8 rows, 1 collision, final_chunk=true
9. Request 42 COMPLETED
10. Post-request plain CDC
11. An error event (WatermarkSequenceException, non-fatal)
12. Final heartbeat before the continuous loop starts

After the intro, the continuous loop emits deterministic (xorshift-seeded) random traffic: plain CDC bursts, chunks of 10-30 rows with 0-5 collisions each, heartbeats, checkpoints, occasional standby pairs, and non-fatal errors until you quit.

Troubleshooting

"terminal too narrow" / columns clipping

Hydroscope is designed around a ~120-column wide terminal. On narrower windows, the source-log and sink-stream tables drop rightmost columns first (kind, note, badge). The reconciler sidebar holds its content but LSNs may clip mid-string.

503 tap_not_enabled

Live mode landing on a DBLog runtime whose tap is disabled. Enable dblog.tap.enabled=true in application.properties and restart. Disconnect banner retries automatically with backoff.

Pace feels wrong

Defaults: 100 ms/event for showcase, 350 ms/event for chunk42 (slow enough to read), 0 for live. Override with --slowdown <ms> at launch or n / m / f at runtime.