feat(logs): streaming foundation, new log flags, and vcr logs alias by valdemarpereira-vonage · Pull Request #93 · Vonage/cloud-runtime-cli

valdemarpereira-vonage · 2026-08-03T06:43:05Z

Rebuilds vcr instance log on a new source-agnostic pkg/logs package, adds time-window and regex filtering, a JSON output mode, and a top-level vcr logs shortcut.

This is phase 1 of 3. It is the foundation and is useful on its own; interactivity and a push-based transport follow.

What's new

New pkg/logs package, deliberately independent of any transport:

File	Responsibility
`source.go`	`Entry`, `Query`, `Page`, and the `Source` interface with a capability struct
`filter.go`	Level threshold, include/exclude RE2 patterns, replica selection
`buffer.go`	Bounded ring buffer for retained entries
`registry.go`	Dynamic replica registry with stable short ids (`r1`, `r2`, …)
`render.go`	Human renderer (date marker + `HH:MM:SS.mmm level message`) and one-object-per-line JSON
`graphql_source.go`	Hasura-backed `Source`: backfill, then poll while following

Command changes:

vcr instance log now runs on pkg/logs. Every original flag keeps its name, shorthand and meaning.
New top-level vcr logs alias — same command, fewer keystrokes.
New flags: --since, --from, --to, --grep, --exclude, --replica, --buffer, --json, --reverse, --utc, --source.

vcr logs -p my-app -n dev --since 15m --log-level error
vcr logs -p my-app -n dev --grep 'pay.*502' --exclude '/health' --json | jq .message
vcr logs -p my-app -n dev --follow

Behaviour changes to `vcr instance log`

vcr instance log is pre-existing, so these are worth a look before merge:

Output format changed. Was 2024-01-15T10:30:00Z [application] msg. Now a ==> YYYY-MM-DD marker on the first entry and on each date change, then 10:30:00.000 info msg. The [source_type] tag is gone (it is still available via --json). Scripts doing awk '{print $1}' or grep '\[application\]' will need updating.
A failed history fetch now exits non-zero. It previously printed a warning and exited 0.
An invalid --log-level now errors instead of silently dropping every line and exiting 0.
The interrupt notice moved from stdout to stderr and its wording changed.
--history no longer caps each follow poll — it now applies to the initial backfill only, which is what it was documented to do.

Fixed along the way

--follow no longer stops after ~10 minutes. It also now survives a transient fetch failure: a failed poll warns to stderr and retries, and only 10 consecutive failures give up (and then exit non-zero). Previously a single Hasura hiccup ended a tail with exit 1.
--to no longer silently reports "no matching log entries" when entries exist. The backing query can only bound the window's start and pages newest-first, so a full page could be discarded entirely client-side. That case is now detected and explained, pointing at --history/--from.
Buffered entries are drained on both the source-error and interrupt paths rather than dropped.
vcr instance log --help no longer shows examples that tell you to run vcr logs — each registration renders its own invocation.

Testing

go build ./..., go vet ./..., go test ./... all clean; go test -race clean on the three touched packages.
golangci-lint run ./... reports the same 3 pre-existing QF1012 findings as main — no new findings.
pkg/logs has unit tests per component, including concurrency tests for the buffer and registry.
Command-level tests cover flag validation, both output modes, the history and follow paths, the error and interrupt drains, and the alias registration.
Every behavioural fix here was verified by mutation: the old behaviour was reintroduced to confirm the new test actually fails, then reverted.
docs/vcr.md, docs/vcr_instance_log.md and docs/vcr_logs.md are regenerated and byte-identical to go run ./docs output.

Known follow-ups (deliberately not in this PR)

The upgrade banner writes to stdout, which interferes with vcr logs --json | jq. Pre-existing behaviour affecting every command, so it is left alone here — but worth fixing now that a machine-readable stdout format exists.
--replica is accepted but rejected at runtime until a replica-capable source lands in phase 3. --buffer is recorded but not yet read (phase 2 uses it).
RenderOptions.JSON and Query.Substring exist for phase 2 consumers and are currently inert.
The rest of docs/ has drifted from the command tree independently of this branch: 17 stale pages, 4 undocumented commands, and 5 orphaned vcr_mongo*.md files whose source no longer exists. Left for a separate cleanup, matching this repo's convention of committing only the doc pages a change affects.

Phases 2 and 3

Phase 2 — TTY detection and hotkeys (change filters live, select a replica, page back in time), a status footer, and the timestamp: {_lt: …} query that back-paging needs.
Phase 3 — an SSE Source against the log-stream endpoint, --source stream, and enabling replica selection end to end.

JSONLine ignored RenderOptions, so the UTC flag only affected human lines. It now normalizes the timestamp on a local copy of the entry the same way Line does, leaving the caller's entry untouched. colorLevel compared the raw level, so a source emitting "ERROR" lost its severity colour; the comparison is now lower-cased while the rendered text keeps its original case. Also documents that the renderer never branches on RenderOptions.JSON (the caller picks Line or JSONLine) and corrects replicaColorIndex's comment, which claimed to read a trailing digit run but folds digits from anywhere in the id.

…d Gray The colouring tests built their scheme with iostreams.Test(), which yields ColorScheme{Enabled: false} and makes every colour method the identity function, so the case-insensitive level match and the whole severity mapping were unpinned. Add tests on a colour-enabled scheme that compare escape wrapping between error/ERROR and require the red, yellow and muted arms to stay distinct while unknown levels stay bare. Also replace the deprecated ColorScheme.Gray with Muted (appearance preserving: Muted falls back to Gray when Accessible is false) to clear staticcheck SA1019, pin the default local-time path for both Line and JSONLine with timezone-derived expectations, drop a vacuous non-mutation assertion, and gofmt the test file.

Adapts the Hasura datastore to logs.Source. History returns one newest-first page newer than Query.From, drops entries after Query.To, and rejects cursor paging with ErrPagingUnsupported since the backing query can only express "newer than". Follow applies Query.Limit to the initial backfill only and the fixed FollowPageSize to every later poll, fixing the old command's re-use of the history limit on every poll.

Nine surviving mutants in pkg/logs/graphql_source.go came from missing assertions, not from a production defect. Close the gaps: - Cover Follow's error path with a live context, so replacing the wrapped error with 'return nil' now fails. - Record the instance id in the fake lister and assert it in a History and a Follow test, so passing "" no longer survives. - Assert the wrapper prefix alongside ErrorIs, so a bare 'return err' fails. - Give the Follow cursor test a non-zero Query.From so the first recorded call discriminates q.From from time.Time{}. - Add a test where the lister cancels the context then fails, pinning cancellation taking precedence over the error. - Add a test that drives a poll cycle with a non-positive interval, so removing the NewGraphQLSource guard panics. Every Follow test now has a bounded lifetime (context timeout plus timeout-guarded channel reads), so a regression fails instead of hanging CI. Also document that this source ignores Query.SourceType and Query.Substring because the backing GraphQL query has no such parameters.

… output flags Adds --since/--from/--to, --grep/--exclude, --buffer, --json, --reverse, --utc and --source. Fixes --follow being bounded by the global --timeout, and --history being re-applied on every poll instead of bounding the backfill. An unrecognised --log-level is now an error rather than silently discarding every line, so the old fail-closed behaviour is preserved explicitly.

…tc and source paths

Ctrl+C ends nearly every real --follow session, but the interrupt branch returned without draining, discarding up to 256 entries the source had already delivered. Both exit paths now call a shared non-blocking drain closure, so a stop never silently drops successful polls. The runHistory error branch was untested: every TestLog row left the datastore's list-logs error nil, so returning nil there passed the suite and vcr logs could exit 0 with empty output on a backend failure. Add a dedicated test pinning the branch, the wrapper, the underlying cause and the absence of the old wrapper stutter. Also reword the history error to failed to fetch log history to match the file's convention, and route the two remaining direct runFollow calls through the bounded helper so a non-returning regression fails fast.

The 'vcr logs' shortcut had no coverage of the root wiring: the existing guard test builds the command inside the log package, so deleting the cmd.AddCommand(logCmd.NewCmdLogs(f)) line left the suite green. - add a root-package test asserting a top-level command with Use == "logs" is present in NewCmdRoot(...).Commands(); asserting on Use also catches swapping in NewCmdInstanceLog, whose Use is "log" - document that root's --version must stay a LOCAL flag: 'vcr logs' uses -v for --exclude, so promoting it to PersistentFlags would panic in pflag - assert --grep in the log package's alias guard test

Both 'vcr logs' and 'vcr instance log' are built by newLogCmd and shared a single hardcoded example block, so 'vcr instance log --help' instructed users to run 'vcr logs ...' - a different command. Parameterise the examples on the fully-qualified invocation.

Covers only the log-related pages. The rest of docs/ has drifted from the command tree independently of this branch and is left for a separate cleanup.

Remediates the two blockers and six should-fix findings from the final whole-branch review of feat/logs-streaming. --follow no longer dies on a transient fetch error. GraphQLSource.Follow absorbs a failed poll, hands it to an injected WithFollowErrorHandler callback and retries on the next tick; only MaxFollowPollFailures consecutive failures return an error. The command installs a handler that prints the warning icon to ErrOut, restoring the pre-branch behaviour the design spec specifies. Resilience lives in the source because the spec gives Follow ownership of its own reconnect behaviour, and because nothing polls once Follow has returned. --to no longer claims a window is empty when it was never reached. The backing query can only bound From, ordered newest-first, so a full page can be discarded in its entirety by the client-side To filter. Page gains WindowTruncated for exactly that case and runHistory replaces the misleading "no matching log entries in range" with an actionable warning naming --history and --from. The proper timestamp: {_lt:} query stays deferred. Also: - print a muted "==> YYYY-MM-DD" marker whenever the calendar date changes, in both modes, human format only, honouring --utc - make JSONLine always emit UTC RFC3339 so machine-readable output does not vary with the host timezone - name --grep/--exclude in pattern errors instead of Filter's internal "include pattern" wording - drop the source's "failed to list logs" prefix so the user sees one "failed to ..." rather than two - reject non-positive --history and --buffer instead of silently substituting 200 and 5000 - cancel the source before draining on interrupt, so a second Ctrl+C is not swallowed while the drain renders entries a stopped producer never sent - return a copy from logs.LevelNames()

The generated reference already listed vcr logs under SEE ALSO, but root's hand-written Long and Example only ever showed vcr instance log, so the new entry point was undiscoverable from vcr --help. Adds it to CORE WORKFLOW and to the examples, and regenerates docs/vcr.md.

The new validators already name the offending flag, so 'failed to validate flags: --history must be a positive number' doubled up. Keep that prefix only on cmdutil.ValidateFlags, where it is the existing repo convention.

vonage-ai-assistant · 2026-08-03T06:44:01Z

AI Adoption Report

	Lines	%
AI-generated	0	0%
Human	3557	100%
Total	3557

Per-commit breakdown (20 commits)

Commit	Message	Added	Human	AI %
`0019472`	fix(log): stop wrapping self-describing flag errors in a gen	7	7	0%
`309d1ec`	docs(root): surface the top-level vcr logs command in root h	25	25	0%
`7d1014e`	fix(logs): survive transient poll failures and stop misrepor	895	895	0%
`a891212`	docs: regenerate CLI reference for the new log flags and vcr	211	211	0%
`022eb52`	fix(log): show each registration's own invocation in its exa	34	34	0%
`ac4d288`	test(root): pin the top-level logs command registration	27	27	0%
`963bf2d`	feat(log): add top-level 'vcr logs' alias	13	13	0%
`1eb9cd1`	fix(log): drain on interrupt and cover the history error bra	168	168	0%
`afba8d7`	fix(log): drain buffered entries on follow error; cover filt	341	341	0%
`7cae11c`	feat(log): rebuild instance log on pkg/logs with new time, f	440	440	0%
`f9006a3`	test(logs): cover graphql source error, cursor and instance-	127	127	0%
`8ee37f1`	feat(logs): add GraphQL log source with backfill-then-poll f	270	270	0%
`292fc4b`	test(logs): pin renderer severity colouring; use Muted over	98	98	0%
`40539cf`	fix(logs): honour UTC in JSON output and match levels case-i	89	89	0%
`52838b9`	feat(logs): add entry renderer with replica column and JSON	158	158	0%
`1c43a90`	test(logs): pin registry resolve precedence and document rep	41	41	0%
`682b8d2`	feat(logs): add dynamic replica registry with stable short i	181	181	0%
`f504848`	test(logs): enforce buffer copy and concurrency guarantees	59	59	0%
`4ea112b`	feat(logs): add bounded ring buffer for retained entries	91	91	0%
`f511c7b`	feat(logs): add source-agnostic log types and filtering	282	282	0%

_{Powered by Git AI Standard v3.0.0 — authorship data from refs/notes/ai (supports sessions-v2 format)}

Copilot

Pull request overview

Introduces a new transport-agnostic pkg/logs foundation and rewires the existing vcr instance log command onto it, while also adding a top-level vcr logs shortcut and regenerating docs.

Changes:

Added pkg/logs (source/query/page model, filtering, rendering, replica registry, buffering, and a Hasura/GraphQL polling source).
Rebuilt vcr instance log to use pkg/logs, added new time-window/regex/output/source flags, and added a top-level vcr logs entry point.
Expanded/updated unit tests and regenerated command docs to reflect the new UX and flags.

Reviewed changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
vcr/root/root.go	Registers the new top-level `vcr logs` command and updates root help text/examples.
vcr/root/root_test.go	Pins the root wiring and help prose to include `vcr logs`.
vcr/instance/log/log.go	Reimplements the log command on `pkg/logs`, adds new flags, and splits history vs follow execution paths.
vcr/instance/log/log_test.go	Adds extensive coverage for new flag parsing, output modes, follow/history behavior, and edge cases.
pkg/logs/source.go	Defines normalized `Entry`, `Query`, `Page`, `Caps`, and the `Source` interface.
pkg/logs/render.go	Adds human and JSON renderers (date markers + time/level formatting).
pkg/logs/render_test.go	Tests renderer formatting, coloring behavior, UTC handling, and date markers.
pkg/logs/registry.go	Adds a concurrency-safe registry mapping hostnames to stable short replica IDs.
pkg/logs/registry_test.go	Tests stable IDs, resolve behavior, and concurrent access.
pkg/logs/graphql_source.go	Implements a polling GraphQL/Hasura `Source` with bounded retry behavior for follow mode.
pkg/logs/graphql_source_test.go	Tests history ordering/window truncation, follow retry semantics, and cancellation behavior.
pkg/logs/filter.go	Adds level threshold + include/exclude regex + source-type + replica filtering.
pkg/logs/filter_test.go	Tests filter matching, summary formatting, and level ladder behavior.
pkg/logs/buffer.go	Adds an in-memory bounded buffer used to retain recent entries.
pkg/logs/buffer_test.go	Tests eviction behavior, snapshot copying, and concurrent add/snapshot safety.
docs/vcr.md	Updates generated root docs (help text, examples, SEE ALSO).
docs/vcr_logs.md	Adds generated docs page for the new top-level `vcr logs` command.
docs/vcr_instance_log.md	Updates generated docs for `vcr instance log` to match the new behavior/flags.

Suppressed comments (1)

vcr/instance/log/log.go:201

--to parsing drops the underlying parse error. Wrapping and returning the time.Parse error gives users a concrete reason the value was rejected while still calling out the expected RFC3339 format.

		t, err := time.Parse(time.RFC3339, opts.To)
		if err != nil {
			return logs.Query{}, nil, fmt.Errorf("invalid --to value %q: expected RFC3339", opts.To)
		}

+		t, err := time.Parse(time.RFC3339, opts.From)
+		if err != nil {
+			return logs.Query{}, nil, fmt.Errorf("invalid --from value %q: expected RFC3339", opts.From)
+		}


+// Buffer retains at most a fixed number of entries in a bounded slice. Once
+// full, adding an entry drops the oldest by shifting the remaining entries down
+// one position, so Add is O(n) in the capacity once the buffer is saturated.
+// It is safe for concurrent use: a source goroutine appends while the render


 		RunE: func(_ *cobra.Command, _ []string) error {
-			ctx, cancel := context.WithDeadline(context.Background(), opts.Deadline())
-			defer cancel()
-
-			return runLog(ctx, &opts)
+			return runLog(&opts)
 		},


valdemarpereira-vonage added 20 commits August 2, 2026 20:29

feat(logs): add source-agnostic log types and filtering

f511c7b

feat(logs): add bounded ring buffer for retained entries

4ea112b

test(logs): enforce buffer copy and concurrency guarantees

f504848

feat(logs): add dynamic replica registry with stable short ids

682b8d2

test(logs): pin registry resolve precedence and document replica fields

1c43a90

feat(logs): add entry renderer with replica column and JSON mode

52838b9

fix(log): drain buffered entries on follow error; cover filter, to, u…

afba8d7

…tc and source paths

feat(log): add top-level 'vcr logs' alias

963bf2d

docs: regenerate CLI reference for the new log flags and vcr logs alias

a891212

Covers only the log-related pages. The rest of docs/ has drifted from the command tree independently of this branch and is left for a separate cleanup.

Copilot AI review requested due to automatic review settings August 3, 2026 06:43

Copilot started reviewing on behalf of valdemarpereira-vonage August 3, 2026 06:43 View session

Copilot AI reviewed Aug 3, 2026

View reviewed changes

Navigation Menu

Uh oh!

feat(logs): streaming foundation, new log flags, and vcr logs alias - #93

feat(logs): streaming foundation, new log flags, and vcr logs alias#93
valdemarpereira-vonage wants to merge 20 commits into
mainfrom
feat/logs-streaming

valdemarpereira-vonage commented Aug 3, 2026

Uh oh!

vonage-ai-assistant Bot commented Aug 3, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

valdemarpereira-vonage commented Aug 3, 2026

What's new

Behaviour changes to vcr instance log

Fixed along the way

Testing

Known follow-ups (deliberately not in this PR)

Phases 2 and 3

Uh oh!

vonage-ai-assistant Bot commented Aug 3, 2026

AI Adoption Report

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Behaviour changes to `vcr instance log`