feat(platform): add GitProvider abstraction for Bitbucket support - #824
feat(platform): add GitProvider abstraction for Bitbucket support#824Agsiegert wants to merge 4 commits into
Conversation
…s-samples#456) Introduce a strategy-pattern GitProvider abstraction across all four packages (agent, cdk handlers, cdk constructs, cli) so the platform supports both GitHub and Bitbucket repositories. The abstraction is driven by a `provider` field on the RepoTable row ("github" | "bitbucket", defaulting to "github" for backward compatibility). The field flows from RepoConfig → orchestrator payload → agent TaskConfig, and provider selection is data-driven at every layer. Key changes: - New Python GitProvider protocol with GitHubProvider + BitbucketProvider (clone, PR creation via REST API 2.0, credential helpers, token patterns) - New TypeScript GitProviderOps interface with provider-dispatched preflight checks (reachability, repo access, PR accessibility) - Bitbucket webhook receiver Lambda (X-Hub-Signature validation, pullrequest:comment_created filtering, @bgagent mention detection) - Blueprint construct persists provider field in DynamoDB - CLI `bgagent repo onboard --provider bitbucket` support - Provider-aware output scanning for Bitbucket token patterns - Self-reclone guard extended for bitbucket.org URL forms Safe to merge: the provider field defaults to "github" everywhere, so the abstraction is inert until a repo is onboarded with provider=bitbucket. Closes aws-samples#456 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add nosemgrep inline directives for urllib.request.urlopen calls in BitbucketProvider — URLs are built from validated repo slugs, not arbitrary user input. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Both are intentional degraded-mode: webhook secret resolution failure skips signature check gracefully; permission fetch failure falls through to next preflight check. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #824 +/- ##
=======================================
Coverage ? 92.20%
=======================================
Files ? 323
Lines ? 90968
Branches ? 9054
=======================================
Hits ? 83880
Misses ? 7088
Partials ? 0 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Review:
|
| tool output | before | after |
|---|---|---|
x-access-token:SECRET@github.com/o/r |
redacted | redacted |
x-access-token:SECRET@github.mycorp.com/o/r |
redacted | leaks |
x-access-token:SECRET@ghe.internal/o/r |
redacted | leaks |
The only existing test (agent/tests/test_output_scanner.py:77) uses a github.com URL, so all 192 agent tests still pass. Suggest reverting to the unanchored pattern and adding a non-github.com case.
Related: agent/src/shell.py:126 (redact_secrets) matches x-access-token: but not x-token-auth:, so Bitbucket URL credentials aren't redacted at that layer at all.
5. cdk/src/handlers/bitbucket-webhook.ts:98 — the webhook fails open on signature verification.
if (webhookSecret) { ...verify... } means verification is skipped entirely whenever resolveWebhookSecret() returns undefined — which happens if the ARN env var is unset, if Secrets Manager throttles or returns AccessDenied (:48-53), or if the secret value is empty (cachedWebhookSecret is then falsy at :42). In that state any unauthenticated caller who POSTs a crafted pullrequest:comment_created body containing @bgagent gets a task dispatched.
cdk/src/handlers/github-webhook.ts:73-81 is the pattern to follow — it 401s a missing or invalid signature unconditionally, and shared/github-webhook-verify.ts rejects an empty secret via isUsableHmacSecret precisely because HMAC('', body) is forgeable. Reusing verifyGitHubRequest's structure would also pick up its 5-minute TTL cache and invalidateGitHubWebhookSecretCache; the module-level cachedWebhookSecret at :39 has no TTL or invalidation, so a warm Lambda rejects every delivery indefinitely after a secret rotation.
The nosemgrep justification on :52 ("skips signature verification gracefully") is what to reconsider here — the masking rule is intended for cases where a caller can safely treat failure as an empty success, which doesn't hold for an auth check.
6. cdk/src/handlers/bitbucket-webhook.ts:193 — dedup isn't rolled back when dispatch fails.
The dedup row is committed at :157-165 before the processor invoke. If Invoke throws, the handler returns 500 with the row already written and a 1h TTL. Bitbucket retries, attribute_not_exists(dedup_key) fails, the handler returns 200 {deduped: true}, and the @bgagent mention is dropped for the rest of the hour with no user-visible signal.
github-webhook.ts:198-212 does the DeleteCommand rollback for exactly this, with the comment "Roll the dedup row back so GitHub's retry can try dispatch again."
7. agent/src/git_provider.py:206 — the Bitbucket credential helper writes the raw token to disk.
helper_script = f'!f() {{ echo "username=x-token-auth"; echo "password={token}"; }}; f'This lands in <repo_dir>/.git/config, inside the workspace the agent controls. agent/src/repo.py:248-255 documents why the GitHub path deliberately avoids this:
Embedding the token in the remote URL would persist it in
.git/configinside the workspace the agent (and any code it runs) fully controls — readable by a prompt-injected step,git remote -v, or anything that copies the tree. The helper resolves credentials at call time from theGH_TOKEN/GITHUB_TOKENenv vars […] so the token never touches disk.
Neither shell.py redaction nor output_scanner._PATTERNS covers the bare password=<token> form, so a git config --list or cat .git/config in tool output would emit it unredacted. (BitbucketProvider.token_patterns() is also never consumed by the scanner, and it only matches the URL form rather than the ATCTT… / ATBB… token formats.) The token is also interpolated into a shell snippet without escaping.
Currently latent since the method has no call site, but it would activate as part of fixing issue 3 — an env-var-based helper matching the GitHub path avoids it.
8. cdk/src/handlers/shared/git-provider.ts:182 — push-permission detection can't return a correct false.
let pushAccess = true; // Default to true if we can access the repo
if (permResp.ok) { ... }Two problems:
- Fail-open.
/2.0/repositories/{repo}/permissions-config/usersis an admin-scoped endpoint, so a normal write-scoped token gets 403 — the common case, not the rare one — andpushAccessstaystrue. Preflight then admits a read-only token and the task fails hours later atgit push, which is the opposite of what preflight is for. - Wrong endpoint. That endpoint lists the explicit permission grants configured on the repo, not the caller's own effective permission. So
perms.some(p => p.permission === 'write' || p.permission === 'admin')is true whenever any user on the repo has write, regardless of what the token can do. The caller's own permission comes from/2.0/user/permissions/repositories?q=repository.full_name="{repo}".
The unit test at cdk/test/handlers/shared/git-provider.test.ts:271 ("defaults pushAccess to true when permissions endpoint fails") locks the fail-open in as intended behaviour, and the mocks return values: [{ permission: 'write' }] with no user identity, which is what hides the second problem. Worth asserting what the check should do here rather than what it currently does.
9. cdk/src/handlers/shared/git-provider.ts:153 — the reachability probe may be unreachable for the chosen auth mode.
checkReachability calls https://api.bitbucket.org/2.0/user. The PR description says "Access tokens (not OAuth app) for Bitbucket auth", and Atlassian's docs describe repository access tokens as "tied to a repository, not a user's account" and state they "can't be used to log in to your Bitbucket account" — so a user-account endpoint looks outside their boundary. If that's right, every Bitbucket preflight fails 401 even with a valid token.
The docs don't publish an exhaustive endpoint table, so this needs the live check that's still unticked in the test plan. A repository-scoped probe (e.g. GET /2.0/repositories/{repo}) would be safe for all credential types. Note the failure would surface to users as GITHUB_UNREACHABLE — bitbucket API returned HTTP 401, naming the wrong provider (see nits).
10. Bitbucket has no credential of its own — it reuses the GitHub secret.
preflight.ts gates on blueprintConfig.github_token_secret_arn and resolves via resolveGitHubToken(...), then hands that token to ops.checkReachability(token) for Bitbucket too; runner.py:133 maps config.github_token into BITBUCKET_TOKEN. Since orchestrator.ts:603 resolves repoConfig?.github_token_secret_arn ?? process.env.GITHUB_TOKEN_SECRET_ARN, a Bitbucket repo with no per-repo override would send the platform-wide GitHub PAT as a Bearer token to api.bitbucket.org. Operator-facing failures also read github_token_resolution / "GitHub token resolution failed" for a Bitbucket repo.
Latent today because of issue 1, but it activates with the fix. Bitbucket needs its own secret field, and the provider needs to be part of the resolver contract rather than implied.
This is the part worth settling before more code lands. ADR-016 names this exact seam — it cites resolve_github_token() as the first of "N hand-rolled resolvers" ("one shared PAT for all repos and users. No per-repo scoping") and observes that "a second provider arriving through the same resolve_<integration>_token() shape is live evidence the seam already exists; it just is not named or unified." But its credential taxonomy (ChannelCredential, McpCredential, McpRegistration) has no git-provider type, and the ADR is still proposed. So there's a genuine decision to record: whether a git-provider credential is a new type or reuses an existing one, and whether it's a direct access token or minted via the Token Vault (#249). A direct token is defensible — #249 Phase 1 was reordered Linear-first because the per-repo GitHub credential work is blocked on #50 — it just needs to be a recorded decision.
C. Correctness and operations
11. cli/src/commands/repo.ts:163 — the --provider default silently resets stored values.
.option('--provider <type>', 'Git provider: github or bitbucket', 'github')This is the only persisted option given a commander default. Verified: with no flag passed, opts.provider === 'github' while opts.computeType === undefined. So in repo-onboard.ts:110, if (options.provider) always fires, which means:
- the
else if (existing && 'provider' in existing)carry-forward branch is unreachable, and bgagent repo onboard ws/repo --max-turns 200on an already-onboarded Bitbucket repo silently rewritesproviderback togithub.
Every sibling option (--compute-type, --model, --token-secret-arn, --poll-interval) deliberately has no default so that omission preserves the stored value. Dropping the default here restores that contract and makes the carry-forward branch live.
Relatedly, RepoConfigRow (cli/src/repo-lookup.ts:30) has no provider field, so bgagent repo show never displays it — an operator can't confirm a repo's provider or notice the reset.
12. cdk/src/handlers/shared/preflight.ts:346 — the read-only permission gate is dropped for Bitbucket.
checkProviderRepoAccess doesn't forward readOnly to ops.checkRepoAccess(repo, token), and if (!readOnly && !result.pushAccess) then short-circuits — so a readOnly Bitbucket task passes with no permission assertion at all. The GitHub path (checkRepoAccess, :210-226) still requires PR_REVIEW_INTERACTION_LEVELS (TRIAGE or better) in that case. A token that can read a repo but not comment on PRs would be admitted, and fail only after the agent has burned a run trying to post.
13. cdk/src/handlers/bitbucket-webhook.ts isn't wired up, and reports success when it no-ops.
There's no API route, secret, dedup table, or processor for it — grep -ri bitbucket cdk/src/constructs cdk/src/stacks returns nothing, and there's no bitbucket-webhook-processor.ts (compare github-webhook-processor.ts, jira-webhook-processor.ts, linear-webhook-processor.ts). So the iteration loop is inert.
Separately, the if (DEDUP_TABLE_NAME) / if (PROCESSOR_FUNCTION_NAME) guards mean that with the env unset the handler skips dedup, skips the invoke, logs 'Bitbucket webhook dispatched', and returns 200 {ok: true, dispatched: true} — so a misconfigured deploy would look healthy. Suggest failing closed on missing required config instead.
If the webhook is out of scope for this PR, splitting it out would shrink the diff considerably. When it does land it'll need the ADR-002 bootstrap bundle update (new route + secret + table + Lambda): bootstrap/policies/*.ts, resource-action-map.ts, BOOTSTRAP_VERSION, regenerated artifacts, and the DEPLOYMENT_ROLES.md golden baseline.
Non-blocking
git-provider.ts:54-146—GitHubProviderOpsis unreachable in production (the preflight wrappers short-circuit onprovider === 'github'before callinggetProviderOps), yet it re-implements the live GitHub checks with less logic:checkRepoAccessdrops theviewerPermission/CONTENTS_WRITE_LEVELSfallback the live path depends on, andfetchViewerPermissionduplicatespreflight.ts:87verbatim.API_TIMEOUT_MSalso duplicatesGITHUB_API_TIMEOUT_MS. Routing GitHub throughGitProviderOpstoo would give one dispatch path instead of two GitHub implementations that will drift.bitbucket-webhook.ts:62—timingSafeEqualthrowsRangeErroron a same-length non-hex signature, because hex decoding truncates and the byte lengths then differ; the string-length precheck on :61 doesn't catch it. Confirmed:X-Hub-Signature: sha256=+ 64zchars yieldsRangeError: Input buffers must have the same byte length, which escapes to the outer catch and returns 500 instead of 401. Validating/^[0-9a-f]{64}$/ifirst fixes it.git_provider.py:349andgit-provider.ts:227both fall back to GitHub for any unrecognised value, so"gitlab"or"Bitbucket "would authenticate against github.com. Failing closed on an unknown provider would be safer.- Naming: non-GitHub emits
check: 'git_provider_reachability'while GitHub emits'github_reachability'in the same result array, and Bitbucket failures reuseGITHUB_UNREACHABLE/INSUFFICIENT_GITHUB_REPO_PERMISSIONS, so users see "GitHub" in messages about Bitbucket repos. Provider-neutral reason codes with the provider in the detail string would read better. cli/src/git-token.ts—validateGitTokenis imported only by its own test; no command calls it, sorepo onboard --provider bitbucketperforms no token validation. The three re-exports have no consumers either (knip: base 113 → head 116 findings, all three from this line). Also, forbitbucketthe function accepts any non-empty string including a GitHub token, andgit-token.test.ts:81asserts that as intended — a pasted GitHub token would pass Bitbucket validation.agent/tests/test_git_provider.pycovers only the trivial accessors;create_pr/view_pr/comment_pr/default_branchhave no tests, so no URL, request-body shape, or response parsing is exercised.test_satisfies_protocolis weaker than it looks —isinstanceagainst aruntime_checkableProtocol checks attribute presence only, not signatures.tokenis accepted but unused in all fourGitHubProviderAPI methods and incredential_helper_config; they rely on ambient env instead. Worth either using it or dropping it from the protocol.providersits undercredentialsinBlueprintPropsbut is top-level inRepoConfig— it's repo identity rather than a credential, so top-level in both would be more consistent.BitbucketProvider.default_branchignores itsrepoargument and requires a clone incwd, unlike the GitHub implementation — it returnsNoneif called before clone, which falls back to a hardcoded base branch.runner.py:131uses a function-local import, unlike the rest of that module.
Documentation
Nothing in this PR. New user-facing surface that needs it:
--providerflag →docs/design/REPO_ONBOARDING.mdanddocs/decisions/ADR-017-operator-cli-repo-onboarding.mdGIT_PROVIDERandBITBUCKET_TOKENenv vars →agent/README.md(peragent/AGENTS.md: "update README for new env vars")- an ADR for the credential-model decision in issue 10, and for the provider-dispatch pattern itself
No doc sources changed, so the Starlight mirror is in sync — but it'll need mise //docs:sync once docs land.
Tests and CI
Green locally: //cdk:compile, //cdk:eslint (no mutation), //agent:typecheck, //:check:types-sync (70 CLI ↔ 84 CDK exports), and 121 cdk + 192 agent + 13 cli tests.
Gaps:
bitbucket-webhook.tshas no test file, whilegithub-webhook,jira-webhook, andlinear-webhookeach do — 213 lines of HMAC, dedup, and dispatch logic untested.- The new preflight dispatch is uncovered. Coverage reports
preflight.tslines329-339, 348-371, 373-382, 386-401unhit — that's the entire non-GitHub branch of all three wrappers. All 28runPreflightCheckscall sites inpreflight.test.tsomit the new argument. - No existing suite was updated. The diff adds three new test files (+517) and touches zero existing tests, despite a changed
runPreflightCheckssignature, a new DynamoDB field inblueprint.ts, a new orchestrator payload key, and a new CLI option.
On CI: only four checks ran on 431d7534 — CodeQL and the three Analyze jobs didn't, which is expected for a fork PR rather than anything you did. Given the security surface here it's worth a maintainer mirroring the branch upstream so CodeQL runs before merge.
How the runtime claims above were verified
- Issue 1: throwaway jest test calling
loadBlueprintConfigwith a mocked DynamoDB row containingprovider: 'bitbucket', asserting on the returned config (removed afterwards). - Issue 4: both regexes run in Python against GitHub Enterprise / self-hosted host forms.
- Issue 11:
commanderparsed with and without--providerto confirm the default is always applied. - Nit 2:
verifyHmacSignatureextracted and run in node against a same-length non-hex signature. - Coverage line numbers from
mise //cdk:test -- test/handlers/shared/preflight. - Knip attribution by running
//:check:deadcode-ratcheton both the merge base and the PR head.
Happy to look again once the plumbing in section A is sorted — that's the piece that makes the rest testable end to end.
Summary
Adds a strategy-pattern GitProvider abstraction layer enabling Bitbucket as a first-class hosting provider alongside GitHub. Closes #456.
agent/src/git_provider.py): PythonGitProviderProtocol withGitHubProviderandBitbucketProviderimplementations — clone, PR creation (via Bitbucket REST API 2.0), credential helpers (x-token-auth), env vars, and token secret patternscdk/src/handlers/shared/git-provider.ts): TypeScriptGitProviderOpsinterface with provider-dispatched preflight checks (reachability, repo access, PR access, viewer permission)cdk/src/handlers/bitbucket-webhook.ts): Lambda handler forpullrequest:comment_createdevents with HMAC signature verification,@bgagentmention filtering, and DynamoDB deduplicationcli/src/git-token.ts,cli/src/commands/repo.ts):--provideroption onrepo onboard, provider-agnostic token validationGitProviderType = 'github' | 'bitbucket'in shared types;providerfield onRepoConfig/BlueprintConfig; backward-compatible (absent defaults to"github")agent/src/output_scanner.py): Bitbucket URL token pattern addedDesign decisions
urllib.requestfor Bitbucket API calls (no external deps) — the agent runtime avoids adding pip dependenciesTaskConfig.git_providerprovider="github")Test plan
agent/tests/test_git_provider.py— 15 tests covering factory, clone commands, remote URLs, env vars, host domains, protocol compliancecdk/test/handlers/shared/git-provider.test.ts— 31 tests covering both provider implementationscli/test/git-token.test.ts— 13 tests covering token validationNotes
Pre-push
security:sast:maskinghas 23 pre-existing findings onmain(unrelated to this PR). The 2 findings in our new files are addressed with justifiednosemgrepcomments.🤖 Generated with Claude Code