📖 Add KubeStellar Hive security self-assessment - #2286
Open
clubanderson wants to merge 1 commit into
Open
Conversation
Adds the CNCF TAG-Security self-assessment for KubeStellar Hive under projects/kubestellar/sub-projects/hive/security-assessment/, alongside the existing Console assessment. Hive orchestrates fleets of AI coding agents that autonomously maintain software projects — filing issues, opening pull requests, reviewing, and at high autonomy levels merging. The security model is therefore unusual for a CNCF project: it runs model output with credentials that can write to source repositories, so the assessment focuses on the boundaries that contain that. Covered: the ACMM autonomy levels and agent mode ladder that gate write and merge capability, ioscan prompt-injection scanning of untrusted input, the MITM proxy enforcing network egress rules, GitHub App credential handling, the append-only audit log that provides attribution, and the private vulnerability reporting process. The assessment states known weaknesses rather than omitting them, including that ioscan's default fail mode is open, that agents share a container rather than being sandboxed per run, and that the project currently has a single maintainer. It also records that no third-party security audit or penetration test has been performed. Source document: https://github.com/kubestellar/hive/blob/v4/src/docs/security-self-assessment.md Relative links from the source have been rewritten to absolute Hive URLs so they resolve from this repository. Signed-off-by: Andrew Anderson <andy@clubanderson.com>
JustinCappos
requested changes
Aug 31, 2026
JustinCappos
left a comment
JustinCappos
left a comment
Contributor
There was a problem hiding this comment.
This needs some work to deal with parts the LLM didn't have the data for. There is also some low-hanging fruit the project could do to improve security.
Comment on lines
+101
to
+104
| - Agents read GitHub issues, PRs, comments, labels, and diffs (untrusted text | ||
| boundary). | ||
| - Agents are "kicked" (launched) with a constructed prompt built from that | ||
| text plus repository/task context. |
Contributor
There was a problem hiding this comment.
How do you deal with a party who creates a malicious issue, comment, etc.? I think you're talking about this in the run-pipeline.sh and operator-selected ACMM level, but I don't see the big picture. If I deploy this, what should I worry about?
Comment on lines
+199
to
+209
| 2. **`ioscan` untrusted-input scanner** | ||
| (`src/pkg/ioscan`, `src/docs/ioscan.md`). Scans GitHub issue/PR/label/ | ||
| author/comment text before it enters an agent kick. Deterministic rules | ||
| (Unicode-steganography normalization, base64 decode-and-rescan, | ||
| prompt-injection phrasing, dangerous-directive and secret-shape detection) | ||
| are always the floor when enabled; blocked text is replaced with a visible | ||
| `[ioscan: content withheld — ...]` marker rather than silently dropped or | ||
| passed raw. An optional LLM-judge classifier | ||
| (`ioscan.classifier.enabled`, default `false`) adds semantic | ||
| plain-English-injection detection on top. **Enabled by default** | ||
| (`ioscan.enabled: true` is the default per `ioscan.md:9`). |
Contributor
There was a problem hiding this comment.
have you red teamed to know how well this works? What should a potential user be aware of here? Is this perfect defense, works decently well, a partial mitigation, etc.?
| - **Token budget** — a seven-day rolling token budget that suppresses kicks | ||
| on exhaustion, limiting denial-of-wallet from a runaway or compromised | ||
| agent loop. | ||
| - **Log scrubbing** (`src/pkg/logscrub`, `security.md`) — redacts recognized |
Contributor
There was a problem hiding this comment.
Is this to prevent an model with access to these from leaking them? What's the purpose?
If so, what if I prompt inject something in which has you emit these in a base 64 encoded format or similar?
Comment on lines
+504
to
+505
| has not pursued OpenSSF Best Practices Badge ("CII Best Practices") | ||
| certification at time of writing — not yet assessed whether this is planned. |
Contributor
There was a problem hiding this comment.
You should do this. At least get a passing badge please. It's not that hard and the things you need to do for this are sane security things you either are or should be doing already.
Comment on lines
+466
to
+468
| assessment identifies, and is exactly the kind of gap CNCF Incubation | ||
| review should weigh: it is a bus-factor and response-capacity risk, not a | ||
| code defect. |
Contributor
There was a problem hiding this comment.
Yes, this is a huge risk. Why not mitigate it?
Comment on lines
+509
to
+512
| Not applicable — Hive does not currently publish named case studies of | ||
| production deployments in the Hive repository. (Not the same claim as "Hive has | ||
| no production users" — simply that no case-study document exists to cite | ||
| here.) |
Contributor
There was a problem hiding this comment.
Looks like AI generated this for you. You should know the answer here and be able to write a correct response
Comment on lines
+525
to
+552
| Items this assessment could not verify from the repository alone, listed | ||
| explicitly rather than guessed at: | ||
|
|
||
| - Whether issue #3760 (cited as the reason SBOM/provenance attestation is | ||
| disabled on container builds) documents a planned path to re-enabling SBOM | ||
| generation, or is a permanent decision. | ||
| - Whether any external channel (mailing list, Slack, Matrix) exists for | ||
| security-relevant project communication beyond GitHub issues/PRs and | ||
| private security advisories. | ||
| - Whether a formal, security-specific incident-response runbook (as distinct | ||
| from the disaster-recovery runbook at `docs/HUB_DISASTER_RECOVERY.md`) | ||
| exists outside the Hive repository (e.g., in an internal-only document) — | ||
| none was found in the public repository. | ||
| - Whether the project intends to pursue OpenSSF Best Practices Badge | ||
| certification. | ||
| - The current numeric OpenSSF Scorecard result — intentionally not quoted | ||
| here since it changes independently of this document; consult the live | ||
| badge/workflow output. | ||
| - Whether any prior informal security review (e.g., a maintainer's own | ||
| adversarial testing beyond what is documented in the ADRs and threat | ||
| model) occurred without producing a public artifact. This assessment | ||
| treats "no third-party audit" as accurate based on the absence of any | ||
| audit report in the repository, but cannot rule out unpublished internal | ||
| review. | ||
| - Whether `.github/CODEOWNERS` enforcement (branch-protection "Require | ||
| review from Code Owners") is planned to be enabled once the project has | ||
| more than one active human maintainer, or is intended to remain advisory | ||
| indefinitely regardless of maintainer count. |
Contributor
There was a problem hiding this comment.
This is the LLM telling you to clarify these (as I see it). Please go through and figure these out and update the document before we merge it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
projects/kubestellar/sub-projects/hive/security-assessment/, alongside the existing Console assessmentWhat Hive is, and why the threat model is unusual
Hive orchestrates fleets of AI coding agents that autonomously maintain software projects — filing issues, opening pull requests, reviewing, and at high autonomy levels merging. Agents run as CLI subprocesses under tmux, in containers or pods, holding GitHub App or PAT credentials.
That means the system runs model output with credentials that can write to source repositories, so the assessment focuses on the boundaries that contain that rather than on a conventional network/data perimeter.
Covered:
ioscan— prompt-injection scanning of untrusted input on the kick path, with canary support0600, distinct from the OAuth/OIDC login pathWeaknesses are stated, not omitted
A self-assessment that claims everything is fine is not useful to a reviewer, so the document names its own gaps. The three most significant:
ioscan.fail_modedefaults toopen. A scanner or classifier outage degrades toward continuing to process untrusted input rather than blocking — a deliberate availability/security tradeoff, documented as such.It also records plainly that no third-party security audit or penetration test has been performed, and that SBOM/provenance attestations are currently disabled on image builds (with the reason, which is a real container-runtime exec failure rather than an oversight).
Notes for reviewers
References