Skip to content

Fix adding authorized keys in Kubernetes and Slurm - #4235

Merged
un-def merged 2 commits into
masterfrom
pr_fix_add_authorized_keys
Sep 1, 2026
Merged

Fix adding authorized keys in Kubernetes and Slurm#4235
un-def merged 2 commits into
masterfrom
pr_fix_add_authorized_keys

Conversation

@un-def

@un-def un-def commented Aug 31, 2026

Copy link
Copy Markdown
Collaborator

The Kubernetes jump pod added the user's key with grep -qvF KEY file, which asks whether some line does not contain the key, not whether no line contains it. The jump pod's authorized_keys holds the project key, so the condition held whether or not the user's key was already there, and every run appended another copy of it. Users keep the same key across runs, so the file of a jump pod, shared by the whole namespace, grew by one duplicate per run. Had the file been empty, the condition would have been false and the key never added at all.

Slurm got the condition right, but matched the whole entry, comment included, so the same key submitted with a different comment was added a second time.

All three implementations -- SSH fleets, Kubernetes, Slurm -- are replaced with get_add_authorized_keys_script(), extracted from the SSH fleet one, the only one of the three that was correct. It returns the script instead of running it, as the call sites execute it in three different ways: paramiko, ssh over the jump pod, and the Slurm client.

  • Keys reach the script as heredoc data instead of being interpolated into its commands, so the shell never parses anything that came from a key.
  • Entries are rebuilt from parse_public_key(), a new helper parsing a public key in OpenSSH disk format, so that nothing unvalidated is written to the file. A key that fails to parse is skipped with a warning: one bad key does not keep the rest out of the file.
  • Entries are deduplicated on the key blob alone, since neither the comment nor the command="/bin/false" options Kubernetes and Slurm prepend are a part of the identity of a key.
  • Kubernetes and Slurm entries now carry the # added by dstack marker the SSH fleet entries have carried since Don't add duplicate keys while provision SSH instance #4179.

un-def and others added 2 commits August 31, 2026 14:47
The Kubernetes jump pod added the user's key with `grep -qvF KEY file`,
which asks whether some line does not contain the key, not whether no
line contains it. The jump pod's `authorized_keys` holds the project
key, so the condition held whether or not the user's key was already
there, and every run appended another copy of it. Users keep the same
key across runs, so the file of a jump pod, shared by the whole
namespace, grew by one duplicate per run. Had the file been empty, the
condition would have been false and the key never added at all.

Slurm got the condition right, but matched the whole entry, comment
included, so the same key submitted with a different comment was added
a second time.

All three implementations -- SSH fleets, Kubernetes, Slurm -- are
replaced with `get_add_authorized_keys_script()`, extracted from the
SSH fleet one, the only one of the three that was correct. It returns
the script instead of running it, as the call sites execute it in three
different ways: paramiko, `ssh` over the jump pod, and the Slurm
client.

* Keys reach the script as heredoc data instead of being interpolated
  into its commands, so the shell never parses anything that came from
  a key.
* Entries are rebuilt from `parse_public_key()`, a new helper parsing
  a public key in OpenSSH disk format, so that nothing unvalidated is
  written to the file. A key that fails to parse is skipped with a
  warning: one bad key does not keep the rest out of the file.
* Entries are deduplicated on the key blob alone, since neither the
  comment nor the `command="/bin/false"` options Kubernetes and Slurm
  prepend are a part of the identity of a key.
* Kubernetes and Slurm entries now carry the `# added by dstack`
  marker the SSH fleet entries have carried since #4179.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
@un-def
un-def merged commit 98745d9 into master Sep 1, 2026
27 checks passed
@un-def
un-def deleted the pr_fix_add_authorized_keys branch September 1, 2026 08:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant