The members' identities¶
A cluster's members (the people deploying into its namespaces: Tenants) sign in with an identity of
their own, a security key, through the cluster's identity provider. The admins keep their certificates
(k8s/kubeconfig.yml, system:masters), which no identity provider stands between. What a member does: the Users
tab (Access).
member ──▶ kubectl / registry ──▶ Dex (auth.<zone>) ──▶ Pocket ID (id.<zone>): a security key
▲ │ ID token: oidc:<email>, oidc:<group>
└── apiserver, the public registry trust Dex alone
Common tasks¶
The members and their groups are the cluster's inventory (inventory/group_vars/all/identity.yml):
k8s_identity_groups: [aubonticket]
k8s_identity_users:
- {username: alice, email: alice@example.com, firstName: Alice, groups: [aubonticket]}
A user's email is their name to the cluster (oidc:<email>, which the tenants' values name), a group's name its
oidc:<group> (and the public registry's rights: Registry); isAdmin: true
makes a Pocket ID admin.
After any change of it, apply it, then give the operator the cluster's new inventory: it applies the configuration every 30 minutes with the inventory it was deployed with, and would put the old one back (a user's groups are replaced as declared).
ansible-playbook $ETHERE_ROOT_DIR/playbooks/k8s/identity.yml
ansible-playbook $ETHERE_ROOT_DIR/playbooks/operator/deploy.yml -e operator_image=<its current image>
Adding a member¶
- Their user (above), applied.
-
Their first security key: a login code,
prints its link (
https://id.<zone>/lc/<code>, validk8s_identity_login_code_ttl): hand it to them; they sign in with it and add their key (Add Passkey). The same for a lost key (another login code; remove the old key in Pocket ID). -
Their rights: a member's entry in the tenants' values (Tenants).
- Their workstation:
curl -fsS https://<zone>/setup.sh | sh(the Users tab: Access).
Removing a member¶
- Their rights first: their entries removed from the tenants' values (their bindings go: the cluster refuses them at once, whatever their tokens).
- Their user removed from the inventory, and deleted in Pocket ID by an admin (
https://id.<zone>, the admin's view): the playbook creates and updates the users it declares, it never deletes one. Until then they can still sign in, with no rights left.
Dex, the broker¶
- One issuer for the cluster: Dex (
playbooks/k8s/identity.yml,inventory/group_vars/all/identity.yml), onauth.<zone>through the Gateway, namespaceethere-identity, two replicas, its state in the API (storagekubernetes: its own kinds,dex.coreos.com), its image pinned by digest through the registry's cache. The apiserver and the public registry trust it alone; the identity providers are its connectors (a passkey provider, GitHub…), added without touching them. - Its configuration is a Secret (
k8s_dex_config): its connectors' secrets are there; local accounts (k8s_dex_static_passwords, bcrypt hashes) are for the tests only, with Dex's password grant. - Its clients:
kubernetes, public (kubelogin: the browser flow on a local port, or the device flow);zot, the public registry's (its secret generated in the cluster: the Secretdex-zot). - Part of the cluster's configuration (
k8s/configure.yml, which the operator runs) when the cluster has a zone (k8s_identity_enabled): its public names.
Pocket ID, the passkey provider¶
- Behind Dex (its connector
pocket-id, Dex's confidential clientdexin Pocket ID), onid.<zone>(k8s_pocket_id_*): one replica (its SQLite database on a Cinder volume, one instance at a time: a server's rotation stops new sign-ins for a minute or two, tokens already issued keep working), its image pinned through the registry's cache, distroless, non-root, read-only; nothing out (no heartbeat, no version check, no icon search). - Security keys only (
k8s_pocket_id_security_keys_only): an external authenticator (the Nitrokey's kind), no synced passkey, user verification required. Its application configuration is the environment's (UI_CONFIG_DISABLED), not its UI's. - Its keys are the cluster's: the data's encryption key and a static API key, generated once in the Secret
pocket-id; Dex's client secret index-pocket-id, given to Pocket ID (which keeps a hash). An admin needs none of them. - Configured through its API (
playbooks/k8s/_pocket_id.yml: a port-forward, the static key): the cluster's groups and users (verified emails: an admin declared them), Dex's client (its callback, PKCE, no consent screen). Its first admin's setup (/setup) is never routed: it closes once a user exists.
The apiserver trusts it¶
- Through the user-data (
provisioning/_userdata.yml), as the API's public name: the issuer is the cluster's, an image serves every cluster. The control plane's servers get/run/ethere/authentication.yaml(k8s_oidc_authentication_config:AuthenticationConfigurationv1), which the apiserver's start script passes (--authentication-config) when it's there: Dex's ID tokens for the clientkubernetes, the useroidc:+ the email (verified), the groupsoidc:+ the group (the tenants chart'suserPrefix). - It fetches Dex's keys through the edge, by the issuer's public name. A cluster boots without Dex (made after
it, by
k8s/configure.yml): the apiserver is ready meanwhile, tokens are refused until Dex answers. - A change of the issuer reaches new servers only (the user-data): a control plane rollout
(
spec.rollout.after, Cluster API providers). - The test cloud's zone isn't public: its servers resolve the issuer to their own edge (
k8s_oidc_issuer_address, the hosts file), and trust Pebble's root (k8s_oidc_ca_bundle).
The members' workstation¶
- A setup script (
share/member-setup.sh.j2,k8s_identity_setup_script), published by the documentation's server (curl -fsS https://<zone>/setup.sh | sh): kubectl at the cluster's version (dl.k8s.io, its checksum), kubelogin (k8s_identity_kubelogin_version, its release's checksums pinned), the cluster's context added to the kubeconfig, a sign-in. Rendered for the cluster: its name, the API, its CA, the issuer. - A kubeconfig, published too (
https://<zone>/kubeconfig,k8s_identity_kubeconfig): the API by its public name, its CA bundle (current and next: it follows the monthly rotation each timek8s/docs.ymlruns), the user kubelogin (kubectl oidc-login, installed in the ethere shell byprofile.d/20kubelogin.sh). Nothing secret.
Not yet¶
GitHub as a connector (a cluster's OAuth application); the users removed from the inventory deleted in Pocket ID (roadmap).