Skip to content

The members' identities

A cluster's members (the people deploying into its namespaces: Tenants) sign in with an identity of their own, a security key, through the cluster's identity provider. The admins keep their certificates (k8s/kubeconfig.yml, system:masters), which no identity provider stands between. What a member does: the Users tab (Access).

member ──▶ kubectl / registry ──▶ Dex (auth.<zone>) ──▶ Pocket ID (id.<zone>): a security key
                 ▲                      │ ID token: oidc:<email>, oidc:<group>
                 └── apiserver, the public registry trust Dex alone

Common tasks

The members and their groups are the cluster's inventory (inventory/group_vars/all/identity.yml):

k8s_identity_groups: [aubonticket]
k8s_identity_users:
  - {username: alice, email: alice@example.com, firstName: Alice, groups: [aubonticket]}

A user's email is their name to the cluster (oidc:<email>, which the tenants' values name), a group's name its oidc:<group> (and the public registry's rights: Registry); isAdmin: true makes a Pocket ID admin.

After any change of it, apply it, then give the operator the cluster's new inventory: it applies the configuration every 30 minutes with the inventory it was deployed with, and would put the old one back (a user's groups are replaced as declared).

ansible-playbook $ETHERE_ROOT_DIR/playbooks/k8s/identity.yml
ansible-playbook $ETHERE_ROOT_DIR/playbooks/operator/deploy.yml -e operator_image=<its current image>

Adding a member

  1. Their user (above), applied.
  2. Their first security key: a login code,

    ansible-playbook $ETHERE_ROOT_DIR/playbooks/k8s/identity.yml -e k8s_identity_login_code=<username>
    

    prints its link (https://id.<zone>/lc/<code>, valid k8s_identity_login_code_ttl): hand it to them; they sign in with it and add their key (Add Passkey). The same for a lost key (another login code; remove the old key in Pocket ID).

  3. Their rights: a member's entry in the tenants' values (Tenants).

  4. Their workstation: curl -fsS https://<zone>/setup.sh | sh (the Users tab: Access).

Removing a member

  1. Their rights first: their entries removed from the tenants' values (their bindings go: the cluster refuses them at once, whatever their tokens).
  2. Their user removed from the inventory, and deleted in Pocket ID by an admin (https://id.<zone>, the admin's view): the playbook creates and updates the users it declares, it never deletes one. Until then they can still sign in, with no rights left.

Dex, the broker

  • One issuer for the cluster: Dex (playbooks/k8s/identity.yml, inventory/group_vars/all/identity.yml), on auth.<zone> through the Gateway, namespace ethere-identity, two replicas, its state in the API (storage kubernetes: its own kinds, dex.coreos.com), its image pinned by digest through the registry's cache. The apiserver and the public registry trust it alone; the identity providers are its connectors (a passkey provider, GitHub…), added without touching them.
  • Its configuration is a Secret (k8s_dex_config): its connectors' secrets are there; local accounts (k8s_dex_static_passwords, bcrypt hashes) are for the tests only, with Dex's password grant.
  • Its clients: kubernetes, public (kubelogin: the browser flow on a local port, or the device flow); zot, the public registry's (its secret generated in the cluster: the Secret dex-zot).
  • Part of the cluster's configuration (k8s/configure.yml, which the operator runs) when the cluster has a zone (k8s_identity_enabled): its public names.

Pocket ID, the passkey provider

  • Behind Dex (its connector pocket-id, Dex's confidential client dex in Pocket ID), on id.<zone> (k8s_pocket_id_*): one replica (its SQLite database on a Cinder volume, one instance at a time: a server's rotation stops new sign-ins for a minute or two, tokens already issued keep working), its image pinned through the registry's cache, distroless, non-root, read-only; nothing out (no heartbeat, no version check, no icon search).
  • Security keys only (k8s_pocket_id_security_keys_only): an external authenticator (the Nitrokey's kind), no synced passkey, user verification required. Its application configuration is the environment's (UI_CONFIG_DISABLED), not its UI's.
  • Its keys are the cluster's: the data's encryption key and a static API key, generated once in the Secret pocket-id; Dex's client secret in dex-pocket-id, given to Pocket ID (which keeps a hash). An admin needs none of them.
  • Configured through its API (playbooks/k8s/_pocket_id.yml: a port-forward, the static key): the cluster's groups and users (verified emails: an admin declared them), Dex's client (its callback, PKCE, no consent screen). Its first admin's setup (/setup) is never routed: it closes once a user exists.

The apiserver trusts it

  • Through the user-data (provisioning/_userdata.yml), as the API's public name: the issuer is the cluster's, an image serves every cluster. The control plane's servers get /run/ethere/authentication.yaml (k8s_oidc_authentication_config: AuthenticationConfiguration v1), which the apiserver's start script passes (--authentication-config) when it's there: Dex's ID tokens for the client kubernetes, the user oidc: + the email (verified), the groups oidc: + the group (the tenants chart's userPrefix).
  • It fetches Dex's keys through the edge, by the issuer's public name. A cluster boots without Dex (made after it, by k8s/configure.yml): the apiserver is ready meanwhile, tokens are refused until Dex answers.
  • A change of the issuer reaches new servers only (the user-data): a control plane rollout (spec.rollout.after, Cluster API providers).
  • The test cloud's zone isn't public: its servers resolve the issuer to their own edge (k8s_oidc_issuer_address, the hosts file), and trust Pebble's root (k8s_oidc_ca_bundle).

The members' workstation

  • A setup script (share/member-setup.sh.j2, k8s_identity_setup_script), published by the documentation's server (curl -fsS https://<zone>/setup.sh | sh): kubectl at the cluster's version (dl.k8s.io, its checksum), kubelogin (k8s_identity_kubelogin_version, its release's checksums pinned), the cluster's context added to the kubeconfig, a sign-in. Rendered for the cluster: its name, the API, its CA, the issuer.
  • A kubeconfig, published too (https://<zone>/kubeconfig, k8s_identity_kubeconfig): the API by its public name, its CA bundle (current and next: it follows the monthly rotation each time k8s/docs.yml runs), the user kubelogin (kubectl oidc-login, installed in the ethere shell by profile.d/20kubelogin.sh). Nothing secret.

Not yet

GitHub as a connector (a cluster's OAuth application); the users removed from the inventory deleted in Pocket ID (roadmap).