PgToolBoxConsole
A PgToolBoxConsole asks the operator to run the console for the CloudNativePG
Clusters of its namespace: the same console the pgtools chart installs
(console.enabled), as a resource the operator keeps converged. Both install paths
stay supported; pick one per namespace. Fields are in the
API reference.
apiVersion: pgtoolbox.fyannk.dev/v1alpha1
kind: PgToolBoxConsole
metadata:
name: console
spec:
auth:
provider: oidc
oidc:
issuerURL: https://idp.example.com/realms/corp
clientID: console-team-a
clientSecretRef:
name: console-team-a-oidc
exposure:
type: ingress
hostname: console.team-a.example.com
An empty spec is a working console: two replicas with persistent history (a
StatefulSet, one 1Gi volume each from the cluster's default StorageClass), token
sign-in, a ClusterIP Service at http://<name>.<namespace>.svc/ (status.url).
history: {persistent: false} keeps the history in memory, in a Deployment.
Chart or PgToolBoxConsole
pgtools chart (console.enabled) | PgToolBoxConsole | |
|---|---|---|
| Who installs | whoever runs helm install in the namespace | anyone who may create a PgToolBoxConsole there |
| Scope | the release namespace, or several (console.watchNamespaces) | its own namespace only |
| Read permissions | a Role the chart writes | the shipped ClusterRole pgtoolbox-console, bound in the namespace |
| Upgrades | helm upgrade | the operator's; the default image follows the operator |
| Settings | every chart value | the common ones (below); the rest stay chart-only |
The console binary, its environment and its mounts are the same: a test renders the chart and compares it with what the operator builds.
What the operator creates
All in the PgToolBoxConsole's namespace, controlled by it (deleted with it), named after
it (<name>, or <name>-console when the name does not already say console):
- the StatefulSet (persistent history, the default) or the Deployment (
history: {persistent: false}), its Service and ServiceAccount, and a PodDisruptionBudget when there is more than one replica; the StatefulSet makes one claim per replica,history-<name>-<ordinal>, deleted with it (persistent history); - RoleBindings of the ServiceAccount to
pgtoolbox-console, unlesspodLogs: falsetopgtoolbox-console-pod-logs, and withrestoreDrills.enabledtopgtoolbox-console-restore-drills(restore drills); - with
restoreDrills.enabled, the<name>-restore-drillsNetworkPolicy around the drill Pods, unlessrestoreDrills.networkPolicy.enabled: false(the drill NetworkPolicy); - the session-key Secret and the revocation-list ConfigMap, created once and
then left to the console (rotating the key signs everyone out), and a Role and
RoleBinding letting the console
getandwatchthat one ConfigMap; - the state ConfigMap
<name>-state, created once and then written by the console (what survives a restart), and a Role and RoleBinding letting the consoleget,watchandupdatethat one ConfigMap; the operator ignores the console's writes to it; - with
digest.delivery, the Lease<name>-digest, created once and then taken by the replica that delivers the daily digests, and a Role and RoleBinding letting the consolegetandupdatethat one Lease andcreateleases (RBAC cannot scope a create by name; the console creates only that one, should it be deleted); - the lifecycle ConfigMap (below);
- unless
spec.instanceCABundleRefis set, the instance CA bundle<name>-instance-ca(below); - for a published exposure, the
<name>-ingressNetworkPolicy that admits only the router, ingress controller or gateway to the console port (trusted proxies); - the Route, Ingress or HTTPRoute of
spec.exposure, with a one-hour timeout for the console's event stream and cookie affinity (pgtoolbox-console-replica) on an Ingress (ingress-nginx's annotations, and Traefik's on the Service) or a Route, so a person keeps seeing one replica's charts;spec.exposure.annotationsoverride it.
The operator writes no cluster-scoped object for a PgToolBoxConsole: the console checks
each reader's permissions with that reader's own token (SelfSubjectAccessReview).
Reconcile loop
One pass of the PgToolBoxConsole controller, in dependency order. Every child is
written with server-side apply, except the session Secret, the revocation
ConfigMap and the state ConfigMap, which are created once:
Before it renders anything, the operator checks that every Secret and
ConfigMap the console Pod would mount and that you name exists in the
namespace. The check reads metadata only, never content, and stalls with
ConfigurationValid=False; the message names each missing object and the
field that references it. The existing workload is left as it is, rather than
rolled to Pods that would wait in ContainerCreating.
| References | Reason | Picked up |
|---|---|---|
Secrets: auth.oidc.clientSecretRef (with the oidc provider), auth.openshift.clientSecretRef (with openshift), prometheus.tokenSecretRef, the digest's chatWebhookSecretRef, webhookSecretRef and smtp.passwordSecretRef, restoreDrills.sealKeySecretRef (with drills on) | SecretNotFound | as soon as the Secret is created (the operator watches Secret metadata) |
ConfigMaps: auth.oidc.caConfigMapRef, auth.openshift.caConfigMapRef, prometheus.caConfigMapRef, instanceCABundleRef, lifecycleConfigMapRef | ConfigMapNotFound | within two minutes: the operator caches only its own ConfigMaps, so it looks again rather than wait for an event |
Missing Secrets are reported first. instanceCABundleRef and
lifecycleConfigMapRef are mounted optional (the operator's own
<name>-instance-ca and <name>-lifecycle may not exist yet), but a ConfigMap
you name and that is absent would leave every Cluster "not verifiable" or the
diagnostics without their end-of-life table, so it stalls too. Not checked: the
platform's kube-root-ca.crt and openshift-service-ca.crt, image pull Secrets
(the kubelet goes on without one) and the Ingress certificate (the ingress
controller reads it). An object that exists without the referenced key is
reported by the Pod instead (WorkloadReady=False, SecretKeyMissing for a
Secret).
An object of a generated name that belongs to someone else stalls the console
with ResourceNotOwned at whichever step meets it. Bindings and policies that a
turned-off setting no longer needs (podLogs, restoreDrills, digest.delivery,
a single replica's PodDisruptionBudget) are deleted when this console controls
them. The controller
is woken by the PgToolBoxConsole, the objects it owns, the console's Pods, and, for the
CA bundle, the namespace's Clusters (created, deleted, relabelled, or naming
another CA Secret) and changes to their server CA Secrets, and the Secrets the
console mounts; it has no periodic
requeue except the two-minute look for a missing ClusterRole or referenced
ConfigMap.
Persistent history
spec.history.persistent is true unless set to false: the console is a
StatefulSet of spec.replicas (two), podManagementPolicy: Parallel, rolling
one Pod at a time, each replica with its own claim history-<name>-<ordinal> of
spec.history.size (1Gi; see sizing)
in spec.history.storageClassName (the cluster's default when omitted). Claims
are deleted with the StatefulSet (so with the PgToolBoxConsole) and kept when scaling
down. Without a default StorageClass, set storageClassName or persistent: false: otherwise the claims stay Pending and WorkloadReady says the Pods
cannot be scheduled.
Switching between persistent and in-memory history, the operator creates the new workload and deletes the old one only once the new one has a replica available: both serve meanwhile, behind the same Service.
Changing the size or the class. A StatefulSet's claim template cannot be
changed, so the operator deletes the StatefulSet without its Pods and creates
it again with the new template (WorkloadReady is RolloutInProgress in
between); the running Pods keep serving and are adopted. Existing claims keep
their size (grow one by editing its PVC, where the StorageClass allows
expansion); claims made later take the new one.
Permissions
The operator binds the shipped console ClusterRoles without holding their rules: it
has the bind verb on those three names only. So a PgToolBoxConsole reads exactly what
pgtoolbox-console grants, in its own namespace, and a person who may create a
PgToolBoxConsole gains nothing: every page still answers with what the reader's own
RBAC allows (a SelfSubjectAccessReview sent with the reader's token). If an installation lacks one it needs,
the console stalls with ConsoleRoleMissing and looks again every two minutes.
The Roles it writes (the revocation list, the state ConfigMap, the digest Lease)
grant only verbs the operator holds itself, as RBAC escalation prevention
requires: hence its get, create and update on leases, which it uses only to
create each delivering console's Lease.
The lifecycle table
The end-of-life dates the diagnostics read come from a ConfigMap,
<workload>-lifecycle, the way CloudNativePG treats its default monitoring
queries:
- None there: the operator creates it with the table built into the operator.
- Created by the operator and untouched: the operator keeps it at its own table, so an operator upgrade brings new end-of-life lines.
- Edited by someone, or annotated
pgtoolbox.fyannk.dev/user-managed: "true": it is theirs; the operator never writes it again. Delete it to go back to the operator's table. spec.lifecycleConfigMapRef: the console reads that ConfigMap; the operator neither creates nor reads it, beyond checking that it exists.
status.lifecycle.keptBy says which (Operator, User, Referenced), and
status.lifecycle.reviewed the date of the operator's table when it is in use.
The instance CA bundle
The console verifies each instance's /pg/status against the server CAs of the
Clusters it shows (why). For
a PgToolBoxConsole there is nothing to do: the operator keeps that bundle itself, in
the ConfigMap <name>-instance-ca (key ca.crt), and the console mounts it.
- For every Cluster of the namespace that
spec.clusterSelectormatches, the operator reads the server CA Secret (status.certificates.serverCASecret, elsespec.certificates.serverCASecret, else<cluster>-ca) and copies itsca.crtonly:ca.keynever leaves the Secret, and only the certificate blocks ofca.crtare copied. The console still reads no Secret, and its ServiceAccount gets no new permission. - A CA shared by several Clusters (one user-managed
serverCASecretfor the namespace) appears once. The order is the Clusters' names, so an unchanged set rewrites nothing. - CloudNativePG gives its self-signed CA 90 days and renews it 7 days before it expires; the renewal, a new Cluster and a deleted one reach the bundle within seconds (the operator watches the Clusters and their CA Secrets), the kubelet refreshes the mounted file and the console reloads it, without a restart.
status.instanceCABundlegives the ConfigMap,keptBy: Operator, how many Clusters are in it and, for those that are not, the Cluster, its Secret and why (SecretNotFound,SecretKeyMissing,InstanceCAInvalid,InstanceCABundleFull). The advisory conditionInstanceCABundleCompleteisFalsewithInstanceCAMissingwhile one is missing, with one Warning Event per transition; it never makes the console un-Ready, and the Cluster reads "not verifiable" until its CA is there.- If a ConfigMap that is not this console's already has the name
<name>-instance-ca(for instance one made by hand before the operator kept it), the operator leaves it alone and reportsInstanceCABundleComplete=FalsewithResourceNotOwned: delete it and the operator takes over.
spec.instanceCABundleRef turns this off: the console mounts the named
ConfigMap, which you keep current, and the operator neither reads nor writes it,
beyond checking that it exists (status.instanceCABundle.keptBy: Referenced). A copy made once by hand stops
verifying when the copied CA certificate expires, within 90 days; keep it current with
one of the automatic ways listed in
the chart's case.
A mirror left from before the reference was set stays in place, unused, until the
PgToolBoxConsole is deleted.
Images
spec.image is optional: the operator runs --default-console-image, which the
chart sets to the console image of the same release. An image from another
repository must be on --console-image-repositories (chart
operator.console.imageRepositories), else ImageNotAllowed; see
images.
Trusted proxies and the ingress NetworkPolicy
The console reads X-Forwarded-For only from a peer it trusts
(client address), so that a
Pod calling the Service directly cannot pose as a new client on every request.
The operator follows the exposure it publishes, not the one requested: a
Route without the Route API, an HTTPRoute without the Gateway API, or a hostname
refused by EXPOSURE_ALLOWED_HOST_SUFFIXES puts nothing in front, so nothing is
trusted.
spec.exposure | NetworkPolicy <name>-ingress | TRUSTED_PROXY_HOPS / TRUSTED_PROXY_CIDRS | ExposureConfined |
|---|---|---|---|
clusterIP | none | none | absent |
route (published) | on by default: the console port admits namespaces labelled policy-group.network.openshift.io/ingress (the OpenShift routers, host-network ones included), plus from | 1 / 0.0.0.0/0,::/0 | True |
ingress, gateway with networkPolicy.from | the console port admits only from | 1 / 0.0.0.0/0,::/0 | True |
a peer of from admits (nearly) everyone | rendered as written | 1 / none: the header is ignored | False, ExposurePeersBroad |
| the API server refuses the rendered policy (its schema, or an admission policy or webhook that denies it) | checked by a dry run first (skipped while the policy in force already is the rendered one): the previous one stays, and no exposure is published or changed | unchanged (the workload is not rolled) | ConfigurationValid=False, ExposurePolicyInvalid, retried every five minutes; ExposureConfined still describes the policy in force, or says ExposureOpen when none is and an exposure is published |
| another NetworkPolicy admits more to the console port | rendered | 1 / none | False, ExposurePolicyBypassed |
an ipBlock peer and no console Pod address yet, never verified (creation, or blocks no earlier Pod address checked), or more than 250 console Pods | rendered | 1 / none until a Pod address is known | False, ExposureTrustUnverified |
an ipBlock peer, the only Pod replaced or scaled from zero after an earlier Pod address verified the same blocks | rendered | unchanged: 1 / 0.0.0.0/0,::/0 (the workload is not rolled) | True |
from widens while a Pod trusts every peer | the previous one stays until the rollout without the grant is through, then widens | 1 / none during that rollout | False, ExposureTrustUnverified |
ingress, gateway without from | none: the controller's namespace differs per installation, so no default is safe | 1 / none: the header is ignored | False, ExposureOpen |
networkPolicy.enabled: false | none | 1 / none | False, ExposureOpen |
the policy goes (exposure or networkPolicy turned off) while a Pod may still trust every peer | kept until the rollout without the grant is through and no Pod carries it | 1 / none | False, ExposurePolicyKept, naming a Pod stuck terminating |
-
spec.exposure.networkPolicy.fromlists the router, ingress controller or gateway Pods (NetworkPolicy peers, at most 16). For the community ingress-nginx:spec:exposure:type: ingresshostname: console.team-a.example.comnetworkPolicy:from:- namespaceSelector:matchLabels:kubernetes.io/metadata.name: ingress-nginxpodSelector:matchLabels:app.kubernetes.io/name: ingress-nginxWithout
froman Ingress or a Gateway has no policy and the console ignoresX-Forwarded-For: every user behind the controller shares its address, so the console charges no per-address sign-in budget and only its per-replica bounds apply.enabled: truewithoutfromis refused for ingress and gateway. Each peer needs apodSelector, anamespaceSelectoror anipBlock(alone in its peer, a CIDR whoseexceptentries lie inside it); selectors take at most 16 labels and 16 expressions, label keys and values in their syntax, an operator amongIn,NotIn,Exists,DoesNotExist, and values forInandNotInonly. The CRD refuses the rest, which the API server would refuse in the policy; what it still refuses (a label-key prefix longer than 253 characters) stalls the console withExposurePolicyInvalidinstead of failing every reconcile. -
Every peer admitted is trusted to write
X-Forwarded-For, and so can choose the address its requests are charged to. List the proxies only, or name them intrustedProxy.cidrs, which then is the trust whatever the policy admits. A peer that admits (nearly) everyone turns the any-peer trust off and reportsExposurePeersBroad. The rule: a selector is broad when it pins no label value (nomatchLabelsand noInexpression; onlyExists,NotIn,DoesNotExist, or nothing), since a key alone is on nearly everything (kubernetes.io/metadata.nameis on every namespace). Broad are anamespaceSelectorthat is, or that pins only labels many namespaces share (pod-security.kubernetes.io/*,security.openshift.io/*,openshift.io/run-level:pod-security.kubernetes.io/enforce: restrictedis on most namespaces), apodSelectorthat is with nonamespaceSelector, and anipBlockwider than/8(IPv4) or/32(IPv6), holding a whole private IPv4 range (10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,100.64.0.0/10, where Pod networks are cut from), a unique-local IPv6 block (insidefc00::/7) wider than/64(kind'sfd00:10:244::/56, a/48), holding a console Pod's own address, or overlapping a Node'sspec.podCIDRs(the Pods of that node: the operator lists the Nodes, uncached, 100 a page and at most 10,000, one lookup at a time for every console, and keeps what it read for a minute; CNIs that do not allocate fromspec.podCIDRs, such as Calico or Cilium's cluster pool, leave it empty, and the console Pods' own addresses are then the only check: theExposureConfinedmessage says that the Node check could not run, or how many Nodes had no range; Calico IPAM or Cilium's cluster pool with kube-controller-manager still fillingspec.podCIDRsleaves ranges there that are not the Pod network, and when a console Pod's address is outside its own Node's range the message says the Node check is not meaningful: the Node ranges prove nothing then, as a peer that covers none of them may still admit the cluster's Pods, though a peer that covers one is still reportedExposurePeersBroad). One lookup runs for at most ten seconds; one that stops keeps the pages it read and the next attempt resumes from them. An installation that does not grant the operatorlistonnodesskips that check and says so in theExposureConfinedmessage. The console-Pod check needs a Pod address: at creation, while no console Pod has one, anipBlockpeer is not trusted (ExposureTrustUnverified), and the trust comes with the Pod's address (one more rollout), never before it. The template then records which blocks were verified (the Pod annotationpgtoolbox.fyannk.dev/trusted-ipblocks, a hash), so that when the only Pod is replaced, or the workload scales from zero, the new Pod keeps the trust while it has no address (the Pod network does not change) instead of rolling again; changing the blocks (narrowing included) needs a new check, and so a Pod address. With at most 16 peers of at most a/8each, blocks together never cover every address either.exceptentries are not counted: a false alarm is the safe side. Labels that a namespace owner may set on their own namespace (a tenant label of Capsule or Rancher, say) cannot be told from an installer's: pinning one is the user's responsibility;kubernetes.io/metadata.nameis set by the API server and is the safe choice. -
Widening waits for the trust to go. When
fromwould admit a source the policy in force does not (a peer added or swapped, a block, a namespace instead of some of its Pods) while a console Pod may still trust every peer (the template carries the grant, its controller has not seen the latest template, or a Pod, terminating ones included, carries it), the operator keeps the policy in force, rolls the workload without the grant (ExposureTrustUnverified), and widens the policy once that is through; the trust is then granted again if the new peers allow it. Narrowing applies at once. A Pod stuck terminating with the grant (a lost node, a stuck finalizer) holds the widening until it is gone: the message names it and when its deletion was asked, so it can be force-deleted, and the operator looks again every five minutes besides watching it. -
NetworkPolicies add up. Another policy of the namespace that selects the console Pods and admits, to the console port, a peer
fromdoes not list (anallow-same-namespacepolicy, an allow-all) lets that peer in whatever this one says. The operator lists the namespace's NetworkPolicies (at most 250; more counts as unverifiable) and matches them against the template's labels and every live console Pod's (pod-template-hash, the StatefulSet's Pod name included); when one does, it trusts no peer and reportsExposurePolicyBypassedwith its name. A foreign peer counts as the exposure's when it is written exactly as one offrom, or lies inside one of its namespace-only peers (itsnamespaceSelectorcarries every label of that peer, whatever itspodSelector). For a Route, the routers' older namespace labelnetwork.openshift.io/policy-group: ingresscounts as the routers', so the OpenShift project template'sallow-from-openshift-ingressis no bypass. The operator caches only its own policies, so it looks again every five minutes while it trusts every peer or another policy bypasses its own: a policy added, or removed, in between is seen within that time. -
On OpenShift, the default project template also creates
allow-same-namespace, which really admits every Pod of the namespace to the console port: the any-peer trust is then off (ExposurePolicyBypassed) and stays off. Name the routers intrustedProxy.cidrs(the router Pods' addresses, or the nodes' for host-network routers) to keep per-client budgets there. -
The operator sees only
networking.k8s.io/v1NetworkPolicies:AdminNetworkPolicy,BaselineAdminNetworkPolicyand CNI-specific kinds (Calico, Cilium) that admit more to the console port are not seen, and the any-peer trust stays on. Where they exist, settrustedProxy.cidrs. -
A Route turns the policy on, and with it cuts every in-cluster client of the Service's
httpport that is not a router: a debug Pod, an in-cluster MCP or AI client, a synthetic probe calling/. Add them tofrom; on a Routefromadds to the routers, it never replaces them. They are then trusted to writeX-Forwarded-Foras well; name the routers intrustedProxy.cidrswhen that matters.kubectl port-forwardgoes through the kubelet and is not affected. -
The probe port (health and
/metrics) stays open to any source: kubelets and Prometheus reach it, and it never readsX-Forwarded-For. Egress is not policed by this policy. -
spec.exposure.trustedProxy.hops(0 to 8, default 1 when published) counts the proxies that append to the header: 2 for a load balancer in front of the router.trustedProxy.cidrsnames them by address instead of trusting every admitted peer, and is the way to keep per-client budgets without the policy (an Ingress whose controller's addresses you know). -
A NetworkPolicy is a boundary only where the CNI enforces it, and the operator cannot observe that:
ExposureConfined=Truesays what is declared, not what the network does. On a CNI that does not enforce policies (kindnet, flannel alone), any Pod can writeX-Forwarded-Forunder the any-peer trust: settrustedProxy.cidrs, ortrustedProxy.hops: 0. -
When the policy goes (the exposure or
networkPolicychanges), the operator first rolls the workload to the narrower trust and deletes the policy only once the rollout is complete (every replica updated and available, no other left) and no console Pod, terminating ones included, still carries its any-peer grant (the Pod annotationpgtoolbox.fyannk.dev/trusts-every-peer;trustedProxy.cidrsof0.0.0.0/0,::/0is your own trust and does not hold the policy back); until then it keeps the policy and says so (ExposureConfined=False,ExposurePolicyKept, present while the policy is kept even with nothing published; a Pod stuck terminating is named), and the workload and its Pods are watched, so it looks again as the rollout progresses. -
While the console is stalled on
HostnameNotAllowed(checked before anything else that stalls, the image included) the operator withdraws the exposure but leaves the workload and this policy as they were, so a stale0.0.0.0/0trust is never left without its policy;ExposureConfinedthen says so (False,ExposurePolicyKept), and is removed only when the console has no ingress policy. -
The advisory
ExposureConfinednever entersReady; its message says which peers the console believes. -
Sign-in budgets follow the trust. With CIDRs (rows with
0.0.0.0/0,::/0ortrustedProxy.cidrs) each client keeps its own: ten OAuth callbacks and twenty pasted tokens a minute per address, under per-replica bounds sized to protect the API server and the IdP (100 code exchanges and 100 reviews of unknown tokens a second). With hops and no CIDRs (ExposureOpen,ExposurePeersBroad,ExposurePolicyBypassed), or with no hops behind a public URL (a clusterIP exposure behind an Ingress of your own), every client may have the proxy's address, so the console charges no per-address budget: callbacks are single-use and share the per-replica bounds, pasted tokens have a budget per token and the same bounds, and the audit log names the proxy instead of the client. A flood from one sender then refuses new sign-ins only above those bounds (rate limits).
Not in the custom resource
Kept chart-only for now: the console's own egress NetworkPolicy, the metrics ServiceMonitor, TLS on the Service, the break-glass settings, the IdP timeout and the cookie flags. Use the chart where you need them.