aws2-7 Exercise 3: Pod Limits, IRSA Trust Conditions, and Manifests Checked Without a Cluster ======================================================================================== Run with Windows 11, Python 3.14 (py launcher) and a venv with jsonschema 4.26.0, PyYAML, cfn-lint 1.57.2; eksctl 0.231.0 and kind 0.31.0 downloaded as binaries into C:/lwaws/c7/bin (SHA-256 checked against the published checksums; not installed); Node 24.18.0 with aws-cdk 2.1145.0, aws-cdk-lib 2.273.0, constructs 10.8.1, typescript 7.0.2, jest 30.5.2 (the chapter 5 install, reached from a junction); kubectl client 1.32.2 (Docker Desktop's). Scratch folder C:/lwaws/c7, nothing installed globally. VERIFICATION LEVEL: 1 for the offline checks below (no AWS account, no credentials, no cluster created). A kind cluster (Kubernetes v1.35.0) was created but Docker Desktop's engine stopped before any experiment ran, so no Kubernetes API server result is reported. Outputs below are saved from the real runs of 2026-10-13. --- max_pods.py --- """Max pods per node with the Amazon VPC CNI. Documented formula (EKS user guide, 'How maxPods is determined'): (number of ENIs x (IPs per ENI - 1)) + 2. Data source 1: aws/amazon-vpc-cni-k8s pkg/vpc/vpc_ip_resource_limit.go (ENILimit, IPv4Limit per instance type), fetched 2026-10-13. Data source 2: aws/amazon-vpc-cni-k8s misc/eni-max-pods.txt (a published list of 'instance type max pods'), fetched the same day. Both files come from the same repository, so agreement shows they are consistent with each other, not that they match AWS today.""" import re, sys def formula(enis, ips_per_eni): return enis * (ips_per_eni - 1) + 2 def load_limits(path="out/vpc_ip_resource_limit.go"): src = open(path, encoding="utf-8").read() out = {} for m in re.finditer(r'\n\t"([a-z0-9.\-]+)": \{\n\t\tENILimit:\s+(\d+),\n\t\tIPv4Limit:\s+(\d+),', src): out[m.group(1)] = (int(m.group(2)), int(m.group(3))) return out def load_list(path="out/eni-max-pods.txt"): out = {} for line in open(path, encoding="utf-8"): if line.strip() and not line.startswith("#"): a, b = line.split()[:2]; out[a] = int(b) return out if __name__ == "__main__": lim, lst = load_limits(), load_list() print("instance types in the limits file:", len(lim), "| in the published list:", len(lst)) both = sorted(set(lim) & set(lst)) bad = [(t, formula(*lim[t]), lst[t]) for t in both if formula(*lim[t]) != lst[t]] print("types in both:", len(both), "| formula disagrees with the list for:", len(bad)) for t, f, l in bad[:12]: print(" %-16s formula %4d list %4d (ENIs, IPs/ENI = %s)" % (t, f, l, lim[t])) print("only in limits file:", len(set(lim) - set(lst)), "| only in list:", len(set(lst) - set(lim))) print() print("%-14s %5s %9s %9s" % ("type", "ENIs", "IPs/ENI", "max pods")) for t in ["t3.small", "t3.medium", "t3.large", "m5.large", "m5.xlarge", "m5.4xlarge", "c5.large", "m6g.large", "m5.24xlarge"]: if t in lim: print("%-14s %5d %9d %9d" % (t, lim[t][0], lim[t][1], formula(*lim[t]))) print() over = [t for t in lim if formula(*lim[t]) > 110] print("types whose formula exceeds 110:", len(over), "of", len(lim), "(a managed node group without a custom AMI caps maxPods at 110 below 30 vCPUs and 250 above, per the page; vCPU counts are not in these files)") --- py max_pods.py (data files fetched from github.com/aws/amazon-vpc-cni-k8s on 2026-10-13: pkg/vpc/vpc_ip_resource_limit.go and misc/eni-max-pods.txt) (saved output of the real run) --- instance types in the limits file: 1392 | in the published list: 1392 types in both: 1392 | formula disagrees with the list for: 0 only in limits file: 0 | only in list: 0 type ENIs IPs/ENI max pods t3.small 3 4 11 t3.medium 3 6 17 t3.large 3 12 35 m5.large 3 10 29 m5.xlarge 4 15 58 m5.4xlarge 8 30 234 c5.large 3 10 29 m6g.large 3 10 29 m5.24xlarge 15 50 737 types whose formula exceeds 110: 952 of 1392 (a managed node group without a custom AMI caps maxPods at 110 below 30 vCPUs and 250 above, per the page; vCPU counts are not in these files) --- trust_eval.py (imports iam_eval.py from chapter 2) --- """Evaluate a role's TRUST policy for a web-identity (IRSA-style) request, reusing the chapter 2 condition evaluator (iam_eval.cond_ok). Not AWS: it applies StringEquals/StringLike semantics to the token claims you give it. The policy shape is the commonly documented IRSA trust policy; the EKS page I read did not print the JSON, so the shape is from my knowledge of the IAM web-identity format.""" import json from iam_eval import cond_ok, glob, L def assume_with_web_identity(trust, provider_arn, claims): for s in trust["Statement"]: if s["Effect"] != "Allow": continue if "sts:AssumeRoleWithWebIdentity" not in L(s["Action"]): continue if provider_arn not in L(s.get("Principal", {}).get("Federated", [])): continue if cond_ok(s.get("Condition", {}), claims): return True return False def irsa_policy(provider_arn, issuer, ns, sa): cond = {"StringEquals": {issuer + ":sub": "system:serviceaccount:%s:%s" % (ns, sa), issuer + ":aud": "sts.amazonaws.com"}} return {"Version": "2012-10-17", "Statement": [{"Effect": "Allow", "Principal": {"Federated": provider_arn}, "Action": "sts:AssumeRoleWithWebIdentity", "Condition": cond}]} if __name__ == "__main__": ISS = "oidc.eks.eu-west-1.amazonaws.com/id/EXAMPLED539D4633E53DE1B71EXAMPLE" PROV = "arn:aws:iam::111122223333:oidc-provider/" + ISS OTHER = "arn:aws:iam::111122223333:oidc-provider/oidc.eks.eu-west-1.amazonaws.com/id/OTHERCLUSTER" claims = lambda ns, sa, aud="sts.amazonaws.com": {ISS + ":sub": "system:serviceaccount:%s:%s" % (ns, sa), ISS + ":aud": aud} strict = irsa_policy(PROV, ISS, "apps", "reports") loose_ns = {"Version": "2012-10-17", "Statement": [{"Effect": "Allow", "Principal": {"Federated": PROV}, "Action": "sts:AssumeRoleWithWebIdentity", "Condition": {"StringLike": {ISS + ":sub": "system:serviceaccount:*:reports"}, "StringEquals": {ISS + ":aud": "sts.amazonaws.com"}}}]} no_sub = {"Version": "2012-10-17", "Statement": [{"Effect": "Allow", "Principal": {"Federated": PROV}, "Action": "sts:AssumeRoleWithWebIdentity", "Condition": {"StringEquals": {ISS + ":aud": "sts.amazonaws.com"}}}]} print(json.dumps(strict, indent=1)); print() rows = [ ("strict policy, apps/reports", strict, PROV, claims("apps", "reports"), True), ("strict policy, wrong namespace (dev/reports)", strict, PROV, claims("dev", "reports"), False), ("strict policy, wrong service account (apps/other)", strict, PROV, claims("apps", "other"), False), ("strict policy, wrong audience", strict, PROV, claims("apps", "reports", "evil.example"), False), ("strict policy, token from another cluster's provider", strict, OTHER, claims("apps", "reports"), False), ("StringLike with * in the namespace, dev/reports", loose_ns, PROV, claims("dev", "reports"), True), ("StringLike with * in the namespace, kube-system/reports", loose_ns, PROV, claims("kube-system", "reports"), True), ("StringLike with * in the namespace, apps/other", loose_ns, PROV, claims("apps", "other"), False), ("no sub condition at all, any service account", no_sub, PROV, claims("kube-system", "anything"), True), ] bad = 0 for label, pol, prov, cl, want in rows: got = assume_with_web_identity(pol, prov, cl) bad += got != want print("%-4s %-58s -> %s" % ("ok" if got == want else "FAIL", label, "allowed" if got else "denied")) print("expectation mismatches:", bad) --- py trust_eval.py (saved output of the real run) --- { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Federated": "arn:aws:iam::111122223333:oidc-provider/oidc.eks.eu-west-1.amazonaws.com/id/EXAMPLED539D4633E53DE1B71EXAMPLE" }, "Action": "sts:AssumeRoleWithWebIdentity", "Condition": { "StringEquals": { "oidc.eks.eu-west-1.amazonaws.com/id/EXAMPLED539D4633E53DE1B71EXAMPLE:sub": "system:serviceaccount:apps:reports", "oidc.eks.eu-west-1.amazonaws.com/id/EXAMPLED539D4633E53DE1B71EXAMPLE:aud": "sts.amazonaws.com" } } } ] } ok strict policy, apps/reports -> allowed ok strict policy, wrong namespace (dev/reports) -> denied ok strict policy, wrong service account (apps/other) -> denied ok strict policy, wrong audience -> denied ok strict policy, token from another cluster's provider -> denied ok StringLike with * in the namespace, dev/reports -> allowed ok StringLike with * in the namespace, kube-system/reports -> allowed ok StringLike with * in the namespace, apps/other -> denied ok no sub condition at all, any service account -> allowed expectation mismatches: 0 --- k8s/app.yaml --- apiVersion: v1 kind: ServiceAccount metadata: name: reports namespace: apps annotations: # IRSA only (Pod Identity needs no annotation): the role the pods may assume through the cluster's OIDC provider eks.amazonaws.com/role-arn: arn:aws:iam::111122223333:role/reports-irsa --- apiVersion: apps/v1 kind: Deployment metadata: name: reports namespace: apps spec: replicas: 3 selector: matchLabels: {app: reports} strategy: type: RollingUpdate rollingUpdate: {maxUnavailable: 0, maxSurge: 1} template: metadata: labels: {app: reports} spec: serviceAccountName: reports terminationGracePeriodSeconds: 20 topologySpreadConstraints: - maxSkew: 1 topologyKey: topology.kubernetes.io/zone whenUnsatisfiable: ScheduleAnyway labelSelector: matchLabels: {app: reports} containers: - name: web image: 111122223333.dkr.ecr.eu-west-1.amazonaws.com/web:1.0.0 ports: - containerPort: 8080 env: - {name: HANDLE_SIGTERM, value: "1"} readinessProbe: httpGet: {path: /health, port: 8080} periodSeconds: 5 livenessProbe: httpGet: {path: /health, port: 8080} initialDelaySeconds: 10 periodSeconds: 15 resources: requests: {cpu: 100m, memory: 128Mi} limits: {memory: 256Mi} securityContext: runAsNonRoot: true runAsUser: 10001 readOnlyRootFilesystem: true allowPrivilegeEscalation: false --- apiVersion: policy/v1 kind: PodDisruptionBudget metadata: name: reports namespace: apps spec: minAvailable: 2 selector: matchLabels: {app: reports} --- apiVersion: v1 kind: Service metadata: name: reports namespace: apps spec: selector: {app: reports} ports: - port: 80 targetPort: 8080 --- apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: reports namespace: apps annotations: alb.ingress.kubernetes.io/scheme: internet-facing alb.ingress.kubernetes.io/target-type: ip alb.ingress.kubernetes.io/healthcheck-path: /health alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS": 443}]' alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:eu-west-1:111122223333:certificate/00000000-0000-0000-0000-000000000000 spec: ingressClassName: alb rules: - http: paths: - path: / pathType: Prefix backend: service: {name: reports, port: {number: 80}} --- validate_k8s.py (schemas: yannh/kubernetes-json-schema v1.35.0-standalone-strict, downloaded into schemas/) --- """Validate Kubernetes manifests two ways, offline. 1. JSON schema: the yannh/kubernetes-json-schema 'v1.35.0-standalone-strict' files (a third-party project that publishes the schemas generated from Kubernetes), Draft 7. 2. Two checks written from the AWS Load Balancer Controller annotations page (read 2026-10-13): scheme in {internal, internet-facing}, target-type in {instance, ip}, and a consistency check of my own (the Ingress health check path must be a path the container's readiness probe also serves). Not Kubernetes itself: a real API server (kind) was planned but Docker Desktop's engine stopped before it could be used.""" import copy, json, yaml, jsonschema SCH = {"ServiceAccount": "serviceaccount-v1", "Deployment": "deployment-apps-v1", "PodDisruptionBudget": "poddisruptionbudget-policy-v1", "Service": "service-v1", "Ingress": "ingress-networking-v1"} schemas = {k: json.load(open("schemas/%s.json" % v, encoding="utf-8")) for k, v in SCH.items()} docs = [d for d in yaml.safe_load_all(open("k8s/app.yaml", encoding="utf-8")) if d] def schema_errors(ds): out = [] for d in ds: for e in jsonschema.Draft7Validator(schemas[d["kind"]]).iter_errors(d): out.append("%s: %s%s" % (d["kind"], ("/".join(map(str, e.path)) + ": ") if e.path else "", e.message[:90].replace("\n", " "))) return out def lbc_errors(ds): out = [] for d in ds: if d["kind"] != "Ingress": continue a = d["metadata"].get("annotations", {}); P = "alb.ingress.kubernetes.io/" if a.get(P + "scheme", "internal") not in ("internal", "internet-facing"): out.append("scheme " + a[P + "scheme"]) if a.get(P + "target-type", "instance") not in ("instance", "ip"): out.append("target-type " + a[P + "target-type"]) dep = [x for x in ds if x["kind"] == "Deployment"][0] probe = dep["spec"]["template"]["spec"]["containers"][0].get("readinessProbe", {}).get("httpGet", {}).get("path") if probe is None: out.append("no readiness probe found") elif a.get(P + "healthcheck-path", "/") != probe: out.append("ALB health check path %s is not the readiness path %s" % (a.get(P + "healthcheck-path", "/"), probe)) return out def mut(f): ds = copy.deepcopy(docs); f(ds); return ds K = lambda ds, kind: [d for d in ds if d["kind"] == kind][0] cases = [ ("the manifests as written", docs), ("misspelt 'readinessprobe' on the container", mut(lambda ds: K(ds, "Deployment")["spec"]["template"]["spec"]["containers"][0].update(readinessprobe=K(ds, "Deployment")["spec"]["template"]["spec"]["containers"][0].pop("readinessProbe")))), ("replicas: '3' (a string)", mut(lambda ds: K(ds, "Deployment")["spec"].update(replicas="3"))), ("containerPort: 'http'", mut(lambda ds: K(ds, "Deployment")["spec"]["template"]["spec"]["containers"][0]["ports"][0].update(containerPort="http"))), ("Deployment without a selector", mut(lambda ds: K(ds, "Deployment")["spec"].pop("selector"))), ("selector that matches no pod label (app: other)", mut(lambda ds: K(ds, "Deployment")["spec"]["selector"].update(matchLabels={"app": "other"}))), ("PDB minAvailable 5 with 3 replicas", mut(lambda ds: K(ds, "PodDisruptionBudget")["spec"].update(minAvailable=5))), ("PDB with both minAvailable and maxUnavailable", mut(lambda ds: K(ds, "PodDisruptionBudget")["spec"].update(maxUnavailable=1))), ("memory limit below the request (64Mi < 128Mi)", mut(lambda ds: K(ds, "Deployment")["spec"]["template"]["spec"]["containers"][0]["resources"]["limits"].update(memory="64Mi"))), ("Ingress scheme 'public'", mut(lambda ds: K(ds, "Ingress")["metadata"]["annotations"].update({"alb.ingress.kubernetes.io/scheme": "public"}))), ("Ingress target-type 'pod'", mut(lambda ds: K(ds, "Ingress")["metadata"]["annotations"].update({"alb.ingress.kubernetes.io/target-type": "pod"}))), ("ALB health check path '/healthz' (readiness serves /health)", mut(lambda ds: K(ds, "Ingress")["metadata"]["annotations"].update({"alb.ingress.kubernetes.io/healthcheck-path": "/healthz"}))), ("misspelt annotation key 'alb.ingress.kubernetes.io/schema'", mut(lambda ds: K(ds, "Ingress")["metadata"]["annotations"].update({"alb.ingress.kubernetes.io/schema": "internet-facing"}))), ] print("%-62s %-34s %s" % ("change", "JSON schema (v1.35.0 strict)", "annotation/consistency checks")) for label, ds in cases: s, l = schema_errors(ds), lbc_errors(ds) print("%-62s %-34s %s" % (label, ("%d error: %s" % (len(s), s[0][:60])) if s else "valid", ("; ".join(l))[:80] or "-")) --- python validate_k8s.py (saved output of the real run) --- change JSON schema (v1.35.0 strict) annotation/consistency checks the manifests as written valid - misspelt 'readinessprobe' on the container 1 error: Deployment: spec/template/spec/containers/0: Additional prop no readiness probe found replicas: '3' (a string) 1 error: Deployment: spec/replicas: '3' is not of type 'integer', 'nu - containerPort: 'http' 1 error: Deployment: spec/template/spec/containers/0/ports/0/containe - Deployment without a selector 1 error: Deployment: spec: 'selector' is a required property - selector that matches no pod label (app: other) valid - PDB minAvailable 5 with 3 replicas valid - PDB with both minAvailable and maxUnavailable valid - memory limit below the request (64Mi < 128Mi) valid - Ingress scheme 'public' valid scheme public Ingress target-type 'pod' valid target-type pod ALB health check path '/healthz' (readiness serves /health) valid ALB health check path /healthz is not the readiness path /health misspelt annotation key 'alb.ingress.kubernetes.io/schema' valid - WHY THIS WORKS AS AN ANSWER --------------------------- The pod limit uses the documented formula, (ENIs x (IPs per ENI - 1)) + 2, with data from two files in the VPC CNI repository; they agree for all 1,392 instance types (t3.medium 17, m5.large 29), which proves consistency between the two files and not that they match AWS today. The trust evaluator reuses the chapter 2 condition model and shows why the sub condition matters: the strict policy allowed one service account and refused a wrong namespace, name, audience and cluster; a * namespace allowed kube-system's service account of that name; no sub condition allowed every service account. The manifest checks are two layers (Kubernetes schemas and checks from the Load Balancer Controller's annotation page) and together caught 7 of 12 changes; the misses (a selector that matches nothing, a PDB for more pods than exist, a PDB with both fields, a limit below the request, a misspelt annotation key) are the cases where only a real API server, or the controller, knows. Not run: a Kubernetes API server (kind cluster v1.35.0 was created but Docker Desktop's engine stopped before experiments); what an API server rejects is my expectation, not a result. The IRSA trust policy JSON shape is the commonly documented web-identity format; the EKS page read did not print it. The annotation checks cover only the values the annotations page listed.