aws2-10 Exercise 1: CloudFront Cache Models and the Origin Access Control Policy ======================================================================== Run with Windows 11, Python 3.14 (py launcher) and a venv with moto 5.2.4 (server mode on port 5000, dummy credentials test/test), boto3 1.43.112, cfn-lint 1.57.2, PyYAML, dnspython 2.9.0; iam_eval.py is the chapter 2 model copied with one fix (see exercise 1). Scratch folder C:/lwaws/c10, nothing installed globally, no Docker. VERIFICATION LEVEL: 1 (no AWS account, no domain, nothing deployed); statements about the real services come from the AWS documentation read on 2026-10-13 and are marked as such in the chapter. Outputs below are saved from the real runs of 2026-10-13. --- cachettl.py --- """A model of how CloudFront decides cache duration, written from the pages read on 2026-10-13: 'Manage how long content stays in the cache (expiration)' - the Cache-Control / Expires table, stale-while-revalidate, stale-if-error 'How CloudFront processes HTTP 4xx and 5xx status codes from your origin' - error caching minimum TTL (10 s default) and cached codes 'Use managed cache policies' - the TTLs and cache keys of the managed policies 'Pay for file invalidation' - 1,000 free paths per month across the account; a wildcard path counts as one It models only what those pages state; where a page is silent the function says so instead of guessing.""" MANAGED = { # name: (min TTL, default TTL, max TTL, headers in key, cookies, query strings, compress) "CachingOptimized": (1, 86400, 31536000, [], "none", "none", True), "CachingOptimizedForUncompressedObjects": (1, 86400, 31536000, [], "none", "none", False), "CachingDisabled": (0, 0, 0, [], "none", "none", False), "UseOriginCacheControlHeaders": (0, 0, 31536000, ["Host", "Origin", "X-HTTP-Method-Override", "X-HTTP-Method", "X-Method-Override"], "all", "none", True), "UseOriginCacheControlHeaders-QueryStrings": (0, 0, 31536000, ["Host", "Origin", "X-HTTP-Method-Override", "X-HTTP-Method", "X-Method-Override"], "all", "all", True), "Amplify": (2, 2, 600, ["Authorization", "CloudFront-Viewer-Country", "Host"], "all", "all", True), "Elemental-MediaPackage": (0, 86400, 31536000, ["Origin"], "none", "allowlist(aws.manifestfilter,start,end,m)", True), } def clamp(v, lo, hi): return max(lo, min(v, hi)) def edge_ttl(minttl, default, maxttl, cc=None, expires_in=None): """seconds CloudFront caches an object. cc: set of Cache-Control directives as {'max-age': 60, 's-maxage': 30, 'no-cache': True, ...}; expires_in: seconds until the Expires header's date (only used when there is no max-age, per the page).""" cc = cc or {} blockers = any(cc.get(k) for k in ("no-cache", "no-store", "private")) if blockers: # table row: Min TTL = 0 -> CloudFront respects the headers (not cached); Min TTL > 0 -> cached for the minimum TTL return minttl if minttl > 0 else 0 if "s-maxage" in cc: v = cc["s-maxage"] # table: with both max-age and s-maxage the s-maxage value is used for CloudFront return min(v, maxttl) if minttl == 0 else clamp(v, minttl, maxttl) if "max-age" in cc: v = cc["max-age"] return min(v, maxttl) if minttl == 0 else clamp(v, minttl, maxttl) if expires_in is not None: return min(expires_in, maxttl) if minttl == 0 else clamp(expires_in, minttl, maxttl) return default if minttl == 0 else max(minttl, default) def stale_window(directive_seconds, maxttl): """page: stale content is served 'up to the value of the directive or the maximum TTL, whichever is less'. The page does not say what that period is measured from, so only the length is modelled.""" return min(directive_seconds, maxttl) ALWAYS_CACHED = {404, 414, 500, 501, 502, 503, 504} CACHED_WITH_HEADER = {400, 403, 405, 412, 415} def error_ttl(status, error_min_ttl=10, cc=None, method="GET"): """seconds an error from the origin is cached, or 0. Page: cached for the maximum of the error caching minimum TTL (10 s default) and the origin's max-age / s-maxage; 400/403/405/412/415 only when the origin sends max-age or s-maxage; GET and HEAD only.""" cc = cc or {} if method not in ("GET", "HEAD"): return 0 hdr = max(cc.get("max-age", 0), cc.get("s-maxage", 0)) if ("max-age" in cc or "s-maxage" in cc) else None if status in ALWAYS_CACHED: return max(error_min_ttl, hdr or 0) if status in CACHED_WITH_HEADER: return max(error_min_ttl, hdr) if hdr is not None else 0 return 0 def invalidation_billable(paths_per_distribution_by_month): """page: the first 1,000 paths a month are free across the whole account; a path with * counts as one; tag paths count too.""" total = sum(paths_per_distribution_by_month) return max(0, total - 1000) --- cache_tests.py --- """Tests for cachettl.py: every cell of the docs' expiration table, the docs' stale and error examples, managed policies against origin headers, the invalidation examples, and a cache-key hit-ratio experiment on a synthetic request log.""" import random from collections import Counter import cachettl as C fails = 0 def check(label, got, want, note=""): global fails ok = got == want fails += not ok print(f"{'ok ' if ok else 'FAIL'} {label:70s} model {got!s:>10} docs {want!s:>10} {note}") print("== the expiration table (min 0 or >0; origin headers) ==") # min TTL 0, default 100, max 1000 check("min 0, max-age 500", C.edge_ttl(0, 100, 1000, {"max-age": 500}), 500, "lesser of max-age and maximum TTL") check("min 0, max-age 5000 (above max)", C.edge_ttl(0, 100, 1000, {"max-age": 5000}), 1000, "capped by maximum TTL") check("min 0, no header", C.edge_ttl(0, 100, 1000), 100, "default TTL") check("min 0, max-age 500 and s-maxage 200", C.edge_ttl(0, 100, 1000, {"max-age": 500, "s-maxage": 200}), 200, "s-maxage for CloudFront") check("min 0, s-maxage 5000", C.edge_ttl(0, 100, 1000, {"max-age": 500, "s-maxage": 5000}), 1000, "capped") check("min 0, Expires in 300 s", C.edge_ttl(0, 100, 1000, expires_in=300), 300, "until Expires or max TTL, sooner") check("min 0, Expires in 9000 s", C.edge_ttl(0, 100, 1000, expires_in=9000), 1000, "capped") check("min 0, no-store", C.edge_ttl(0, 100, 1000, {"no-store": True}), 0, "headers respected") check("min 0, private", C.edge_ttl(0, 100, 1000, {"private": True}), 0, "headers respected") check("min 0, no-cache with max-age 500", C.edge_ttl(0, 100, 1000, {"no-cache": True, "max-age": 500}), 0, "headers respected") # min TTL 50 check("min 50, max-age 500", C.edge_ttl(50, 100, 1000, {"max-age": 500}), 500, "min < max-age < max") check("min 50, max-age 10", C.edge_ttl(50, 100, 1000, {"max-age": 10}), 50, "max-age < min -> min") check("min 50, max-age 5000", C.edge_ttl(50, 100, 1000, {"max-age": 5000}), 1000, "max-age > max -> max") check("min 50, no header, default 100", C.edge_ttl(50, 100, 1000), 100, "greater of min and default") check("min 500, no header, default 100", C.edge_ttl(500, 100, 1000), 500, "greater of min and default") check("min 50, s-maxage 10 (max-age 500)", C.edge_ttl(50, 100, 1000, {"max-age": 500, "s-maxage": 10}), 50, "s-maxage < min -> min") check("min 50, Expires in 10 s", C.edge_ttl(50, 100, 1000, expires_in=10), 50, "Expires < min -> min") check("min 50, Expires in 9000 s", C.edge_ttl(50, 100, 1000, expires_in=9000), 1000, "Expires > max -> max") check("min 50, no-store", C.edge_ttl(50, 100, 1000, {"no-store": True}), 50, "the page's warning: cached for the minimum TTL anyway") print("\n== stale directives and errors (page examples) ==") check("stale-while-revalidate=600, max TTL 31,536,000", C.stale_window(600, 31536000), 600, "example: 10 minutes") check("stale-if-error=86400, max TTL 3600", C.stale_window(86400, 3600), 3600, "'whichever is less'") check("404, no headers", C.error_ttl(404), 10, "error caching minimum TTL, 10 s") check("404 with max-age=60", C.error_ttl(404, cc={"max-age": 60}), 60, "maximum of the two") check("503, error min TTL 30, max-age 5", C.error_ttl(503, 30, {"max-age": 5}), 30, "maximum of the two") check("403 without a header", C.error_ttl(403), 0, "only cached with max-age / s-maxage") check("403 with s-maxage=20", C.error_ttl(403, cc={"s-maxage": 20}), 20, "cached with the header") check("500 for a POST", C.error_ttl(500, method="POST"), 0, "GET and HEAD only") check("418 (not in either list)", C.error_ttl(418), 0, "not listed") print("\n== invalidation (page examples) ==") check("3 distributions x 600 paths", C.invalidation_billable([600, 600, 600]), 800, "'charge for 800 invalidation paths'") check("500 path + 600 tag invalidations", C.invalidation_billable([500, 600]), 100, "'charged for 100'") check("a single /* wildcard path", C.invalidation_billable([1]), 0, "counts as one path") print("\n== managed cache policies against origin headers (seconds CloudFront keeps an object) ==") SCEN = [("no header", None), ("max-age=60", {"max-age": 60}), ("no-store", {"no-store": True}), ("private", {"private": True}), ("max-age=3 years", {"max-age": 94608000})] print(f"{'policy':44s}" + "".join(f"{n:>18s}" for n, _ in SCEN)) for name, (mn, df, mx, *_rest) in C.MANAGED.items(): print(f"{name:44s}" + "".join(f"{C.edge_ttl(mn, df, mx, cc):>18,d}" for _, cc in SCEN)) print("\n== cache key: upper bound of the hit ratio on a synthetic request log ==") random.seed(10) paths = [f"/p/{i}" for i in range(30)] weights = [1 / (i + 1) for i in range(30)] users = [f"u{i}" for i in range(200)] log = [] for _ in range(20000): q = {} if random.random() < 0.6: q["utm_source"] = random.choice(["mail", "tw", "fb", "ads", "blog"]) if random.random() < 0.2: q["v"] = random.choice(["1", "2"]) log.append({"path": random.choices(paths, weights)[0], "query": q, "cookie": random.choice(users), "ae": random.choices(["gzip", "br", "identity"], [0.7, 0.2, 0.1])[0], "country": random.choice(["GB", "DE", "FR", "US", "JP"]), "host": "www.example.com", "auth": random.choice(["", "tok"])}) def key(req, policy): mn, df, mx, headers, cookies, qs, compress = C.MANAGED[policy] if isinstance(policy, str) else policy k = [req["path"]] if qs == "all": k.append(tuple(sorted(req["query"].items()))) elif qs.startswith("allowlist"): allowed = set(qs[qs.index("(") + 1:-1].split(",")) k.append(tuple(sorted((a, b) for a, b in req["query"].items() if a in allowed))) if cookies == "all": k.append(req["cookie"]) for h in headers: k.append({"Host": req["host"], "CloudFront-Viewer-Country": req["country"], "Authorization": req["auth"]}.get(h, "")) if compress: k.append(req["ae"] if req["ae"] in ("gzip", "br") else "none") # normalised Accept-Encoding return tuple(k) CUSTOM = (1, 86400, 31536000, [], "none", "allowlist(v)", True) rows = [("CachingOptimized", "CachingOptimized"), ("custom: only the 'v' query string", CUSTOM), ("UseOriginCacheControlHeaders", "UseOriginCacheControlHeaders"), ("UseOriginCacheControlHeaders-QueryStrings", "UseOriginCacheControlHeaders-QueryStrings"), ("Amplify", "Amplify")] print(f"{'policy':46s} {'distinct keys':>14s} {'hit-ratio ceiling':>18s}") for label, pol in rows: keys = Counter(key(r, pol) for r in log) print(f"{label:46s} {len(keys):14,d} {1 - len(keys) / len(log):17.1%}") print(f"\n{20000:,} requests, {len(paths)} paths; 'ceiling' = 1 - distinct keys / requests, i.e. every repeat of a key is a hit (no expiry, no eviction)") print("\nfailed checks:", fails) --- py cache_tests.py (saved output of the real run) --- == the expiration table (min 0 or >0; origin headers) == ok min 0, max-age 500 model 500 docs 500 lesser of max-age and maximum TTL ok min 0, max-age 5000 (above max) model 1000 docs 1000 capped by maximum TTL ok min 0, no header model 100 docs 100 default TTL ok min 0, max-age 500 and s-maxage 200 model 200 docs 200 s-maxage for CloudFront ok min 0, s-maxage 5000 model 1000 docs 1000 capped ok min 0, Expires in 300 s model 300 docs 300 until Expires or max TTL, sooner ok min 0, Expires in 9000 s model 1000 docs 1000 capped ok min 0, no-store model 0 docs 0 headers respected ok min 0, private model 0 docs 0 headers respected ok min 0, no-cache with max-age 500 model 0 docs 0 headers respected ok min 50, max-age 500 model 500 docs 500 min < max-age < max ok min 50, max-age 10 model 50 docs 50 max-age < min -> min ok min 50, max-age 5000 model 1000 docs 1000 max-age > max -> max ok min 50, no header, default 100 model 100 docs 100 greater of min and default ok min 500, no header, default 100 model 500 docs 500 greater of min and default ok min 50, s-maxage 10 (max-age 500) model 50 docs 50 s-maxage < min -> min ok min 50, Expires in 10 s model 50 docs 50 Expires < min -> min ok min 50, Expires in 9000 s model 1000 docs 1000 Expires > max -> max ok min 50, no-store model 50 docs 50 the page's warning: cached for the minimum TTL anyway == stale directives and errors (page examples) == ok stale-while-revalidate=600, max TTL 31,536,000 model 600 docs 600 example: 10 minutes ok stale-if-error=86400, max TTL 3600 model 3600 docs 3600 'whichever is less' ok 404, no headers model 10 docs 10 error caching minimum TTL, 10 s ok 404 with max-age=60 model 60 docs 60 maximum of the two ok 503, error min TTL 30, max-age 5 model 30 docs 30 maximum of the two ok 403 without a header model 0 docs 0 only cached with max-age / s-maxage ok 403 with s-maxage=20 model 20 docs 20 cached with the header ok 500 for a POST model 0 docs 0 GET and HEAD only ok 418 (not in either list) model 0 docs 0 not listed == invalidation (page examples) == ok 3 distributions x 600 paths model 800 docs 800 'charge for 800 invalidation paths' ok 500 path + 600 tag invalidations model 100 docs 100 'charged for 100' ok a single /* wildcard path model 0 docs 0 counts as one path == managed cache policies against origin headers (seconds CloudFront keeps an object) == policy no header max-age=60 no-store private max-age=3 years CachingOptimized 86,400 60 1 1 31,536,000 CachingOptimizedForUncompressedObjects 86,400 60 1 1 31,536,000 CachingDisabled 0 0 0 0 0 UseOriginCacheControlHeaders 0 60 0 0 31,536,000 UseOriginCacheControlHeaders-QueryStrings 0 60 0 0 31,536,000 Amplify 2 60 2 2 600 Elemental-MediaPackage 86,400 60 0 0 31,536,000 == cache key: upper bound of the hit ratio on a synthetic request log == policy distinct keys hit-ratio ceiling CachingOptimized 90 99.6% custom: only the 'v' query string 269 98.7% UseOriginCacheControlHeaders 7,786 61.1% UseOriginCacheControlHeaders-QueryStrings 15,458 22.7% Amplify 19,240 3.8% 20,000 requests, 30 paths; 'ceiling' = 1 - distinct keys / requests, i.e. every repeat of a key is a hit (no expiry, no eviction) failed checks: 0 --- oac_policy.py --- """S3 origin access control: the bucket policy statement printed on the CloudFront 'Restrict access to an Amazon S3 origin' page (read 2026-10-13), evaluated with the chapter 2 model of IAM evaluation (iam_eval.py, a MODEL of documented rules, not AWS). Each case says what the page implies, then what the model decides.""" import copy, json, shutil, sys sys.path.insert(0, "C:/lwaws/c10") import iam_eval as E BUCKET = "arn:aws:s3:::amzn-s3-demo-bucket" DIST = "arn:aws:cloudfront::111122223333:distribution/E1EXAMPLE" OTHER = "arn:aws:cloudfront::999988887777:distribution/E2OTHER" OAI = "arn:aws:iam::cloudfront:user/CloudFront Origin Access Identity E3LEGACY" READ_ONLY = {"Version": "2012-10-17", "Statement": [{ "Sid": "AllowCloudFrontServicePrincipalReadOnly", "Effect": "Allow", "Principal": {"Service": "cloudfront.amazonaws.com"}, "Action": "s3:GetObject", "Resource": BUCKET + "/*", "Condition": {"StringEquals": {"AWS:SourceArn": DIST}}}]} NO_CONDITION = copy.deepcopy(READ_ONLY); del NO_CONDITION["Statement"][0]["Condition"] BAD_RESOURCE = copy.deepcopy(READ_ONLY); BAD_RESOURCE["Statement"][0]["Resource"] = BUCKET MIGRATION = copy.deepcopy(READ_ONLY) MIGRATION["Statement"].append({"Sid": "AllowLegacyOAIReadOnly", "Effect": "Allow", "Principal": {"AWS": OAI}, "Action": "s3:GetObject", "Resource": BUCKET + "/*"}) KEY_POLICY = {"Version": "2012-10-17", "Statement": [{ "Sid": "AllowCloudFrontServicePrincipalSSE-KMS", "Effect": "Allow", "Principal": {"Service": ["cloudfront.amazonaws.com"]}, "Action": ["kms:Decrypt", "kms:Encrypt", "kms:GenerateDataKey*"], "Resource": "*", "Condition": {"StringEquals": {"AWS:SourceArn": DIST}}}]} CF = {"type": "service", "name": "cloudfront.amazonaws.com", "account": "111122223333"} OAI_USER = {"type": "user", "arn": OAI, "account": "cloudfront"} STRANGER = {"type": "user", "arn": "arn:aws:iam::444455556666:user/someone", "account": "444455556666"} def ask(policy, who, action, res, src=None, kms=False): ctx = {"aws:SourceArn": src} if src else {} req = {"who": who, "action": action, "resource": res, "ctx": ctx} try: ok, why = E.decide(req, {"resource": [policy]}, explicit_resource_allow=kms) except E.NotModelled as e: return None, "not modelled: " + str(e) return ok, why CASES = [ ("docs read-only policy: CloudFront, its own distribution, GetObject", READ_ONLY, CF, "s3:GetObject", BUCKET + "/photo.jpg", DIST, False, True), ("same policy: CloudFront on behalf of ANOTHER distribution", READ_ONLY, CF, "s3:GetObject", BUCKET + "/photo.jpg", OTHER, False, False), ("same policy: CloudFront request with no SourceArn", READ_ONLY, CF, "s3:GetObject", BUCKET + "/photo.jpg", None, False, False), ("same policy: PutObject (read-only statement)", READ_ONLY, CF, "s3:PutObject", BUCKET + "/photo.jpg", DIST, False, False), ("same policy: an unrelated user in another account", READ_ONLY, STRANGER, "s3:GetObject", BUCKET + "/photo.jpg", None, False, False), ("mistake: the Condition left out, another distribution", NO_CONDITION, CF, "s3:GetObject", BUCKET + "/photo.jpg", OTHER, False, True), ("mistake: Resource is the bucket, not bucket/*", BAD_RESOURCE, CF, "s3:GetObject", BUCKET + "/photo.jpg", DIST, False, False), ("migration policy (OAC + legacy OAI): the OAI user", MIGRATION, OAI_USER, "s3:GetObject", BUCKET + "/photo.jpg", None, False, True), ("migration policy: CloudFront with OAC", MIGRATION, CF, "s3:GetObject", BUCKET + "/photo.jpg", DIST, False, True), ("KMS key policy from the page: Decrypt for the distribution", KEY_POLICY, CF, "kms:Decrypt", "arn:aws:kms:eu-west-1:111122223333:key/abc", DIST, True, True), ("KMS key policy: Decrypt for another distribution", KEY_POLICY, CF, "kms:Decrypt", "arn:aws:kms:eu-west-1:111122223333:key/abc", OTHER, True, False), ("KMS key policy: ScheduleKeyDeletion (not listed)", KEY_POLICY, CF, "kms:ScheduleKeyDeletion", "arn:aws:kms:eu-west-1:111122223333:key/abc", DIST, True, False), ] print(f"{'case':66s} {'expect':7s} {'model'}") bad = 0 for label, pol, who, act, res, src, kms, want in CASES: ok, why = ask(pol, who, act, res, src, kms) bad += ok != want print(f"{label:66s} {str(want):7s} {str(ok):6s} {why}") print(f"\n{len(CASES) - bad} of {len(CASES)} cases as expected from the page") --- the one change to the chapter 2 model (iam_eval.py, grant_kind) --- arn = who.get("arn") # chapter 10 fix: a service requester has no ARN, so it cannot match an AWS principal if arn is None: continue --- py oac_policy.py (saved output of the real run) --- case expect model docs read-only policy: CloudFront, its own distribution, GetObject True True resource-based policy grants the service directly same policy: CloudFront on behalf of ANOTHER distribution False False no identity-based Allow (implicit deny) same policy: CloudFront request with no SourceArn False False no identity-based Allow (implicit deny) same policy: PutObject (read-only statement) False False no identity-based Allow (implicit deny) same policy: an unrelated user in another account False False no identity-based Allow (implicit deny) mistake: the Condition left out, another distribution True True resource-based policy grants the service directly mistake: Resource is the bucket, not bucket/* False False no identity-based Allow (implicit deny) migration policy (OAC + legacy OAI): the OAI user True True resource-based policy grants the user directly migration policy: CloudFront with OAC True True resource-based policy grants the service directly KMS key policy from the page: Decrypt for the distribution True True resource-based policy grants the service directly KMS key policy: Decrypt for another distribution False False this resource policy must explicitly allow the principal (it does not) KMS key policy: ScheduleKeyDeletion (not listed) False False this resource policy must explicitly allow the principal (it does not) 12 of 12 cases as expected from the page -------------------------- Reading the result: every cell of the documented expiration table, the stale, error-caching and invalidation rules, and 12 origin-access-control policy cases agree with the pages. These tests compare my model with my own reading of the same pages, so they show faithfulness to the page, not behaviour of the service. The hit-ratio ceilings (99.6% for CachingOptimized, 98.7% for a custom 'v'-only key, 61.1%, 22.7%, 3.8%) come from a synthetic log I generated, not from real traffic. Mistake made and fixed: the chapter 2 IAM model crashed with KeyError('arn') when a SERVICE requester (no ARN) was compared with a statement that names an AWS principal (the migration policy has both an OAC and an OAI statement). It should have answered 'no match'. I fixed it in this chapter's copy (arn = who.get('arn'); skip when None). How to avoid it: when you extend a model's test set with a new kind of requester, run it against every statement shape, not only the one it was built for.