aws2-12 Exercise 3: Regression Edits, an Allowlist Lesson and a Cost Sketch ======================================================================== Run with Windows 11, Node 24.18.0, aws-cdk 2.1145.0, aws-cdk-lib 2.273.0, constructs 10.8.1, TypeScript 7.0.2, Jest 30.5.2 (node_modules is a directory junction to chapter 5's), Python 3.14 (py launcher) with cfn-lint 1.57.2, PyYAML, moto 5.2.4 (in-process mock_aws), boto3. Scratch folder C:/lwaws/c12; nothing installed globally; Docker not used. Build with: node node_modules/typescript/bin/tsc -p tsconfig.json Synthesise with: AWS_EC2_METADATA_DISABLED=true node node_modules/aws-cdk/bin/cdk synth -c hardened=true -o cdk.out.hard -q (and hardened=false -o cdk.out.draft). VERIFICATION LEVEL: 1 (no AWS account; nothing deployed); statements about the Well-Architected Framework and prices come from pages read on 2026-10-13 and are marked as such in the chapter. Outputs below are saved from the real runs of 2026-10-13. --- mutate_wa.py --- """Regression test for the review: change ONE thing in the hardened template (as if someone edited the stack) and ask (a) cfn-lint 1.57.2 (anything beyond the baseline W3005 warnings) and (b) wa_review.py (a check that passed now FAILs).""" import copy, json, os, re, subprocess, tempfile import wa_review as W B = "C:/lwaws/c12" base = json.load(open(f"{B}/cdk.out.hard/Orders.template.json", encoding="utf-8")) base_fail = {i for i, p, s, x in W.check(base) if s == "FAIL"} def rs(t, typ, name=None): return [(k, v) for k, v in t["Resources"].items() if v["Type"] == typ and (name is None or name in k)] def one(t, typ, name=None): r = rs(t, typ, name); assert len(r) == 1, (typ, name, [k for k, _ in r]); return r[0] def work_queue(t): return one(t, "AWS::SQS::Queue", "OrdersQueue")[1] def M_drop(t, typ, name, *path): p = one(t, typ, name)[1]["Properties"] for k in path[:-1]: p = p[k] del p[path[-1]] def M_set(t, typ, name, path, val): p = one(t, typ, name)[1]["Properties"] for k in path[:-1]: p = p[k] p[path[-1]] = val MUT = [ ("work queue visibility timeout 185 -> 30", lambda t: M_set(t, "AWS::SQS::Queue", "OrdersQueue", ["VisibilityTimeout"], 30)), ("work queue loses its redrive policy", lambda t: M_drop(t, "AWS::SQS::Queue", "OrdersQueue", "RedrivePolicy")), ("worker timeout 30 -> 300 (queue timeout now too short)", lambda t: M_set(t, "AWS::Lambda::Function", "WorkerFn", ["Timeout"], 300)), ("table point-in-time recovery off", lambda t: M_set(t, "AWS::DynamoDB::Table", "Orders", ["PointInTimeRecoverySpecification", "PointInTimeRecoveryEnabled"], False)), ("table deletion protection off", lambda t: M_set(t, "AWS::DynamoDB::Table", "Orders", ["DeletionProtectionEnabled"], False)), ("table encryption removed", lambda t: M_drop(t, "AWS::DynamoDB::Table", "Orders", "SSESpecification")), ("table DeletionPolicy Retain -> Delete", lambda t: one(t, "AWS::DynamoDB::Table", "Orders")[1].__setitem__("DeletionPolicy", "Delete")), ("site bucket BlockPublicPolicy false", lambda t: M_set(t, "AWS::S3::Bucket", "Site", ["PublicAccessBlockConfiguration", "BlockPublicPolicy"], False)), ("KMS key rotation off", lambda t: M_set(t, "AWS::KMS::Key", "DataKey", ["EnableKeyRotation"], False)), ("CloudFront default behaviour allow-all", lambda t: M_set(t, "AWS::CloudFront::Distribution", "Cdn", ["DistributionConfig", "DefaultCacheBehavior", "ViewerProtocolPolicy"], "allow-all")), ("CloudFront price class all", lambda t: M_set(t, "AWS::CloudFront::Distribution", "Cdn", ["DistributionConfig", "PriceClass"], "PriceClass_All")), ("DLQ alarm loses its action", lambda t: M_drop(t, "AWS::CloudWatch::Alarm", "DlqNotEmpty", "AlarmActions")), ("DLQ alarm removed", lambda t: t["Resources"].pop(one(t, "AWS::CloudWatch::Alarm", "DlqNotEmpty")[0])), ("consumer loses partial-batch reporting", lambda t: M_drop(t, "AWS::Lambda::EventSourceMapping", None, "FunctionResponseTypes")), ("consumer concurrency cap removed", lambda t: M_drop(t, "AWS::Lambda::EventSourceMapping", None, "ScalingConfig")), ("worker on x86_64", lambda t: M_set(t, "AWS::Lambda::Function", "WorkerFn", ["Architectures"], ["x86_64"])), ("API function tracing PassThrough", lambda t: M_set(t, "AWS::Lambda::Function", "ApiFn", ["TracingConfig", "Mode"], "PassThrough")), ("API stage throttle removed", lambda t: M_drop(t, "AWS::ApiGatewayV2::Stage", "Stage", "DefaultRouteSettings")), ("queue policy's TLS deny removed", lambda t: one(t, "AWS::SQS::QueuePolicy", "OrdersQueue")[1]["Properties"]["PolicyDocument"]["Statement"].clear()), ("an IAM user and access key added", lambda t: (t["Resources"].__setitem__("U", {"Type": "AWS::IAM::User", "Properties": {}}), t["Resources"].__setitem__("K", {"Type": "AWS::IAM::AccessKey", "Properties": {"UserName": {"Ref": "U"}}}))), ("a statement with Action * on Resource * added", lambda t: rs(t, "AWS::IAM::Policy", "ApiFn")[0][1]["Properties"]["PolicyDocument"]["Statement"].append({"Effect": "Allow", "Action": "*", "Resource": "*"})), ("API log group retention removed", lambda t: M_drop(t, "AWS::Logs::LogGroup", "ApiLogs", "RetentionInDays")), ("table tags removed", lambda t: M_drop(t, "AWS::DynamoDB::Table", "Orders", "Tags")), ("table billing PROVISIONED without capacity", lambda t: M_set(t, "AWS::DynamoDB::Table", "Orders", ["BillingMode"], "PROVISIONED")), ("worker memory 256 -> 64", lambda t: M_set(t, "AWS::Lambda::Function", "WorkerFn", ["MemorySize"], 64)), ("worker timeout 900 -> 901", lambda t: M_set(t, "AWS::Lambda::Function", "WorkerFn", ["Timeout"], 901)), ("work queue retention 14 days -> 0 seconds", lambda t: M_set(t, "AWS::SQS::Queue", "OrdersQueue", ["MessageRetentionPeriod"], 0)), ("API route method POST -> POSTT", lambda t: one(t, "AWS::ApiGatewayV2::Route")[1]["Properties"].__setitem__("RouteKey", "POSTT /api/orders")), ("event source batch size 10 -> 11", lambda t: M_set(t, "AWS::Lambda::EventSourceMapping", None, ["BatchSize"], 11)), ] def lint(t): f = tempfile.NamedTemporaryFile("w", suffix=".json", delete=False, encoding="utf-8", dir=B) json.dump(t, f); f.close() p = subprocess.run([f"{B}/venv/Scripts/cfn-lint.exe", f.name], capture_output=True, text=True) os.unlink(f.name) codes = sorted(set(re.findall(r"^([EW]\d{4})", p.stdout, re.M)) - {"W3005"}) return bool(codes), codes print(f"{'change':62s} lint review") print(f"{'(control: unchanged hardened template)':62s} {int(lint(base)[0]):4d} {int(bool({i for i,p,s,x in W.check(base) if s=='FAIL'} - base_fail)):6d} baseline FAILs: {sorted(base_fail)}") n = a = b = both = none = 0 for label, fn in MUT: t = copy.deepcopy(base); fn(t) li, codes = lint(t) try: new = sorted({i for i, p, s, x in W.check(t) if s == "FAIL"} - base_fail); rv = bool(new) except Exception as e: new, rv = ["crashed:" + type(e).__name__], False # a crash is not a catch print(f"{label[:62]:62s} {int(li):4d} {int(rv):6d} {','.join(codes) if li else '':10s} {','.join(new)}") n += 1; a += li; b += rv; both += (li and rv); none += (not li and not rv) print(f"\n{n} changes: cfn-lint {a}, review {b}, both {both}, neither {none}") --- py mutate_wa.py (saved output of the real run) --- change lint review (control: unchanged hardened template) 0 0 baseline FAILs: ['OPS7', 'SEC10', 'SEC4', 'SEC6'] work queue visibility timeout 185 -> 30 0 1 REL2 work queue loses its redrive policy 0 1 REL1 worker timeout 30 -> 300 (queue timeout now too short) 1 1 E3505 REL2 table point-in-time recovery off 0 1 REL4 table deletion protection off 0 1 REL5 table encryption removed 0 1 SEC1 table DeletionPolicy Retain -> Delete 0 1 REL6 site bucket BlockPublicPolicy false 0 1 SEC8 KMS key rotation off 0 1 SEC9 CloudFront default behaviour allow-all 0 1 SEC7 CloudFront price class all 0 1 COST2 DLQ alarm loses its action 0 1 OPS6 DLQ alarm removed 0 1 OPS2 consumer loses partial-batch reporting 0 1 REL3 consumer concurrency cap removed 0 1 REL8 worker on x86_64 0 1 PERF1,SUS1 API function tracing PassThrough 0 1 SEC3 API stage throttle removed 0 1 REL7 queue policy's TLS deny removed 0 1 SEC2 an IAM user and access key added 0 1 SEC5 a statement with Action * on Resource * added 0 0 API log group retention removed 0 1 COST4,OPS1,SUS3 table tags removed 0 1 COST3 table billing PROVISIONED without capacity 1 1 E3639 COST1 worker memory 256 -> 64 1 0 E3034 worker timeout 900 -> 901 1 1 E3505,E3717 REL2 work queue retention 14 days -> 0 seconds 1 0 E3034 API route method POST -> POSTT 0 0 event source batch size 10 -> 11 0 0 29 changes: cfn-lint 5, review 24, both 3, neither 3 --- cost12.py --- """Monthly cost sketch for the capstone stack, from pricing pages fetched on 2026-10-13 (fetch tool; rates mostly from worked examples; US East (N. Virginia) where a Region is named, otherwise not stated). THE AWS PRICING CALCULATOR WAS NOT USED: it is an interactive web tool and I cannot drive it from here. READ: API Gateway HTTP API $1.00 per million (first 300M); Lambda $0.20 per million requests, $0.0000166667 per GB-s x86, $0.0000133334 per GB-s Arm (Arm figure from one example), free tier 1M requests and 400,000 GB-s; DynamoDB on demand $0.625 per million write request units, $0.125 per million read, storage $0.25 per GB, point-in-time recovery $0.20 per GB-month; CloudWatch alarm $0.10 per standard alarm metric, logs $0.50 per GB ingested, free tier 10 alarm metrics and 5 GB of logs; KMS $1 per key per month and $0.03 per 10,000 requests (20,000 free); Secrets Manager $0.40 per secret per month and $0.05 per 10,000 calls; CloudFront flat-rate plans. NOT READ OR NOT ON THE PAGE TEXT: SQS request price (free tier 1M requests only), CloudFront pay-as-you-go prices, S3, SNS, X-Ray, data transfer, DynamoDB read/write unit sizes beyond 1 KB/4 KB assumptions.""" import math def month(orders, batch=10, api_ms=100, worker_ms=200, mem_gb=0.25, kb=1.0, logs_kb_per_order=2.0, stored_gb=0.5): inv_api = orders inv_wrk = math.ceil(orders / batch) req = (inv_api + inv_wrk) / 1e6 gbs = inv_api * api_ms / 1000 * mem_gb + inv_wrk * worker_ms / 1000 * mem_gb * batch # a batch of 10 takes ~ 10x per-message work lam_req = max(inv_api + inv_wrk - 1e6, 0) / 1e6 * 0.20 lam_dur = max(gbs - 400_000, 0) * 0.0000133334 gbs_nofree = gbs * 0.0000133334 api = orders / 1e6 * 1.00 wru = orders * math.ceil(kb) # 1 WRU per 1 KB written ddb = wru / 1e6 * 0.625 + stored_gb * 0.25 + stored_gb * 0.20 alarms = 5 * 0.10 alarms_billed = max(5 - 10, 0) * 0.10 logs_gb = orders * logs_kb_per_order / 1e6 logs = max(logs_gb - 5, 0) * 0.50 sqs_requests = orders * (1 + 1 / batch + 1 / batch) # send + receive + delete per batch return dict(orders=orders, api=api, lam_req=lam_req, lam_dur=lam_dur, ddb=ddb, alarms=alarms_billed, logs=logs, key=1.00, secret=0.40, lam_req_nofree=(inv_api + inv_wrk) / 1e6 * 0.20, lam_dur_nofree=gbs_nofree, alarms_nofree=alarms, logs_nofree=logs_gb * 0.50, sqs_requests=sqs_requests, gbs=gbs, logs_gb=logs_gb) for orders in (100_000, 1_000_000, 10_000_000): m = month(orders) priced = m["api"] + m["lam_req"] + m["lam_dur"] + m["ddb"] + m["alarms"] + m["logs"] + m["key"] + m["secret"] nofree = m["api"] + m["lam_req_nofree"] + m["lam_dur_nofree"] + m["ddb"] + m["alarms_nofree"] + m["logs_nofree"] + m["key"] + m["secret"] print(f"== {orders:,} orders a month (1 KB item, batch of 10, 100 ms API / 200 ms per message, 256 MB Arm) ==") print(f" API Gateway HTTP API ${m['api']:9.2f}") print(f" Lambda requests ${m['lam_req']:9.2f} (free tier applied; without it ${m['lam_req_nofree']:.2f})") print(f" Lambda duration ${m['lam_dur']:9.2f} ({m['gbs']:,.0f} GB-s; without free tier ${m['lam_dur_nofree']:.2f})") print(f" DynamoDB (writes, 0.5 GB stored, PITR) ${m['ddb']:.2f}") print(f" CloudWatch alarms ${m['alarms']:9.2f} (5 alarms inside the 10-alarm free tier; ${m['alarms_nofree']:.2f} without it)") print(f" CloudWatch logs ${m['logs']:9.2f} ({m['logs_gb']:.1f} GB ingested; ${m['logs_nofree']:.2f} without the 5 GB free tier)") print(f" KMS key ${m['key']:.2f} + secret ${m['secret']:.2f} (request charges not estimated)") print(f" priced lines total ${priced:9.2f} (without any free tier ${nofree:.2f})") print(f" SQS requests (not priced): {m['sqs_requests']:,.0f} -> free tier covers 1,000,000") print("\nNOT in these totals: CloudFront (flat-rate plans read: Free $0, Pro $15, Business $200, Premium $1,000 per distribution; pay-as-you-go not read), SQS beyond the free 1M requests,") print("S3 storage and requests, X-Ray, SNS, KMS request charges, data transfer, a WAF web ACL ($5 + $1 per rule + $0.60 per million requests, if you add one).") --- py cost12.py (saved output of the real run) --- == 100,000 orders a month (1 KB item, batch of 10, 100 ms API / 200 ms per message, 256 MB Arm) == API Gateway HTTP API $ 0.10 Lambda requests $ 0.00 (free tier applied; without it $0.02) Lambda duration $ 0.00 (7,500 GB-s; without free tier $0.10) DynamoDB (writes, 0.5 GB stored, PITR) $0.29 CloudWatch alarms $ 0.00 (5 alarms inside the 10-alarm free tier; $0.50 without it) CloudWatch logs $ 0.00 (0.2 GB ingested; $0.10 without the 5 GB free tier) KMS key $1.00 + secret $0.40 (request charges not estimated) priced lines total $ 1.79 (without any free tier $2.51) SQS requests (not priced): 120,000 -> free tier covers 1,000,000 == 1,000,000 orders a month (1 KB item, batch of 10, 100 ms API / 200 ms per message, 256 MB Arm) == API Gateway HTTP API $ 1.00 Lambda requests $ 0.02 (free tier applied; without it $0.22) Lambda duration $ 0.00 (75,000 GB-s; without free tier $1.00) DynamoDB (writes, 0.5 GB stored, PITR) $0.85 CloudWatch alarms $ 0.00 (5 alarms inside the 10-alarm free tier; $0.50 without it) CloudWatch logs $ 0.00 (2.0 GB ingested; $1.00 without the 5 GB free tier) KMS key $1.00 + secret $0.40 (request charges not estimated) priced lines total $ 3.27 (without any free tier $5.97) SQS requests (not priced): 1,200,000 -> free tier covers 1,000,000 == 10,000,000 orders a month (1 KB item, batch of 10, 100 ms API / 200 ms per message, 256 MB Arm) == API Gateway HTTP API $ 10.00 Lambda requests $ 2.00 (free tier applied; without it $2.20) Lambda duration $ 4.67 (750,000 GB-s; without free tier $10.00) DynamoDB (writes, 0.5 GB stored, PITR) $6.47 CloudWatch alarms $ 0.00 (5 alarms inside the 10-alarm free tier; $0.50 without it) CloudWatch logs $ 7.50 (20.0 GB ingested; $10.00 without the 5 GB free tier) KMS key $1.00 + secret $0.40 (request charges not estimated) priced lines total $ 32.04 (without any free tier $40.58) SQS requests (not priced): 12,000,000 -> free tier covers 1,000,000 NOT in these totals: CloudFront (flat-rate plans read: Free $0, Pro $15, Business $200, Premium $1,000 per distribution; pay-as-you-go not read), SQS beyond the free 1M requests, S3 storage and requests, X-Ray, SNS, KMS request charges, data transfer, a WAF web ACL ($5 + $1 per rule + $0.60 per million requests, if you add one). -------------------------- Reading the result: of 29 one-change regressions of the hardened template, the review flagged 24 and cfn-lint 5 (both 3, neither 3). cfn-lint's E3505 compared the queue's visibility timeout with the function timeout (it caught timeout 300 against visibility 185) but not the six-times rule (it passed a visibility timeout of 30); this differs from my chapter 9 result for a differently shaped template and I did not investigate why. The three changes missed by both: an Action '*' / Resource '*' statement added to the API function's policy, a route method typo POSTT and a batch size of 11. Mistake made and kept in the chapter: the wildcard statement was missed by the review because SEC4 already FAILED on the baseline (X-Ray), and my harness counts a regression only when a passing check starts to fail. A check that already fails cannot report a regression. Fix: keep accepted findings in an allowlist (accepted: the two X-Ray actions on Resource '*') and let every other wildcard fail. Cost sketch: from worked examples on the pricing pages (US East (N. Virginia) where stated); the AWS Pricing Calculator was NOT used; CloudFront pay-as-you-go prices, SQS request prices beyond the free million, S3, X-Ray, SNS, KMS request charges and data transfer are not in the totals.