aws2-1 Exercise 2: Model SCP Evaluation and Test It Against the Documented Scenarios ====================================================================================== Run with Python 3.14.0 and cfn-lint 1.57.2 on Windows 11. VERIFICATION LEVEL: 1 (run here, no AWS account, no AWS CLI). What the scripts check is MY reading of AWS documentation pages read on 2026-10-12 (through a page-fetch tool that summarises pages); it is not AWS's own behaviour, and AWS may change its limits and rules. The 'docs say' column is my reading of the prose of each scenario on the SCP evaluation page; the figures are images I could not see, so in scenarios 1 and 4 I assumed accounts not described as having extra SCPs have only FullAWSAccess. --- scp_eval.py --- """scp_eval.py - a small MODEL of the SCP evaluation rules in AWS's "SCP evaluation" page. It is my implementation of the documented rules, not AWS. Rules modelled: * an action is allowed for a member account only if, at EVERY level from the root down to the account itself, at least one attached SCP has an Allow statement that matches it (page: "How SCPs work with Allow") * a Deny in ANY attached SCP at ANY level on the path blocks it (page: "How SCPs work with Deny") * several SCPs at one level combine as a union of their Allow statements (page: Scenario 7) * the management account is not restricted by SCPs (page: "SCPs don't affect users or roles in the management account") * an SCP grants nothing: the identity-based policy must also allow the action (page: "SCP effects on permissions") Not modelled (the model raises an error rather than guessing): Condition, Resource other than "*", resource-based policies, permissions boundaries, session policies, service-linked roles. """ from fnmatch import fnmatchcase class NotModelled(Exception): pass def match(patterns, action): a = action.lower() if isinstance(patterns, str): patterns = [patterns] return any(fnmatchcase(a, p.lower()) for p in patterns) def stmt_hits(s, action): if "Condition" in s: raise NotModelled("Condition") if s.get("Resource", "*") != "*" and s.get("NotResource") is None: raise NotModelled("Resource " + str(s["Resource"])) if "NotResource" in s: raise NotModelled("NotResource") if "Action" in s: return match(s["Action"], action) return not match(s["NotAction"], action) def level_verdict(scps, action): allow = any(s["Effect"] == "Allow" and stmt_hits(s, action) for p in scps for s in p) deny = any(s["Effect"] == "Deny" and stmt_hits(s, action) for p in scps for s in p) return allow, deny def scp_allows(path_levels, action): """path_levels: list (root ... account) of lists of policies (each policy = list of statements).""" for lv in path_levels: allow, deny = level_verdict(lv, action) if deny or not allow: return False return True def effective(path_levels, identity_allow, action, management=False): if management: return match(identity_allow, action) # SCPs do not apply return scp_allows(path_levels, action) and match(identity_allow, action) # ---- the AWS-managed policy and the small policies the docs' scenarios use FULL = [{"Effect": "Allow", "Action": "*", "Resource": "*"}] def allow(*svcs): return [{"Effect": "Allow", "Action": [s + ":*" for s in svcs], "Resource": "*"}] def deny(*svcs): return [{"Effect": "Deny", "Action": [s + ":*" for s in svcs], "Resource": "*"}] PROBES = ["ec2:RunInstances", "s3:GetObject", "iam:CreateRole", "cloudwatch:PutMetricData", "lambda:InvokeFunction"] ALL = {p.split(":")[0] for p in PROBES} # scenario -> account -> (levels from root to account, what the docs say is allowed: set of services, or 'all-but-X') SCEN = {} SCEN["1 deny at OU and account"] = { "A": ([[FULL], [FULL, deny("s3")], [FULL]], ALL - {"s3"}), "B": ([[FULL], [FULL, deny("s3")], [FULL, deny("ec2")]], ALL - {"s3", "ec2"}), } SCEN["2 allow-list at the OU"] = { "A": ([[FULL], [allow("ec2")], [FULL]], {"ec2"}), "B": ([[FULL], [allow("ec2")], [FULL]], {"ec2"}), } SCEN["3 root has only a Deny"] = { "A": ([[deny("s3")], [FULL], [FULL]], set()), } SCEN["4 layered deny, two OU levels"] = { "D (Test)": ([[FULL], [FULL], [FULL, deny("ec2")], [FULL]], ALL - {"ec2"}), "E (Production)": ([[FULL], [FULL], [FULL], [FULL]], ALL), } SCEN["6 root deny beats lower allow"] = { "D (Test)": ([[FULL, deny("s3")], [allow("s3")], [FULL]], set()), "E (Production)": ([[FULL, deny("s3")], [FULL], [FULL]], ALL - {"s3"}), } SCEN["7 two allow-list SCPs at the root"] = { "A": ([[allow("iam", "ec2"), allow("s3", "cloudwatch")], [FULL], [FULL]], {"iam", "ec2", "s3", "cloudwatch"}), } def run(flip=None): wrong, rows = 0, [] for sname, accts in SCEN.items(): for an, (levels, expected) in accts.items(): exp = set(expected) if flip == (sname, an): exp = exp ^ {"lambda"} # control: a deliberately wrong expectation got = {p.split(":")[0] for p in PROBES if scp_allows(levels, p)} ok = got == exp wrong += (not ok) rows.append((sname, an, ",".join(sorted(exp)) or "(none)", ",".join(sorted(got)) or "(none)", "match" if ok else "MISMATCH")) return rows, wrong if __name__ == "__main__": rows, wrong = run() print(f"{'scenario':<34}{'account':<16}{'docs say allowed':<42}{'model says':<42}") for r in rows: print(f"{r[0]:<34}{r[1]:<16}{r[2]:<42}{r[3]:<42}{r[4]}") print(f"mismatches: {wrong} of {len(rows)}\n") rows, wrong = run(flip=("1 deny at OU and account", "A")) print(f"control (one expectation flipped on purpose): mismatches {wrong} of {len(rows)}\n") print("-- the SCP grants nothing: same account, three identity policies") lv = [[FULL], [FULL], [FULL]] for label, ident in (("no identity policy", []), ("identity policy allows s3:*", ["s3:*"]), ("AdministratorAccess (*)", ["*"])): print(f" {label:<30} s3:GetObject -> {effective(lv, ident, 's3:GetObject')}, ec2:RunInstances -> {effective(lv, ident, 'ec2:RunInstances')}") print("-- same account, but an SCP Deny on ec2, with AdministratorAccess") lv = [[FULL], [FULL, deny('ec2')], [FULL]] print(f" ec2:RunInstances -> {effective(lv, ['*'], 'ec2:RunInstances')}, s3:GetObject -> {effective(lv, ['*'], 's3:GetObject')}") print("-- the management account with the same deny") print(f" ec2:RunInstances -> {effective(lv, ['*'], 'ec2:RunInstances', management=True)} (SCPs do not apply there)") print("-- what the model refuses to guess") cond = [{"Effect": "Deny", "NotAction": ["iam:*"], "Resource": "*", "Condition": {"StringNotEquals": {"aws:RequestedRegion": ["eu-west-1"]}}}] try: scp_allows([[FULL, cond]], "ec2:RunInstances") except NotModelled as e: print(" NotModelled:", e) --- py -X utf8 scp_eval.py --- scenario account docs say allowed model says 1 deny at OU and account A cloudwatch,ec2,iam,lambda cloudwatch,ec2,iam,lambda match 1 deny at OU and account B cloudwatch,iam,lambda cloudwatch,iam,lambda match 2 allow-list at the OU A ec2 ec2 match 2 allow-list at the OU B ec2 ec2 match 3 root has only a Deny A (none) (none) match 4 layered deny, two OU levels D (Test) cloudwatch,iam,lambda,s3 cloudwatch,iam,lambda,s3 match 4 layered deny, two OU levels E (Production) cloudwatch,ec2,iam,lambda,s3 cloudwatch,ec2,iam,lambda,s3 match 6 root deny beats lower allow D (Test) (none) (none) match 6 root deny beats lower allow E (Production) cloudwatch,ec2,iam,lambda cloudwatch,ec2,iam,lambda match 7 two allow-list SCPs at the root A cloudwatch,ec2,iam,s3 cloudwatch,ec2,iam,s3 match mismatches: 0 of 10 control (one expectation flipped on purpose): mismatches 1 of 10 -- the SCP grants nothing: same account, three identity policies no identity policy s3:GetObject -> False, ec2:RunInstances -> False identity policy allows s3:* s3:GetObject -> True, ec2:RunInstances -> False AdministratorAccess (*) s3:GetObject -> True, ec2:RunInstances -> True -- same account, but an SCP Deny on ec2, with AdministratorAccess ec2:RunInstances -> False, s3:GetObject -> True -- the management account with the same deny ec2:RunInstances -> True (SCPs do not apply there) -- what the model refuses to guess NotModelled: Condition WHY THIS WORKS AS AN ANSWER --------------------------- The expectations come from the documentation, not from the model's own output, so a match is evidence that the model agrees with the documentation. All ten account results match. The control (one expectation flipped on purpose) reports exactly one mismatch, which shows the comparison can fail. The model keeps the two rules that matter apart: Allow must be present at every level (a root with only a Deny gives no access at all, scenario 3) and a Deny at any level wins (a root deny of S3 beats an allow at the OU, scenario 6); several SCPs at one level are a union (scenario 7). The extra runs show that an SCP grants nothing (no identity policy means no access) and that an administrator is still blocked by a deny, but the management account is not. Raising an error on Condition is deliberate: the model says what it does not know rather than returning a guess. Not run: real requests against AWS, conditions, resource-based policies, permission boundaries, session policies, service-linked roles, and RCPs. AWS Intermediate/Advanced 2 covers the full evaluation order.