prize18-9 Exercise 3: Finding Bidirectional Controls in Text ============================================================= Plan: walk the lines and columns, test each character against the nine code points (U+202A to U+202E and U+2066 to U+2069), and report line, column and name. Stripping simply drops those characters. Columns count code points, starting at 1. ex3.py: import unicodedata as ud BIDI = set(range(0x202A, 0x202F)) | set(range(0x2066, 0x206A)) def scan(text): hits = [] for lineno, line in enumerate(text.splitlines(), 1): for col, c in enumerate(line, 1): if ord(c) in BIDI: hits.append((lineno, col, f"U+{ord(c):04X} {ud.name(c)}")) return hits def strip_bidi(text): return "".join(c for c in text if ord(c) not in BIDI) source = "x = 1\n# note \u202e reversed \u2069\nprint('ok')\n" for hit in scan(source): print(hit) clean = strip_bidi(source) print(scan(clean), len(source), len(clean)) print(ascii(source.splitlines()[1]), ascii(clean.splitlines()[1])) Real output (Python 3.14): (2, 8, 'U+202E RIGHT-TO-LEFT OVERRIDE') (2, 19, 'U+2069 POP DIRECTIONAL ISOLATE') [] 38 36 '# note \u202e reversed \u2069' '# note reversed ' Two characters were found and removed (38 down to 36 code points). In the escaped view the controls are plainly visible; in an ordinary viewer they are invisible, which is the point. Note that the sample is also unbalanced (an override with no matching terminator, and an isolate terminator without an opener), the kind of pattern the Trojan Source defences recommend tools should warn about. Real right-to-left text legitimately uses such controls, so report first and strip only where policy says so.