Undefined Behavior Deep Dive
"Undefined behavior" has appeared in nearly every chapter of this course. This one finally defines it precisely, explains the genuinely counter-intuitive reason it's dangerous, and catalogs every instance flagged so far in one place.
What "Undefined Behavior" Actually Means
The C standard defines three genuinely different categories, and conflating them is a real, common mistake:
- Unspecified behavior — the standard allows several possible results; the implementation picks one, but doesn't have to document which
- Implementation-defined behavior — unspecified, but the implementation must document its choice consistently (
c3-1's bit-field ordering is exactly this) - Undefined behavior — the standard imposes literally no requirements at all. Anything is a conforming result — including things that look nonsensical, like appearing to work perfectly, or genuinely misbehaving in a way that has nothing obviously to do with the actual bug
Why the Compiler Is Allowed to Assume UB Never Happens
This is the real, counter-intuitive payoff. Compilers don't merely fail to detect UB — they actively use its absence as a proven fact during optimization. If code contains a signed overflow, the compiler is standard-sanctioned in assuming it never actually occurs, and can eliminate branches or checks that logically depended on that "impossible" case — producing results that look like a compiler bug, but are the compiler correctly exploiting a guarantee the standard itself grants it. The C community's own phrase for the theoretical extreme of this is that UB could licitly make "demons fly out of your nose" — a real, standard-permitted outcome, however absurd it sounds.
A Consolidated Catalog, By Chapter
- Signed integer overflow —
c1-2 - Out-of-bounds array access —
c1-6 - Dereferencing NULL or a dangling pointer —
c1-7 - Buffer overflow via strcpy —
c1-8 - Use-after-free, double-free —
c2-2,c2-3 - Reading the "wrong" union member —
c2-1, in most cases - Excessive or negative shifts —
c3-1 - An unprotected data race —
c3-3
Signed Overflow, Revisited With the Real Mechanism
The programmer intended this loop to terminate once i overflows. But since signed overflow is UB, the compiler is entitled to assume i + 1 > i is always true — overflow "can't happen" — and may optimize this into a genuine infinite loop, silently removing the termination condition the programmer relied on. This is a real, documented class of surprising optimizer behavior, not a hypothetical.
Strict Aliasing
A new rule: the compiler assumes two pointers of different, unrelated types never point at the same memory (with narrow exceptions like char *). Reinterpreting a float * as an int * and dereferencing it violates this — undefined behavior — and permits the compiler to reorder or cache reads through such pointers in ways that produce genuinely wrong results if the assumption turns out to be false. c2-1's union type-punning gotcha is closely related: a union is one of the narrow, standard-sanctioned ways to reinterpret bits in some cases — a raw pointer cast between unrelated types is not, and violates strict aliasing outright.
Rust's Answer
Rust's entire type system and borrow checker exist substantially to make every category on this chapter's own catalog structurally impossible to write in safe code — not detected as a separate check layered on top, but literally inexpressible. rust3-2's own unsafe blocks are the deliberate, opt-in escape hatch — the one place a Rust programmer takes on exactly the same responsibility C always has, by choice, in a clearly marked and narrow scope.
| UB category | C | Rust |
|---|---|---|
| Signed overflow | UB — compiler may assume it never happens | panics in debug builds, wraps in release — always defined |
| Out-of-bounds access | UB — no check at all | panics — checked on every access |
| Type punning | UB via raw pointer casts; unions are narrowly sanctioned | requires an explicit unsafe block |
-fsanitize=undefined compiles in instrumentation specifically for UB classes like signed overflow and strict-aliasing violations — genuinely distinct from c2-3's AddressSanitizer, which targets memory-safety bugs specifically. Real projects commonly run both together.
Coding Challenges
For each entry in this chapter's consolidated UB catalog, name which specific chapter first introduced it and, in one sentence each, what triggers it.
📄 View solutionExplain why "the compiler is allowed to assume UB never happens" is a genuinely different, stronger claim than "the compiler doesn't check for UB" — and why that distinction is what makes the i + 1 > i infinite-loop example possible.
📄 View solutionExplain the difference between unspecified, implementation-defined, and undefined behavior, giving one concrete example of each from this course.
📄 View solutionChapter 4 Quick Reference
- Unspecified — one of several allowed results, undocumented; implementation-defined — unspecified, but documented; undefined — no requirements at all
- The compiler actively assumes UB never happens, and optimizes on that assumption — not merely "fails to catch it"
- A full UB catalog spans nearly every chapter of this course, from signed overflow to data races
- Strict aliasing — pointers of unrelated types are assumed never to overlap; violating it is UB
-fsanitize=undefined(UBSan) — a distinct tool from ASan, targeting overflow/aliasing-class UB specifically- Rust's type system makes this entire catalog structurally inexpressible in safe code —
unsafeis the deliberate, narrow opt-out - Next chapter: Debugging & Tooling — gdb, valgrind, ASan/UBSan, and static analysis, brought together