Learning SED

Chapter 1 — Introduction to SED

What Is SED?

sed stands for Stream EDitor. Unlike a conventional text editor where you open a file, move a cursor around, and make interactive changes, SED processes text as a stream — it reads input line by line, applies a set of instructions to each line, and writes the result to standard output. You never "open" a file in the traditional sense; you pass text through a pipeline of transformations.

SED was written by Lee McMahon at Bell Labs in 1973–74, based directly on the earlier ed line editor. It became a standard part of Unix and has been included in every Unix-like operating system since. Today you will find it on Linux, macOS, BSD, and anywhere else that follows the POSIX standard. It is one of the oldest tools in the Unix toolchain that is still in daily use by millions of developers and sysadmins worldwide.

The key insight that makes SED powerful is this: most text-processing tasks reduce to transformations on a stream of lines. Finding and replacing text, deleting certain lines, extracting specific sections, reformatting data — SED handles all of these with a compact command syntax that can be expressed in a single shell command or a small script file.

Where SED Fits in the Toolchain

Before diving into syntax, it helps to understand when to reach for SED versus its close relatives:

SED is the right choice when you need to modify text — change words, rearrange lines, remove sections, or insert new content — especially when that modification needs to be applied consistently across a large file or integrated into an automated script.

Basic Syntax

Every SED command follows the same fundamental structure:

sed [options] 'script' file

Breaking that down:

• sed — the command itself
• [options] — optional flags that modify SED's behaviour (covered below)
• 'script' — one or more SED commands telling it what to do, enclosed in single quotes
• file — the input file; if omitted, SED reads from standard input

The simplest SED script is a single command. Here is the classic substitution — replacing one word with another:

sed 's/hello/goodbye/' greetings.txt

This reads every line of greetings.txt, replaces the first occurrence of hello with goodbye on each line, and prints the result. The original file is unchanged. The output goes to your terminal (standard output).

You can also pipe input directly into SED:

echo "hello world" | sed 's/hello/goodbye/'
goodbye world

How SED Processes Text — The Cycle

Understanding SED's internal processing cycle is the key to understanding everything else about it. SED works in a loop — one iteration per line of input:

1. Read — SED reads one line from the input (stripping the trailing newline) and places it in the pattern space
2. Execute — SED runs all commands in the script against the pattern space, modifying it as instructed
3. Print — unless told otherwise, SED prints the contents of the pattern space to standard output, followed by a newline
4. Clear — the pattern space is cleared and the cycle repeats with the next line

The pattern space is SED's working buffer — a temporary area in memory where the current line lives while SED processes it. Most SED commands read from and write to the pattern space.

There is also a second buffer called the hold space — a persistent area that survives between cycles. This lets you save content from one line and use it when processing a later line. The hold space starts empty and is covered in depth in Chapter 6.

Your First SED Commands

Substitution — s

The substitute command s is what most people use SED for. Its syntax is:

s/pattern/replacement/flags

For now, the basic form without flags replaces the first match on each line:

# Replace first occurrence of "cat" with "dog" on each line
echo "the cat sat on the cat mat" | sed 's/cat/dog/'
the dog sat on the cat mat

# Add the g (global) flag to replace ALL occurrences on each line
echo "the cat sat on the cat mat" | sed 's/cat/dog/g'
the dog sat on the dog mat

Substitution is covered thoroughly in Chapter 2 — for now, just get comfortable with the basic form.

Delete — d

The delete command removes lines from the output entirely. Combine it with an address (a line number or pattern) to target specific lines:

# Delete line 3
sed '3d' file.txt

# Delete every line containing "DEBUG"
sed '/DEBUG/d' logfile.txt

# Delete blank lines (lines that match the empty-line pattern)
sed '/^$/d' file.txt

Print — p

The print command explicitly prints the pattern space. On its own this causes each line to be printed twice (once by p, once by SED's automatic print at end of cycle). It becomes useful with the -n option, which suppresses the automatic print:

# Print only lines containing "ERROR" (suppress all others)
sed -n '/ERROR/p' logfile.txt

# Print only lines 5 through 10
sed -n '5,10p' file.txt

Common Options

In-Place Editing with -i

By default SED never touches your input file — it only writes to standard output. The -i option changes this, making SED modify the file directly. This is one of the most commonly used options in shell scripts.

# Replace "version=1.0" with "version=2.0" inside config.txt
sed -i 's/version=1.0/version=2.0/' config.txt

sed -i.bak 's/old/new/g' important.conf
# Original saved as: important.conf.bak
# Modified version:  important.conf

Reading From Standard Input vs Files

SED is equally happy reading from a file argument or from a pipe. This flexibility is what makes it so composable with other tools.

# From a file
sed 's/foo/bar/' input.txt

# From a pipe
cat input.txt | sed 's/foo/bar/'

# From another command's output
grep "ERROR" logfile.txt | sed 's/ERROR/FAULT/'

# From here-string
sed 's/hello/hi/' <<< "hello world"
hi world

# Multiple files — SED processes them as one continuous stream
sed 's/foo/bar/' file1.txt file2.txt file3.txt

When multiple files are given, SED treats them as a single concatenated input. Line numbers are counted continuously across all files.

Running Multiple Commands

SED can run more than one command in a single invocation. There are three ways to do this:

Semicolons (for simple cases)

# Two substitutions separated by a semicolon
sed 's/foo/bar/; s/baz/qux/' file.txt

Multiple -e flags

# Each -e introduces one command
sed -e 's/foo/bar/' -e 's/baz/qux/' -e '/DEBUG/d' file.txt

A script file with -f

# Write commands into a file, one per line
# Contents of cleanup.sed:
#   s/foo/bar/g
#   s/baz/qux/g
#   /^$/d
sed -f cleanup.sed file.txt

Script files are the cleanest option for anything longer than two or three commands. They are easier to read, easier to maintain, and can be version-controlled like any other code.

GNU SED vs BSD SED

There are two main flavours of SED you will encounter:

Throughout this course, examples use GNU SED conventions (Linux). Where a macOS difference matters, it will be noted. If you are on macOS and want GNU SED behaviour, you can install it with Homebrew: brew install gnu-sed — it installs as gsed.

A Practical First Example

Let us put the basics together with a realistic scenario. Suppose you have a configuration file and you want to update a setting without opening an editor:

# app.conf — before
debug=false
log_level=warn
max_connections=10
server_name=localhost

# Change log_level from warn to info, and increase max_connections to 50
sed -i.bak -e 's/log_level=warn/log_level=info/' \
           -e 's/max_connections=10/max_connections=50/' app.conf

# app.conf — after
debug=false
log_level=info
max_connections=50
server_name=localhost

The original is preserved as app.conf.bak. The changes are applied atomically in a single SED pass — the file is read once, both substitutions run on each line, and the result is written back.

Quick Reference — Chapter 1