HOME

Learn sed

Contact Me

sed

sed means 'The Stream Editor', acts on a data stream. Everything in sed is instruction which has 2 parts : address and action.

sed reads from standard input and/or file(s), applies specified changes which are typically mostly editing changes, and writes to standard output.

sed doesn't alter its input file(s)*, but writes the data to standard output, which by default is your monitor screen.

sed's commands are mostly limited to editing type commands.

sed (which stands for Stream EDitor) is a simple but powerful computer program used to apply various pre-specified textual transformations to a sequential stream of text data.

It reads input files line by line, edits each line according to rules specified in its simple language (the sed script), and then outputs the line.


SYNTAX

sed options 'address action' file(s)
-- note that address and action are in single quotes

Address can be specified using line addressing OR Context addressing.


SOME OF THE ACTIONS AVAILABLE IN sed

Action			Description
-----------------------------------------
p			Prints the line
d			Deletes the line
s/pattern1/pattern2/	Substitutes the first occurrence of pattern1 with pattern2.

For illustration purpose, below is sample data file named ts

DATA
-------
=> cat ts
alpha
beta
gamma
delta
epsilon
zeta

LINE ADDRESSING


1) Printing a range of lines (without specifying -n option, this will give duplicate lines). Priting line numbers 1-3:



=> sed -n '1,3p' ts
alpha
beta
gamma

2) Quit after printing first 4 lines :

=> sed '4q' ts
alpha
beta
gamma
delta

3) Reversing line selection criteria ($ denotes last line).

Here , we are printing all lines except the last one:

=> sed -n '$!p' ts
alpha
beta
gamma
delta
epsilon

This will print all lines except 5th and the last one:

=> sed -n '5,$!p' ts
alpha
beta
gamma
delta

4) Specify all in a single line. Print 1st, 2-4th and last line. See usage of -e option here:

=> sed -n -e '1p' -e '2,4p' -e '$p' ts
alpha
beta
gamma
delta
zeta

5) If there are a large number of commands to be executed, put them in a file and supply that file to sed.

This example will take commands from ip.txt and use on data in ts file :

sed -f file ts

Lets put some substitution commands in ip.txt:

=> cat ip.txt
s/skiing/hiking/g
s/hiking/biking/g

echo "I enjoy hiking and he enjoys skiing" | sed -f ip.txt
Output--> I enjoy biking and he enjoys biking

6) Delete first 2 lines (this will NOT delete from file, just from output)

=> sed '1,2 d' ts
gamma
delta
epsilon
zeta

7) Writing to a file (w command). This example searches 'delta' in ts and writes to f1 file:

=> sed -n '/delta/w f1' ts

=> cat f1
delta

Matching multiple pattern for writing to a file.
This example matches lines containing 'delta' and 'zeta' words in file ts and write those to file named f1: => sed -n -e '/delta/w f1' -e '/zeta/w f1' ts => cat f1 delta zeta

CONTEXT ADDRESSING


1) Search something in file ts and print it:

=> sed -n '/beta/p' ts
beta

=> sed -n '/beta/,/epsilon/p' ts
beta
gamma
delta
epsilon

2) Delete a search string from output

=> sed '/gamma/d' ts
alpha
beta
delta
epsilon
zeta

--Deleting/excluding line numbers 1,2

=> sed '1,2 d' ts
gamma
delta
epsilon
zeta

3) Deleting blank lines from a file:

Alter ts file and introduce some blank lines (use vi editor's o command. Check my article on vi editor for more info.):

--Sample input file (see the blank lines in file):
=> cat ts
alpha
beta
gamma

delta
epsilon

zeta

--This removes blank lines and displays results. Input file is not altered. Less than sign (<) means take input from ts file. 

=> sed '/^$/d' < ts
alpha
beta
gamma
delta
epsilon
zeta

--We can direct output after removing blank lines to another file using greater than operator (>).

=>sed '/^$/d' < ts > ts_out

=> cat ts_out
alpha
beta
gamma
delta
epsilon
zeta

An example:

--Sample file 
$ cat fruit_prices.txt Fruit Price/lbs Banana 0.89 Paech 0.79 Kiwi 1.50 Pineapple 1.29 Apple 0.99 Mango 2.20

Suppose its required to print out a list of those fruits that cost less than $1 per pound.

$ sed -n '/0\.[0-9][0-9]$/p' fruit_prices.txt 
Banana 0.89
Paech 0.79
Apple 0.99

SUBSTITUTION using sed


Basic Syntax --> 

sed "s/oldval/newval/g" file1 > file2

In its simplest form, its like :

=> echo Sunday | sed 's/day/night/'
Sunnight

echo "I enjoy hiking and he enjoys skiing" | sed -e 's/skiing/hiking/g' -e 's/hiking/biking/g'
Output ---> I enjoy biking and he enjoys biking

Flag 'g' stands for global.

Again consider input file ts:

=> cat ts
alpha
beta
gamma
delta
epsilon
zeta

Specifying address. We can specify substitution to search only the given line addresses. This example targets only first 3 lines:

=> sed '1,3s/gamma/GAMMA/g' ts
alpha
beta
GAMMA
delta
epsilon
zeta

No address means substitution will apply to all the lines:

=> sed 's/epsilon/S/g' ts
alpha
beta
gamma
delta
S
zeta

Below addressing is same as above, remember that $ means last.
So this example targets 1 to last ($) lines in ts file :

sed '1,$s/l/L/g' ts
=> sed '1,$s/l/L/g' ts
aLpha
beta
gamma
deLta
epsiLon
zeta

Without global flag, it will only change the first occurrence of the word on a line.
This will replace only first occurrence of 'm' in any line

sed '1,$s/m/M/' ts 
=> sed '1,$s/m/M/' ts
alpha
beta
gaMma
delta
epsilon
zeta

By putting 'g' flag, it will replace all occurrences of 'm' in any line:

sed '1,$s/m/M/g' ts  
=> sed '1,$s/m/M/g' ts
alpha
beta
gaMMa
delta
epsilon
zeta

Multiple Substitutions using sed


Put commands separated by -e option

=> sed -e 's/alpha/A/g' -e 's/zeta/Z/g' -e 's/m/M/g' ts
A
beta
gaMMa
delta
epsilon
Z

When there are multiple substitution options, better place them in a file, and give that file to sed using -f option.

=> cat ip
s/alpha/A/g
s/zeta/Z/g
s/m/M/g

=> sed -f ip ts
A
beta
gaMMa
delta
epsilon
Z

Print the line number of a pattern and find the number of lines in a file :

=> cat ts
alpha
beta
gamma

delta

epsilon
zeta

--this prints line number where 'delta' appears:

=> sed -n '/delta/=' ts
5

--this tells line number of end of file, i.e. number of lines in a file:

=> sed -n '$=' ts
8


I hope above article provided a good glimpse of Unix's sed command. Do some more hands-on to get comfortable with it.
Remember, sed supports regular expressions too, so do consider practising them too.


For further reading, just google Unix sed command.



Thanks...

TOP