sed means 'The Stream Editor', acts on a data stream. Everything in sed is instruction which has 2 parts : address and action.
sed reads from standard input and/or file(s), applies specified changes which are typically mostly editing changes, and writes to standard output.
sed doesn't alter its input file(s)*, but writes the data to standard output, which by default is your monitor screen.
sed's commands are mostly limited to editing type commands.
sed (which stands for Stream EDitor) is a simple but powerful computer program used to apply various pre-specified textual transformations to a sequential stream of text data.
It reads input files line by line, edits each line according to rules specified in its simple language (the sed script), and then outputs the line.
sed options 'address action' file(s)
-- note that address and action are in single quotes
Address can be specified using line addressing OR Context addressing.
SOME OF THE ACTIONS AVAILABLE IN sed
Action Description ----------------------------------------- p Prints the line d Deletes the line s/pattern1/pattern2/ Substitutes the first occurrence of pattern1 with pattern2.
For illustration purpose, below is sample data file named ts
DATA ------- => cat ts alpha beta gamma delta epsilon zeta
1) Printing a range of lines (without specifying -n option, this will give duplicate lines). Priting line numbers 1-3:
=> sed -n '1,3p' ts alpha beta gamma
2) Quit after printing first 4 lines :
=> sed '4q' ts alpha beta gamma delta
3) Reversing line selection criteria ($ denotes last line).
Here , we are printing all lines except the last one: => sed -n '$!p' ts alpha beta gamma delta epsilon This will print all lines except 5th and the last one: => sed -n '5,$!p' ts alpha beta gamma delta
4) Specify all in a single line. Print 1st, 2-4th and last line. See usage of -e option here:
=> sed -n -e '1p' -e '2,4p' -e '$p' ts alpha beta gamma delta zeta
5) If there are a large number of commands to be executed, put them in a file and supply that file to sed.
This example will take commands from ip.txt and use on data in ts file :
sed -f file ts
Lets put some substitution commands in ip.txt:
=> cat ip.txt s/skiing/hiking/g s/hiking/biking/g echo "I enjoy hiking and he enjoys skiing" | sed -f ip.txt Output--> I enjoy biking and he enjoys biking
6) Delete first 2 lines (this will NOT delete from file, just from output)
=> sed '1,2 d' ts gamma delta epsilon zeta
7) Writing to a file (w command). This example searches 'delta' in ts and writes to f1 file:
=> sed -n '/delta/w f1' ts => cat f1 delta Matching multiple pattern for writing to a file.
This example matches lines containing 'delta' and 'zeta' words in file ts and write those to file named f1: => sed -n -e '/delta/w f1' -e '/zeta/w f1' ts => cat f1 delta zeta
1) Search something in file ts and print it:
=> sed -n '/beta/p' ts beta => sed -n '/beta/,/epsilon/p' ts beta gamma delta epsilon
2) Delete a search string from output
=> sed '/gamma/d' ts alpha beta delta epsilon zeta --Deleting/excluding line numbers 1,2 => sed '1,2 d' ts gamma delta epsilon zeta
3) Deleting blank lines from a file:
Alter ts file and introduce some blank lines (use vi editor's o command. Check my article on vi editor for more info.):
--Sample input file (see the blank lines in file): => cat ts alpha beta gamma delta epsilon zeta --This removes blank lines and displays results. Input file is not altered. Less than sign (<) means take input from ts file. => sed '/^$/d' < ts alpha beta gamma delta epsilon zeta --We can direct output after removing blank lines to another file using greater than operator (>). =>sed '/^$/d' < ts > ts_out => cat ts_out alpha beta gamma delta epsilon zeta
$ cat fruit_prices.txt Fruit Price/lbs Banana 0.89 Paech 0.79 Kiwi 1.50 Pineapple 1.29 Apple 0.99 Mango 2.20
Suppose its required to print out a list of those fruits that cost less than $1 per pound.
$ sed -n '/0\.[0-9][0-9]$/p' fruit_prices.txt Banana 0.89 Paech 0.79 Apple 0.99
SUBSTITUTION using sed
Basic Syntax --> sed "s/oldval/newval/g" file1 > file2 In its simplest form, its like : => echo Sunday | sed 's/day/night/' Sunnight echo "I enjoy hiking and he enjoys skiing" | sed -e 's/skiing/hiking/g' -e 's/hiking/biking/g' Output ---> I enjoy biking and he enjoys biking Flag 'g' stands for global.
Again consider input file ts:
=> cat ts alpha beta gamma delta epsilon zeta
Specifying address. We can specify substitution to search only the given line addresses. This example targets only first 3 lines:
=> sed '1,3s/gamma/GAMMA/g' ts alpha beta GAMMA delta epsilon zeta
No address means substitution will apply to all the lines:
=> sed 's/epsilon/S/g' ts alpha beta gamma delta S zeta
Below addressing is same as above, remember that $ means last.
So this example targets 1 to last ($) lines in ts file :
sed '1,$s/l/L/g' ts => sed '1,$s/l/L/g' ts aLpha beta gamma deLta epsiLon zeta
Without global flag, it will only change the first occurrence of the word on a line.
This will replace only first occurrence of 'm' in any line
sed '1,$s/m/M/' ts => sed '1,$s/m/M/' ts alpha beta gaMma delta epsilon zeta
By putting 'g' flag, it will replace all occurrences of 'm' in any line:
sed '1,$s/m/M/g' ts => sed '1,$s/m/M/g' ts alpha beta gaMMa delta epsilon zeta
Multiple Substitutions using sed
Put commands separated by -e option
=> sed -e 's/alpha/A/g' -e 's/zeta/Z/g' -e 's/m/M/g' ts A beta gaMMa delta epsilon Z
When there are multiple substitution options, better place them in a file, and give that file to sed using -f option.
=> cat ip s/alpha/A/g s/zeta/Z/g s/m/M/g => sed -f ip ts A beta gaMMa delta epsilon Z
Print the line number of a pattern and find the number of lines in a file :
=> cat ts alpha beta gamma delta epsilon zeta --this prints line number where 'delta' appears: => sed -n '/delta/=' ts 5 --this tells line number of end of file, i.e. number of lines in a file: => sed -n '$=' ts 8
I hope above article provided a good glimpse of Unix's sed command. Do some more hands-on to get comfortable with it.
Remember, sed supports regular expressions too, so do consider practising them too.
For further reading, just google Unix sed command.