Chapter 5. Additional Sed Commands

28. Append Line After (a command)

You can insert a new line after a specific location by using the sed append command (a).

Syntax:

$ sed '[address] a the-line-to-append' input-file

Add a new record to the employee.txt file after line number:

$ sed '2 a 203,Jack Johnson,Engineer' employee.txt
101,John Doe,CEO
102,Jason Smith,IT Manager
203,Jack Johnson,Engineer
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,Jane Miller,Sales Manager

Add a new record to the end of the employee.txt file:

$ sed '$ a 106,Jack Johnson,Engineer' employee.txt
101,John Doe,CEO
102,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,Jane Miller,Sales Manager
106,Jack Johnson,Engineer

You can also append multiple lines using the sed a command.

Add two lines after the line that matches 'Jason':

$ sed '/Jason/a\
203,Jack Johnson,Engineer\
204,Mark Smith,Sales Engineer' employee.txt
101,John Doe,CEO
102,Jason Smith,IT Manager
203,Jack Johnson,Engineer
204,Mark Smith,Sales Engineer
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,Jane Miller,Sales Manager

29. Insert Line Before (i command)

The sed insert command (i) works just like the append command except that it inserts a line before a specific location instead of after the location.

Syntax:

$ sed '[address] i the-line-to-insert' input-file

Insert a new record before line number 2 of the employee.txt file:

$ sed '2 i 203,Jack Johnson,Engineer' employee.txt
101,John Doe,CEO
203,Jack Johnson,Engineer
102,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,Jane Miller,Sales Manager

Insert a new record before the last line of the employee.txt file:

$ sed '$ i 108,Jack Johnson,Engineer' employee.txt
101,John Doe,CEO
102,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
108,Jack Johnson,Engineer
105,Jane Miller,Sales Manager

You can also insert multiple lines using the sed i command.

Insert two lines before the line that matches 'Jason':

$ sed '/Jason/i\
203,Jack Johnson,Engineer\
204,Mark Smith,Sales Engineer' employee.txt
101,John Doe,CEO
203,Jack Johnson,Engineer
204,Mark Smith,Sales Engineer
102,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,Jane Miller,Sales Manager

30. Change Line (c command)

The sed change command (c) lets you replace an existing line with new text.

Syntax:

$ sed '[address] c the-line-to-insert' input-file

Delete the record at line number 2 and replace it with a new record:

$ sed '2 c 202,Jack Johnson,Engineer' employee.txt
101,John Doe,CEO
202,Jack Johnson,Engineer
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,Jane Miller,Sales Manager

You can also replace a single line with multiple lines.

Delete the line that matches 'Raj' and replaces it with two new lines:

$ sed '/Raj/c\
203,Jack Johnson,Engineer\
204,Mark Smith,Sales Engineer' employee.txt
101,John Doe,CEO
102,Jason Smith,IT Manager
203,Jack Johnson,Engineer
204,Mark Smith,Sales Engineer
104,Anand Ram,Developer
105,Jane Miller,Sales Manager

31. Combine a, i, and c Commands

You can also combine the a, i, and c commands. the following sed example does all these three things:

  • a - Append 'Jack Johnson' after 'Jason'
  • i - Insert 'Mark Smith' before 'Jason'
  • c - Change 'Jason' to 'Joe Mason'
$ sed '/Jason/ { a\ 204,Jack Johnson,Engineer i\ 202,Mark Smith,Sales Engineer c\ 203,Joe Mason,Sysadmin }' employee.txt
101,John Doe,CEO
202,Mark Smith,Sales Engineer
203,Joe Mason,Sysadmin
204,Jack Johnson,Engineer
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,Jane Miller,Sales Manager

32. Print Hidden Characters (l command)

The sed l command prints the hidden characters, for example, \t for tab, and $ for end of the line.

For testing, create a test file with the following content. Make sure to use the tab key between the fields in this file.

$ cat tabfile.txt
fname         First Name
lname         Last Name
mname         Middle Name

Executing the sed l command will display \t for tab, and $ for EOL:

$ sed -n l tabfile.txt
fname\tFirst Name$
lname\tLast Name$
mname\tMiddle Name$

When you specify a number followed by the l command, the output line is wrapped at the nth number using a non printable character as shown in the example below. This works only on GNU sed.

$ sed -n 'l 20' employee.txt
101,John Doe,CEO$
102,Jason Smith,IT \
Manager$
103,Raj Reddy,Sysad\
min$
104,Anand Ram,Devel\
oper$
105,Jane Miller,Sal\
es Manager$

33. Print Line Numbers (= command)

The sed = command prints line numbers followed by the line content from the input-file.

Print all line numbers:

$ sed = employee.txt
1
101,John Doe,CEO
2
102,Jason Smith,IT Manager
3
103,Raj Reddy,Sysadmin
4
104,Anand Ram,Developer
5
105,Jane Miller,Sales Manager

Note: You can print the line number and the line content in the same line by combining = command with N command (more on this later).

Print line numbers only for lines 1,2 and 3:

$ sed '1,3 =' employee.txt
1
101,John Doe,CEO
2
102,Jason Smith,IT Manager
3
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,Jane Miller,Sales Manager

Print the line number only for those lines that contain the keyword Jane. This still prints the original line content from the intput-file:

$ sed '/Jane/ =' employee.txt
101,John Doe,CEO
102,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
5
105,Jane Miller,Sales Manager

If you want to know only the line numbers of lines that contains the keyword (i.e. without printing the original lines from the file), use -n option along with = as shown below.

$ sed -n '/Raj/ =' employee.txt
3

Print the total number of lines in a file:

$ sed -n '$ =' employee.txt
5

34. Change Case (using the y 'transform' command)

The sed y command transforms characters by position. A convenient use for this is to convert upper case to lower case and vice versa.

In this example character "a" will be transformed to A, b to B, c to C, etc.:

$ sed 'y/abcde/ABCDE/' employee.txt
101,John DoE,CEO
102,JAson Smith,IT MAnAgEr
103,RAj REDDy,SysADmin
104,AnAnD RAm,DEvElopEr
105,JAnE MillEr,SAlEs MAnAgEr

Transform all lower-case letters to upper-case:

$ sed 'y/abcdefghijklmnopqrstuvwxyz/ABCDEFGHIJKLMNOPQRSTUVWXYZ/' employee.txt
101,JOHN DOE,CEO
102,JASON SMITH,IT MANAGER
103,RAJ REDDY,SYSADMIN
104,ANAND RAM,DEVELOPER
105,JANE MILLER,SALES MANAGER

The above command should be executed in a single line as shown below.

sed 'y/abcdefghijklmnopqrstuvwxyz/ABCDEFGHIJKLMNOPQRSTUVWXYZ/' employee.txt

35. Multiple Files in Command Line

In all our previous sed examples, we passed only one input file. You can also pass multiple input files as shown below.

The following example searches for root in the /etc/passwd file and prints it:

$ sed -n '/root/ p' /etc/passwd
root:x:0:0:root:/root:/bin/bash

The following example searches for root in the /etc/group and prints it:

$ sed -n '/root/ p' /etc/group
root:x:0:

Search for root in both the /etc/passwd and /etc/group file:

$ sed -n '/root/ p' /etc/passwd /etc/group
root:x:0:0:root:/root:/bin/bash
root:x:0:

36. Quit Sed (q command)

The sed q command causes sed to quit executing commands. As we discussed earlier, the normal sed execution flow is Read, Execute, Print, Repeat. When sed executes the q command, it simply quits without executing the rest of the sed commands, and without repeating the rest of the lines from the input-file.

Quit after printing the 1st line:

$ sed 'q' employee.txt
101,John Doe,CEO

Quit after the 5th line. So, this prints the 1st 5 lines:

$ sed '5 q' employee.txt
101,John Doe,CEO
102,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,Jane Miller,Sales Manager

Print all the lines until the 1st line that contains the keyword 'Manager':

$ sed '/Manager/q' employee.txt
101,John Doe,CEO
102,Jason Smith,IT Manager

Note: q command doesn't take range of address. It works only on a single address. (or a single pattern)

37. Read from File (r command)

The sed r command will read the content of another file and print it at a specified location while processing the input-file. The following example will read the content of log.txt file and print it after printing the last line of employee.txt. Basically this combines both employee.txt and log.txt and prints the result.

$ sed '$ r log.txt' employee.txt

You can also specify a pattern with the r command. The following example will read the content of log.txt and print it after the line that matches 'Raj' in the employee.txt.

Insert the log.txt file after the 'Raj' keyword in the employee.txt file:

$ sed '/Raj/ r log.txt' employee.txt

38. Simulating Unix commands in sed (cat, grep, head)

We have already seen examples that worked very much like other standard UNIX commands. Using sed you can simulate many commands. Do this just to learn how sed works.

Cat in sed

cat employee.txt

Each of the following sed commands produces the same output as the cat command above.

sed 's/JUNK/&/p' employee.txt
sed -n 'p' employee.txt
sed 'n' employee.txt
sed 'N' employee.txt

Grep in sed

Simple grep:

grep Jane employee.txt

Each of the following sed commands produces the same output as the grep command above.

sed -n 's/Jane/&/p' employee.txt
sed -n '/Jane/ p' employee.txt

grep -v (print non-matching lines):

grep -v Jane employee.txt

The following sed command is equivalent to the above "grep -v" command.

sed -n '/Jane/ !p' employee.txt

Head in sed

head -10 /etc/passwd

Each of the following sed commands produces the same output as the head command above.

sed '11,$ d' /etc/passwd
sed -n '1,10 p' /etc/passwd
sed '10 q' /etc/passwd

39. Sed Command Line Options

-n option

We already discussed this option and we have used it in many examples. The sed option -n suppresses the default printing that happens as part of the standard sed flow.

You can also use --quiet, or –-silent instead of -n. They are identical in function.

All of the following commands are the same:

sed -n 'p' employee.txt
sed --quiet 'p' employee.txt
sed --silent 'p' employee.txt

-f option

You can also combine multiple sed-commands in a file and call the sed script file using the -f option. We demonstrated this earlier. You can also use -–file.

All of the following commands are the same:

sed -n -f test-script.sed /etc/passwd
sed -n --file=test-script.sed /etc/passwd

-e option

Use -e to execute a sed command script from the command line. You can use multiple -e options from the command line. You can also use –-expression.

All of the following commands are the same:

sed -n -e '/root/ p' /etc/passwd
sed -n --expression '/root/ p' /etc/passwd

-i option

As we already discussed sed doesn't touch the input file. It always prints to standard output, Or you can use the w command to write the output to a different file. We also showed how sed can use the -i option to modify the input file directly.

Replace John with Johnny in the original employee.txt file:

sed -i 's/John/Johnny/' employee.txt

Perform the same command but take a backup by passing an extension to -i:

sed -ibak 's/John/Johnny/' employee.txt

Instead of -i, you can also use –-in-place.

Both of the following commands are the same:

sed -ibak 's/John/Johnny/' employee.txt
sed --in-place=bak 's/John/Johnny/' employee.txt

-c option

This should be used in conjunction with sed option -i. Sed option -i typically uses a temporary file to create the changes and renames it to the original input-file when the operation is completed. This might cause file ownership to change. When you use -c along with -i, the input file ownership will not change. You can also use –-copy.

Both of the following commands are the same:

sed -ibak -c 's/John/Johnny/' employee.txt
sed --in-place=bak --copy 's/John/Johnny/' employee.txt

-l option

Specify the line length. This needs to be used in conjunction with the sed l command. The value you specify in the -l option will be used as the line size. You can also use –-line-length.

All the following commands are the same.

sed -n -l 20 'l' employee.txt
sed -n --line-length=20 employee.txt

Please note that you can also achieve the same output without specifying -n option as shown below.

sed -n 'l 20' employee.txt --posix option

40. Print Pattern Space (n command)

The sed n command prints the current pattern space and fetches the next line from the input-file. This happens in the middle of command execution, and so it can change the normal flow if it occurs between other commands.

Print the pattern space for each line:

$ sed n employee.txt
101,John Doe,CEO
102,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,Jane Miller,Sales Manager

If you specify -n flag when you are using the n command, sed will not print anything.

$ sed -n n employee.txt

As we discussed earlier, normal sed execution flow is Read, Execute (all available sed commands), Print, Repeat.

The sed n command lets you change that flow. The sed n command will print the current pattern space, clear the current pattern space, read the next line from the input-file, and continue the command flow.

Let us assume that you have 2 sed commands before and 2 after the n command as shown below.

sed-command-1
sed-command-2
n
sed-command-3
sed-command-4

In this case, sed-command-1 and sed-command-2 will be applied to the current line in the pattern space; when sed encounters the n command, it will clear the current line from the pattern space, read the next line from the input-file, and apply sed-command-3 and sedcommand-4 to this newly read line in the sed pattern space.

Note: The sed n command by itself is relatively useless as you see in the above examples. However, it is extremely powerful when combined with the sed hold pattern commands that are discussed in the following hacks.