sed

Powerful streaming text editor

Supplementary instructions

sed is a stream editor. It is a very important tool in text processing. It can be used perfectly with regular expressions and has extraordinary functions. During processing, the currently processed line is stored in a temporary buffer, called "pattern space", and then the sed command is used to process the contents of the buffer. After the processing is completed, the contents of the buffer are sent to the screen. Then process the next line, and repeat until the end of the file. The file contents are not changed unless you use redirection to store the output. Sed is mainly used to automatically edit one or more files; simplify repeated operations on files; write conversion programs, etc.

sed options, commands, replacement tags

Command format

sed [options] 'command' file(s)
sed [options] -f scriptfile file(s)

Options

-e<script> or --expression=<script>: Process the input text file with the script specified in the option;
-f<script file> or --file=<script file>: Process the input text file with the script file specified in the option;
-h or --help: display help;
-n or --quiet or --silent: only display the results after script processing;
-V or --version: Display version information.

Parameters

File: Specify a list of text files to be processed.

sed command

a\ # Insert text below the current line.
i\ # Insert text above the current line.
c\ # Change the selected line to new text.
d # Delete, delete the selected row.
D # Delete the first line of the template block.
s #Replace specified characters
h #Copy the contents of the template block to the buffer in memory.
H # Append the contents of the template block to the buffer in memory.
g # Get the contents of the memory buffer and replace the text in the current template block.
G # Get the contents of the memory buffer and append it to the text of the current template block.
l # List cannot print a list of characters.
n # Read the next input line and use the next command to process the new line instead of the first command.
N # Append the next input line after the template block and embed a new line between them, changing the current line number.
p # Print the lines of the template block.
P # (uppercase) prints the first line of the template block.
q #Exit Sed.
b labelable # Branch to the marked place in the script, or branch to the end of the script if the branch does not exist.
r file # Read lines from file.
t label #if branch, starting from the last line, once the condition is met or T, t command, it will cause branching to the command with the label, or to the end of the script.
T label # Error branch, starting from the last line. Once an error or T, t command occurs, it will cause the branch to the command with the label, or to the end of the script.
w file # Write and append the template block to the end of file.
W file # Write and append the first line of the template block to the end of file.
! # Indicates that the following commands will take effect on all unselected lines.
= # Print the current line number.
# # Extend comments until the next newline character.

sed replacement tag

g # means full replacement within the line.
p # means print line.
w # means writing lines to a file.
x # means swapping the text in the template block and the text in the buffer.
y # means translating one character into another character (but not used in regular expressions)
\1 # Substring matching tag
& # Matched string tag

sed metacharacter set

^ # Match the beginning of the line, such as: /^sed/ matches all lines starting with sed.
$ # Matches the end of a line, such as: /sed$/ matches all lines ending with sed.
. # Matches any character that is not a newline character, such as: /s.d/ matches s followed by any character, and finally d.
* # Match 0 or more characters, such as: /*sed/ Matches all lines where the template is one or more spaces followed by sed.
[] # Matches characters within a specified range, such as /[sS]ed/ matches sed and Sed.
[^] # Matches a character that is not within the specified range, such as: /[^A-RT-Z]ed/ Matches lines starting with a letter that does not contain A-R and T-Z, followed by ed.
\(..\) # Match substrings and save matching characters, such as s/\(love\)able/\1rs, loveable is replaced by lovers.
& # Save search characters to replace other characters, such as s/love/ **&** /, love becomes **love**.
\< # Matches the beginning of a word, such as:/\<love/ Matches lines containing words starting with love.
\> # Matches the end of a word, such as /love\>/ Matches lines containing words ending with love.
x\{m\} # Repeat character x, m times, such as: /0\{5\}/ matches lines containing 5 0s.
x\{m,\} # Repeat the character x at least m times, such as: /0\{5,\}/ matches lines with at least 5 0s.
x\{m,n\} # Repeat the character x, at least m times and no more than n times, such as: /0\{5,10\}/ matches lines with 5 to 10 zeros.

sed usage example

Replacement operation: s command

Replace strings in text:

sed 's/book/books/' file

-n option and p command are used together to print only those lines where substitutions occur:

sed -n 's/test/TEST/p' file

Edit the file directly with option -i, which will replace all books matching each line in the file file with books:

sed -i 's/book/books/g' file

Comprehensive replacement of mark g

Using the suffix /g tag replaces all matches in each line:

sed 's/book/books/g' file

When you need to replace from the Nth match, you can use /Ng:

echo sksksksksksk | sed 's/sk/SK/2g'
skSKSKSKSKSK

echo sksksksksksk | sed 's/sk/SK/3g'
skskSKSKSKSK

echo sksksksksksk | sed 's/sk/SK/4g'
skskskSKSKSK

Delimiter

The character / in the above command is used as a delimiter in sed, and any delimiter can also be used:

sed 's:test:TEXT:g'
sed 's|test|TEXT|g'

When delimiters appear inside a style, they need to be escaped:

sed 's/\/bin/\/usr\/local\/bin/g'

Delete operation: d command

Remove blank lines:

sed '/^$/d' file

Delete line 2 of the file:

sed '2d' file

Delete all lines from line 2 to the end of the file:

sed '2,$d' file

Delete the last line of the file:

sed '$d' file

Delete all lines starting with test in the file:

sed '/^test/'d file

Matched string tag &

The regular expression \w+ matches each word, replacing it with [&], which corresponds to the previously matched word:

echo this is a test line | sed 's/\w\+/[&]/g'
[this] [is] [a] [test] [line]

All lines starting with 192.168.0.1 will be replaced with itself plus localhost:

sed 's/^192.168.0.1/&localhost/' file
192.168.0.1localhost

Substring matching tag\1

Matches part of a given pattern:

echo this is digit 7 in a number | sed 's/digit \([0-9]\)/\1/'
this is 7 in a number

The digit 7 in the command was replaced with 7. The substring matched by the pattern is 7, (..) is used to match the substring, and the first matched substring is marked as \1, and so on for the second matched result. It is \2, for example:

echo aaa BBB | sed 's/\([a-z]\+\) \([A-Z]\+\)/\2 \1/'
BBB aaa

love is marked as 1, all loveables will be replaced with lovers, and printed out:

sed -n 's/\(love\)able/\1rs/p' file

Get the ip by replacing:

ifconfig ens32 | sed -n '/inet /p' | sed 's/inet \([0-9.]\+\).*/\1/'
192.168.75.126

Case conversion U/L

\u: Convert first letter to uppercase
\U: Convert all to uppercase
\l: Convert the first letter to lowercase
\L: Convert all to lowercase

Convert first letter to uppercase:

[root@node6 ~]# sed 's/^[a-z]\+/\u&/' passwd
Root0:0:root:/root:/bin/bash
Bin1:1:bin:/bin:/sbin/nologin
Daemon2:2:daemon:/sbin:/sbin/nologin
Adm3:4:adm:/var/adm:/sbin/nologin
Lp4:7:lp:/var/spool/lpd:/sbin/nologin
Sync5:0:sync:/sbin:/bin/sync

All matched characters are converted to uppercase:

[root@node6 ~]# sed 's/^[a-z]\+/\U&/' passwd
ROOT0:0:root:/root:/bin/bash
BIN1:1:bin:/bin:/sbin/nologin

Combine multiple expressions

1. Replace multiple strings in text:

sed -e 's/old_string/new_string/g' -e 's/another_old_string/another_new_string/g' file.txt

2. Delete multiple lines in text:

sed -e '1d' -e '/pattern/d' file.txt

3. Insert multiple lines into the text:

sed -e '1i\inserted_line1' -e '2i\inserted_line2' file.txt

Among them, -e means specifying an expression, and multiple expressions are separated by -e. Each expression can be a sed command, such as s, d, i, etc.

Quote

Sed expressions can be quoted using single quotes, but if the expression contains variable strings, double quotes are required.

test=hello
echo hello WORLD | sed "s/$test/HELLO"
HELLO WORLD

Range of selected rows:, (comma)

All lines within the range determined by templates test and check are printed:

sed -n '/test/,/check/p' file

Print all lines starting from line 5 to the first line containing test:

sed -n '5,/^test/p' file

For the lines between template test and west, replace the end of each line with the string aaa bbb:

sed '/test/,/west/s/$/aaa bbb/' file

Multi-point editing: e command

The -e option allows multiple commands to be executed on the same line:

sed -e '1,5d' -e 's/test/check/' file

The first command of the sed expression above deletes lines 1 to 5, and the second command replaces test with check. The order in which commands are executed affects the results. If both commands are substitution commands, then the first substitution command will affect the results of the second substitution command.

The equivalent command to -e is --expression:

sed --expression='s/test/check/' --expression='/love/d' file

Read from file: r command

The contents of file are read in and displayed after the line matching test. If multiple lines are matched, the contents of file will be displayed below all matching lines:

sed '/test/r file' filename

Write to file: w command

In the example all lines containing test are written to file:

sed -n '/test/w file' example

Append (below the line): a\command

Append this is a test line to the line starting with test:

sed '/^test/a\this is a test line' file

Insert this is a test line after line 2 in the test.conf file:

sed -i '2a\this is a test line' test.conf

Insert (on line): i\ command

Append this is a test line to the front of the line starting with test:

sed '/^test/i\this is a test line' file

Insert this is a test line before line 5 of the test.conf file:

sed -i '5i\this is a test line' test.conf

Replace the specified line: c\ command

Replace the lines starting with root with new content:

[root@node6 ~]# sed '/^root/c this is new line!' passwd
this is new line!
bin1:1:bin:/bin:/sbin/nologin

If you are replacing a specified range, please note that sed does not replace each line, but replaces the entire range as a whole:

[root@node6 ~]# nl passwd | sed '1,5c\ this is dangerous!'
      this is dangerous!
      6 sync5:0:sync:/sbin:/bin/sync
      7 shutdown6:0:shutdown:/sbin:/sbin/shutdown

If you want to uniformly replace the first to fifth lines with the same content, you can use the following command to achieve this:

[root@node5 ~]# sed '1{:a;s/.*/lutxixia/;n;6!ba}' passwd
lutxixia
lutxixia
lutxixia
lutxixia
lutxixia
sync5:0:sync:/sbin:/bin/sync

in:
:a is to set a loop label
s/.*/lutixia/ replaces each matched line with lutixia characters.
n is to read the next line
6! is to exit the loop after reading the sixth line and terminate the operation. If not, continue the loop.
ba means if the sixth line is not reached, jump to a and continue the loop.

Next: n command

If test is matched, move to the next line after the matching line, replace aa in this line with bb, print the line, and continue:

sed '/test/{ n; s/aa/bb/; }' file

Transformation: y command

Convert all abcde in lines 1 to 10 to uppercase. Note that regular expression metacharacters cannot use this command:

sed '1,10y/abcde/ABCDE/' file

Exit: q command

After printing the first 10 lines, exit sed:

sed '10q' file

Until the first match is found, exit sed:

[root@node4 ~]# sed '/nginx/q' nginx.yml
---
- hosts: nginx

Keep and get: h command and G command

When sed processes a file, each line is saved in a temporary buffer called the pattern space. Unless the line is deleted or the output is cancelled, all processed lines will be printed on the screen. Then the pattern space is cleared and a new line is stored for processing.

sed -e '/test/h' -e '$G' file

In this example, after the line matching test is found, it will be stored in the pattern space. The h command will copy it and store it in a special buffer called the holding buffer. The second statement means that when the last line is reached, the G command takes out the line held in the buffer, puts it back into the pattern space, and appends it to the end of the line that now exists in the pattern space. In this example it is appended to the last line. Simply put, any line containing test is copied and appended to the end of the file.

Keep and swap: h command and x command

Swap the pattern space and hold the contents of the buffer. That is, swap the lines containing test and check:

sed -e '/test/h' -e '/check/x' file

Script scriptfile

The sed script is a sed command list. When starting Sed, the -f option is used to guide the script file name. Sed is very picky about the commands entered in the script. There cannot be any whitespace or text at the end of the command. If there are multiple commands on a line, they must be separated by semicolons. Lines starting with # are comment lines and cannot span lines.

sed [options] -f scriptfile file(s)

Print odd or even lines

method 1:

sed -n 'p;n' test.txt #odd lines
sed -n 'n;p' test.txt #even lines

Method 2:

sed -n '1~2p' test.txt #odd lines
sed -n '2~2p' test.txt #even lines

Print the next line of the matched string

grep -A 1 SCC URFILE
sed -n '/SCC/{n;p}' URFILE
awk '/SCC/{getline; print}' URFILE