xargs

A filter for passing arguments to other commands

Supplementary instructions

The xargs command is a filter for passing parameters to other commands and a tool for combining multiple commands. It is good at converting standard input data into command line arguments. xargs can handle pipes or stdin and convert them into command arguments for a specific command. xargs can also convert single-line or multi-line text input into other formats, such as multi-line to single-line, single-line to multi-line. The default command for xargs is echo, and spaces are the default delimiters. This means that the input piped to xargs will contain newlines and whitespace, but through xargs processing, newlines and whitespace will be replaced by spaces. xargs is one of the important components for building single-line commands.

xargs command usage

xargs is used as a replacement tool to read input data and reformat it before outputting it.

Define a test file with multiple lines of text data:

cat test.txt

a b c d e f g
h i j k l m n
o pq
r s t
u v w x y z

Multi-line input and single-line output:

cat test.txt | xargs

a b c d e f g h i j k l m n o p q r s t u v w x y z

Use -n for multi-line output

-n option Multi-line output:

cat test.txt | xargs -n3

a b c
d e f
g h i
j k l
m n o
pqr
s tu
v x
z

Use -d to split input

-d option allows you to customize a delimiter:

echo "nameXnameXnameXname" | xargs -dX

name name name name

Use with the -n option:

echo "nameXnameXnameXname" | xargs -dX -n2

name name
name name

Read stdin

Read stdin and pass formatted parameters to the command

Assume a command is sk.sh and a file holding parameters arg.txt:

#!/bin/bash
#sk.sh command content, print out all parameters.

echo $*

arg.txt file content:

cat arg.txt

aaa
bbb
ccc

Combined with -I option

An option -I of xargs, use -I to specify a replacement string {}. This string will be replaced when xargs expands. When -I is used in combination with xargs, each parameter command will be executed once. :

cat arg.txt | xargs -I {} ./sk.sh -p {} -l

-p aaa -l
-p bbb -l
-p ccc -l

Copy all image files to the /data/images directory:

ls *.jpg | xargs -n1 -I{} cp {} /data/images

Used in combination with find command

xargs used in combination with find

When using rm to delete too many files, you may get an error message: /bin/rm Argument list too long. Use xargs to avoid this problem:

find . -type f -name "*.log" -print0 | xargs -0 rm -f

xargs -0 uses \0 as delimiter.

Count the number of lines of all php files in a source code directory:

find . -type f -name "*.php" -print0 | xargs -0 wc -l

Find all jpg files and compress them:

find . -type f -name "*.jpg" -print | xargs tar -czvf images.tar.gz

Print out the executed command

Combined with the -t option you can print out the commands executed by xargs

 ls | xargs -t -I{} echo {}

Will output the file list in the current directory and the executed echo command

Use the -p option to confirm the executed command

The -p option will pop up a confirmation when executing each command. You can use the -p parameter when you need to confirm each operation very accurately. For example, search for the .log file in the current directory and delete it every time. need confirmation:

 find . -maxdepth 1 -name "*.log" | xargs -p -I{} rm {}

Execute multiple commands

Use the -I option to have xargs execute multiple commands

 cat foo.txt
 one
 two
 three

 cat foo.txt | xargs -I % sh -c 'echo %; mkdir %'
 one
 two
 three

 ls
 one two three

other apps

xargs other applications

If you have a file that contains a lot of URLs that you wish to download, you can use xargs to download them all:

cat url-list.txt | xargs wget -c

Subshells (Subshells)

Running a shell script starts another command interpreter, as if your commands were interpreted at the command line prompt, similar to a series of commands in a batch file. Each shell script effectively runs in a child process of the parent shell. The parent shell is the process that gives you command prompts in a controlling terminal or in an xterm window.

cmd1 | ( cmd2; cmd3; cmd4 ) | cmd5

If cmd2 is cd /, then the working directory of the sub-shell will be changed. This change is only limited to the inside of the sub-shell. cmd5 is completely unaware of the changes in the working directory. A subshell is a sequence of commands embedded within parentheses (), and the variables defined inside the subshell are local variables.

A subshell can be used to set temporary environment variables for a set of commands:

COMMAND1
COMMAND2
COMMAND3
(
   IFS=:
   PATH=/bin
   unsetTERMINFO
   set -C
   shift 5
   COMMAND4
   COMMAND5
   exit 3 # Just exit from the subshell.
)
# The parent shell is not affected and variable values are not changed.
COMMAND6
COMMAND7