Linux Presenting Data

内容概要: understanding input and output, redirecting output, redirecting input, creating your own direction, list opening file descriptiors, suppressing command output, using temporary files, logging messages…

1. Understanding Input and Output:

File Descriptor: is a non-negative integer, which uniquely identifies open file in a session. Each process is allowed to have up to 9 open file descriptors at a time. The bash shell reserves the first three file descriptors(STDIN=0, STDOUT=1,STDERR=2) for special purpose.
  • STDIN
  • STDOUT
  • STDERR
    1.1 Temporary Redirections
    When you redirect to a file descriptor, you must precede the file descriptor number with an ampersand sign(&).
    1
    2
    3
    4
    5
    6
    # test.sh
    #!/bin/bash

    #默认情况下，标准错误其实也重定向到标准输出
    echo "This is an error!" > &2
    echo "This is normal ouput" > &1
By default, linux directs the STDERR output to STDOUT!
1
2
3
4
5
6
7
8
9
10
11
#运行上述脚本
$ bash test.sh
This is an error!
This is normal ouput

#把标出错误重新定位到err
$ bash test.sh 2>err
This is normal ouput

$ cat err
    1	This is an error!
1.2 Permanent Redirections

It is tedious having to redirect every echo statement. Instead, you can tell the shell to redirect a specify file descriptor for the duration of the script by using the exex command.
exec : starts a new shell, and redirects the STDOUT file descriptor to a file. All output in the script that goes to STDOUT is instead redirected to the file.

1
2
3
4
5
6
7
8
9
10
11
#!/bin/bash

#redirecting all output to a file
exec 1>testout
exec 2>testerr

echo "This is a test of redirecting all output"
echo "from a script to another file"
echo "without having to redirect every individual line"

echo "but this should go to the testerr file" >&2

Once you redirect STDOUT & STDERR, you can’t easily redirect them back to their original location. If you need to switch back and forth with your redirection, then you should learn the following sectoin “Creating Your Own Redirection”

1.3 Redirecting Input In Scripts
1
2
3
4
5
6
7
8
9
10
#!/bin/bash

exec 0< awk.md
count=1

while read line
do
	echo "Line #$count = $line"
	count=$[$count+1] 
done

2. Creating Your own Redirection

Just as with the standard file descriptor, once you assign an alternative file descriptor to a file locaion, that redirection stays permanent until you reassign it.

2.1 Create output file descriptors
1
2
3
4
5
6
7
8
9
10
11
#!/bin/bash

# assign a file descriptor to a file
exec 3>testout

echo "THis should display on the monitor!"
echo "and this should be stored in the file">&3
echo "Then this should be back on the monitor"

#列出打开的文件描述符
lsof -a -p $$ -d 0,1,2,3

运行结果：

1
2
3
4
5
6
7
8
$ bash tttt 
THis should display on the monitor!
Then this should be back on the monitor
COMMAND  PID USER   FD   TYPE DEVICE SIZEs/OFF   NODE NAME
bash    9927  jay    0u   CHR 136,11      0t0     14 /dev/pts/11
bash    9927  jay    1u   CHR 136,11      0t0     14 /dev/pts/11
bash    9927  jay    2u   CHR 136,11      0t0     14 /dev/pts/11
bash    9927  jay    3w   REG    8,5       38 307361 /home/jay/blog/node/source/_posts/testout

2.2 Redirect file descriptors
1
2
3
4
5
6
7
8
9
10
11
12
#!/bin/bash


# redirect FD_3 to FD_1
exec 3>&1
exec 1>testfile

echo "Now FD 3 will go to stdout">&3
echo "but stdout will go to testfile">&1
# now things come back to normal
exec 1>&3
echo "Now things come back to normal"
2.3 Create input file descriptors
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
#!/bin/bash

# FD_5 hold stdin
exec 5<&0
# read date from a file
exec 0<LinuxPresentingData.md

count=1
while read line
do
	echo "Linne #$count : "$line
	count=$[$count+1]
done

exec 0<&5

read -p "Are you done now?" answer
case $answer in
y|Y) echo "Goodbye";;
N|n) ecno "SOrry, this is the end!";;
esac
2.4 Create read-write file descriptors
1
2
3
4
5
6
7
#!/bin/bash

exec 3<>testfile

read line <&3
echo "Read　: $line"
echo "This is a copy for your line">&3
2.5 Close file descriptors
1
2
# This statement closes file descriptor 3, preventing it from beging used any more in your script.
exec 3>&-
2.6 List open file descriptors

With only 9 file descroptors availiable to you, sometimes you have to keep track of file open on the system. Then you have lsof command.
There are plenty of command line parameters and options. The most commonly used are -p, which is used to specify a process ID(PID), and -d, which is used to specify the file descriptor number to display.

例如，显示当前进程所有打开的文件(其中$$表示当前进程ID):
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
# 显示当前进程所有打开的文件($$表示当前进程ID)
$ lsof -p $$
COMMAND  PID USER   FD   TYPE DEVICE SIZE/OFF    NODE NAME
bash    5830  jay  cwd    DIR    8,5     4096  301946 /home/jay
bash    5830  jay  rtd    DIR    8,5     4096       2 /
bash    5830  jay  txt    REG    8,5  1037528  654088 /bin/bash
bash    5830  jay  mem    REG    8,5    47600  408923 /lib/x86_64-linux-gnu/libnss_files-2.23.so
bash    5830  jay  mem    REG    8,5    47648  408472 /lib/x86_64-linux-gnu/libnss_nis-2.23.so
bash    5830  jay  mem    REG    8,5    93128  408909 /lib/x86_64-linux-gnu/libnsl-2.23.so
bash    5830  jay  mem    REG    8,5    35688  408910 /lib/x86_64-linux-gnu/libnss_compat-2.23.so
bash    5830  jay  mem    REG    8,5  4763056 1177526 /usr/lib/locale/locale-archive
bash    5830  jay  mem    REG    8,5  1864888  408618 /lib/x86_64-linux-gnu/libc-2.23.so
bash    5830  jay  mem    REG    8,5    14608  408476 /lib/x86_64-linux-gnu/libdl-2.23.so
bash    5830  jay  mem    REG    8,5   167240  397839 /lib/x86_64-linux-gnu/libtinfo.so.5.9
bash    5830  jay  mem    REG    8,5   162632  408254 /lib/x86_64-linux-gnu/ld-2.23.so
bash    5830  jay  mem    REG    8,5   102200 1706171 /usr/share/locale-langpack/zh_CN/LC_MESSAGES/bash.mo
bash    5830  jay  mem    REG    8,5    26258 1443216 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache
bash    5830  jay    0u   CHR  136,2      0t0       5 /dev/pts/2
bash    5830  jay    1u   CHR  136,2      0t0       5 /dev/pts/2
bash    5830  jay    2u   CHR  136,2      0t0       5 /dev/pts/2
bash    5830  jay  255u   CHR  136,2      0t0       5 /dev/pts/2
例如，显示FD 0,1,2所关联的文件(这里会包含所有进程的)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
$ lsof -d 0,1,2
COMMAND     PID USER   FD   TYPE             DEVICE SIZE/OFF   NODE NAME
systemd    1442  jay    0r   CHR                1,3      0t0      6 /dev/null
systemd    1442  jay    1u  unix 0x0000000000000000      0t0  20246 type=STREAM
systemd    1442  jay    2u  unix 0x0000000000000000      0t0  20246 type=STREAM
upstart    1452  jay    0r   CHR                1,3      0t0      6 /dev/null
upstart    1452  jay    1w   CHR                1,3      0t0      6 /dev/null
upstart    1452  jay    2w   REG                8,5        0 262841 /home/jay/.xsession-errors
upstart-u  1525  jay    0u   CHR                1,3      0t0      6 /dev/null
upstart-u  1525  jay    1u   CHR                1,3      0t0      6 /dev/null
upstart-u  1525  jay    2u   CHR                1,3      0t0      6 /dev/null
dbus-daem  1533  jay    0u   CHR                1,3      0t0      6 /dev/null
dbus-daem  1533  jay    1u   CHR                1,3      0t0      6 /dev/null
dbus-daem  1533  jay    2u   CHR              136,0      0t0      3 /dev/pts/0
window-st  1545  jay    0u   CHR                1,3      0t0      6 /dev/null
例如，显示当前进程的，FD为0,1,2所关联的文件(使用-a表示AND关系)
1
2
3
4
5
$ of -a -p $$ -d 0,1,2
COMMAND  PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
bash    5830  jay    0u   CHR  136,2      0t0    5 /dev/pts/2
bash    5830  jay    1u   CHR  136,2      0t0    5 /dev/pts/2
bash    5830  jay    2u   CHR  136,2      0t0    5 /dev/pts/2
2.7 Suppressing command output
1
$ ls -al > /dev/null

3. Using Temporary Files

The linux system contains special directory location reserved for temporary files: /tmp
create a local temporary file, the command will replaces the six X’s with a six-character code to ensure the filename is unique in the directory

1
2
3
4
5
6
7
$ mktemp test.XXXXX

#create a temporary file in **/tmp**
$ mktemp -t test.XXXXXX

#create a temporary directory
$ mkdtemp -d dir.XXXXXX

举例说明
1
2
3
4
5
6
7
8
#!/bin/bash

name=`mktemp -t -d dir.XXXXX`
cd $name

file=`mktemp testfile.XXXXX`

ls -al >> $file

4. Logging Messages

The tee command sends data from STDIN to two destinations at the same time.

1
2
3
4
name=`mktemp log.XXXXXX`

# -a indicates append
$ date | tee -a $name