[Linux] Command / Knowledge

Excerpts as memorandums of things that you think you can use after learning linux commands and things that you want to remember. We plan to update it each time.

index

-Command -less (view the contents of the file) -Scroll / Search -touch (create an empty file) -[diff (diff comparison)](#diff difference comparison) -[sed (text replacement process)](#sed text replacement process) -[Refer backward](#Refer backward) -[Delete the line containing a specific character string](#Delete the line containing a specific character string) -[Save the replaced file by overwriting](# Save the replaced file by overwriting) -[cut (cut out text)](#cut cut out text) -[Specify range & change delimiter](#Change range specification delimiter) -Knowledge -> (Redirect) -[File Descriptor (I / O number)](# File descriptor I / O number)

• /dev/null --Permissions -[Grant execution right](# Grant execution right)

command

less --Used when viewing the contents of a long file. --You can scroll and search when viewing the contents of a file.

scroll

Search

touch --Originally a command to update / change the time stamp of a file. --If the file does not exist, create an empty file. --The time stamp is updated when the file exists, so you don't have to worry about accidentally overwriting the file with the contents. ** **

diff (difference comparison)

--A command to display the differences between files.

$ cat sample1.csv
Column 1,Column 2,Column 3
1,2,3
a,b,c
Ah,I,U
$ cat sample2.csv
Column 1,Column 2,Column 3
1,2,3
d,e,f
Ah,I,U
$ diff sample1.csv sample2.csv
3c3
< a,b,c
---
> d,e,f

--Try to display the difference from the above csv file and add the original header (column line) to the beginning.

$ diff sample1.csv sample2.csv | grep -E "^>" | sed -e 's/^> //g' | sed "1 i `head -n 1 sample1.csv`"
Column 1,Column 2,Column 3
d,e,f

sed (text replacement process)

--A scripting language for editing text files. --Specify the -e option for replacement processing. (Optional)

$ cat sample2.csv
Column 1,Column 2,Column 3
1,2,3
d,e,f
Ah,I,U
$ sed -e 's/d,e,f/a,b,c/g' sample2.csv
Column 1,Column 2,Column 3
1,2,3
a,b,c
Ah,I,U

#In the above writing style, sample2.csv has not been overwritten
$ cat sample2.csv
Column 1,Column 2,Column 3
1,2,3
d,e,f
Ah,I,U

[Reference article] bioinformatics(sed) Notes on sed command

Back reference

--When you want to use the character string before replacement in the part after replacement. --Try to extract a specific column from csv.

$ cat sample.csv
name,age,address,job
Sato,30,Tokyo,teacher
Kato,18,Kanagawa Prefecture,student
Ikarashi,25,Saitama,Doctor
$ sed -e "s/\(.*\),\(.*\),\(.*\),\(.*\)/\1,\3,\4/g" sample.csv
name,address,job
Sato,Tokyo,teacher
Kato,Kanagawa Prefecture,student
Ikarashi,Saitama,Doctor

Delete lines that contain a specific string

$ sed '/^Sato/d' sample.csv
name,age,address,job
Kato,18,Kanagawa Prefecture,student
Ikarashi,25,Saitama,Doctor

Deleted the line containing "Sato" at the beginning.

[Reference article] How to delete the line specified by sed (with sample code)

Overwrite and save the replaced file

--You can overwrite and save with the i option.

Consider the case of the deletion described above.

$ sed '/^Sato/d' sample.csv
name,age,address,job
Kato,18,Kanagawa Prefecture,student
Ikarashi,25,Saitama,Doctor
#This is not overwritten
$ cat sample.csv
name,age,address,job
Sato,30,Tokyo,teacher
Kato,18,Kanagawa Prefecture,student
Ikarashi,25,Saitama,Doctor
$ sed -i '/^Sato/d' sample.csv
$ cat sample.csv
name,age,address,job
Kato,18,Kanagawa Prefecture,student
Ikarashi,25,Saitama,Doctor

[Reference article] Replace with sed and overwrite the file

cut (cut out text)

--This time, only the part specified in fixed length or field unit is cut out from the line of text. --The default delimiter is tab.

Specify range & change delimiter

cut -d ',' -f 3-8 sample.csv

--Change delimiter to comma with -d',' --Cut out the 3rd to 8th texts with -f 3-8 --If you set -f 3,8, only the 3rd and 8th will be cut out.

[Reference article] [Cut] command-cut out from a line in fixed length or field units

knowledge

redirect

--Ability to change the input / output destination.

File descriptor (input / output number)

--The standard I / O is numbered as a file descriptor, and these numbers are often used for redirection.

Input redirection

Ability to input from a file

$ cat < /etc/hosts
127.0.0.1   localhost
::1         localhost

Output redirection

Save the command execution result to a file without displaying it on the screen.

$ ls work
abc.txt  hoge  test
$ ls work > output.txt
$ cat output.txt
abc.txt
hoge
test

--The form of command> output file. --The file is automatically created even if you execute it when there is no output file.

Error output redirection

Save the error to a file without displaying it on the screen.

$ ls hoge
ls:Unable to access hoge:There is no such file or directory
$ ls hoge 2> error.txt
$ cat error.txt 
ls:Unable to access hoge:There is no such file or directory

--The form of command 2> output file.

Combine standard output and error output

$ ls / hoge > output.txt 2>&1
$ cat output.txt 
ls:Unable to access hoge:There is no such file or directory
/:
bin
boot
dev
etc
home

--The form command 2> output file 2> & 1.

Addition by redirect

--> will be overwritten if the same file is specified. -->> is added instead of overwritten.

$ echo one > output.txt
$ cat output.txt 
one
$ echo two >> output.txt
$ cat output.txt 
one
two

[Reference article] You can understand everything in 5 minutes! Linux Redirection Usage and Types Summary

/dev/null --Nothing is returned when you specify it as the input destination, and even if you specify it as the output destination, the data disappears and nothing is displayed. ――I think the analogy of the first reference article is easy to understand.

Black hole files. It may inhale but not exhale.

It is often used when you do not want to display the output

hoge.sh > /dev/null

In this case, the error output will be displayed normally, so if you want to discard the error output as well, write as follows. (For bash)

hoge.sh > /dev/null 2 > &1

[Reference article] [What is / dev / null | IT terminology dictionary that makes you feel like "I understand" but "I don't understand" [https://wa3.i-3-i.info/word11733.html) [Linux] Two purposes for engineers to use / dev / null Linux, what is "/ dev / null"?

permission

--Access rights / privileges for folders and files. --There are 3 execution rights and 3 ownership rights. ――In other words, it defines who has "what authority".

For example, when the shell says "you don't have permission" like this:

$ cat test.sh 
echo "abcde"
$ ./test.sh
-bash: ./test.sh:No permission
$ ls -l test.sh 
-rw-r--r--.1 root root 0 October 6 03:04 test.sh

As you can see, there is no execution right (x) for the shell.

Grant of execution right

If you do not have the execution right as described above, add it with the chmod command.

$ chmod u+x test.sh
$ ./test.sh
abcde
$ ls -l test.sh 
-rwxr--r--.1 root root 13 October 6 03:53 test.sh

[Reference article] Permissions | IT Glossary that makes you feel like you understand even if you don't understand it About permissions [File permissions and attributes](https://wiki.archlinux.jp/index.php/%E3%83%95%E3%82%A1%E3%82%A4%E3%83%AB%E3%81] % AE% E3% 83% 91% E3% 83% BC% E3% 83% 9F% E3% 83% 83% E3% 82% B7% E3% 83% A7% E3% 83% B3% E3% 81% A8 % E5% B1% 9E% E6% 80% A7)