A memo of how to deal with problems and forget about how to use bash and linux when you start learning.
-[Usage memo](#Usage memo) -[Command replacement](#command replacement) -[Time Format](#Time Format) --[Number of files in directory](#Number of files in directory) -[Search by part of file name or directory name](#Search by part of file name or directory name) --[Move files found with find to another directory](Move files found with #find to another directory) -[Count the number of lines](# Count the number of lines) -[Delete line breaks on last line](#Delete line breaks on last line) --[Insert selected text into file](#Insert selected text into file) -[Examine a specific command](#Examine a specific command) -History --Process -[You can't have more than 10 sed backreferences (should)](You can't have more than 10 #sed backreferences) -[On what line the specific string is in (find the location of the csv column)](# Find the location of the csv column where the specific string is) -[Error handling](# error handling) -[/ bin / bash ^ M: Wrong interpreter: No such file or directory](#binbashm-Wrong interpreter-No such file or directory)
-- (back quote) `` and the contents enclosed in $ () are recognized as commands.
--Basically, it is recommended to use $ ().
――Because it can be nested.
[Reference article] bash Tips --Command Substitution and Arithmetic Expansion, Parameter Expansion
--When you want to output in the form of a specific date / time. --Used with the date command.
For example, if you want to use a format like "yyyyMMddHHmmss" (20201025131540)
date "+%Y%m%d%H%M%S"
[Reference article] date command
--Check how many files exist in the directory.
ls -1U | wc -l
[Reference article] [Linux] Find out how many files are under a certain directory
--Find the corresponding file or directory from partial text with the find command.
For example, if you want to search for files containing ".csv" in the work directory
find work -name \*.csv
[Reference article] Search files and more! Detailed summary of find command [Linux command collection]
--I want to move to another directory after finding a file with the find command. --Use xargs.
find -name pf_diff_\*.csv | xargs -I% mv % other_dir
--The -I option of xrags allows subsequent characters to be treated like variables that store the results of the previous command.
--Therefore, % contains the result of find -name pf_diff_ \ *. Csv.
[Reference article] Move (mv) files searched by find at once
--Strictly speaking, wc -l is not the number of lines but the number of line breaks.
--Therefore, if the last line does not include a line break, the number will be one less.
grep -c '' sample.txt
[Reference article] wc command (to count the number of lines with grep) It's not "wc -l file" that counts the number of lines in a file wc -l cannot correctly count the number of lines in a file that does not have a line break at the end of the sentence
--Some files have line breaks on the last line, even though you shouldn't have entered them yourself. --At least in my case, this had to be removed as it interfered with later processing.
For example, the following csv file
test.csv
"name","price","classification"
"pasta","250","carbohydrate"
"cabbage","150","vegetables"
"butter","120","fat"
← This line has only line breaks and no spaces or characters
One line under " butter "," 120 "," fat " is a line break without permission.
The following processing is added to eliminate this.
cat test.csv | head -c -1
You don't have to worry about the cat part, but the important thing is the head -c -1 after the |.
Since it is a process to display other than the last character, the line feed on the last line is ignored as a result.
--Insert some text into a file that already exists. --This time, insert the first line (column) of the csv file (test.csv) at the beginning of another csv file (new.csv).
sed -i "1 i $(head -n 1 test.csv)" new.csv
Or
head -n 1 test.csv | { read v ;sed -i "1 i $v" new.csv; }
-- sed -i" 1 i ○○ " "" "can be single quoted, but use" " when expanding variables. --The -i option in sed is for editing and overwriting files. --If this is missing, the output will be output, but the file has not been overwritten. --The sed i command ( i at "1 i ○○" `) inserts text before the target.
[Reference article] Insert a string on the first line of the file with the sed command Linux: How to edit (in-place) with the sed command Operation of sed a command and i command (insertion and addition of specified character string) [Linux command] Assign the contents received from the pipe (standard input) with bash to the variable
--Use history to look up past commands, but use grep if you want to see only the history of a particular command.
For example, if you want to display the history of the ls command
history | grep ls
--Use ps or ps aux to check the process, but also use grep for specific commands and strings.
For example, when you want to display only about inotify wait
ps aux | grep inotifywait
This is the way I arrived without knowing how to turn it off when I ran inotifywait in the background.
--When I used to handle a csv file with 10 or more columns in my business, I was in trouble because the back references (/ 1 and / 2) were not recognized correctly after 10.
Here's how I thought about it.
cat sample.csv | cut -d ',' -f 10-11 | sed -e "s/\(.*\),\(.*\)/\1,\2/g"
--Cut out the 10th to 11th texts separated by cut -d',' -f 10-11.
--sed -e "s / \ (. * \), \ (. * \) / \ 1, \ 2 / g" refers backward
--Since only the 10th and 11th are cut out by cut, the back references are not \ 10 and \ 11, but can be suppressed to small numbers such as \ 1 and \ 2.
[Reference article] bioinformatics(sed)
――This is also the story of csv that I dealt with in business, the number of columns is lightly over 200, and it was necessary to extract about 20 from them and create another file. ――I thought about the method of line breaks separated by commas ⇒ what line the corresponding column is in.
$ cat sample.csv
First column,Second column,Third column,4th column,5th column
a,b,c,d,e
$ head -n 1 sample.csv | sed -e "s/,/,\n/g" | grep -e "Third column" -n
3:Third column,
In this way, you can search for the column you are looking for.
[Reference article] Output only the line number of the corresponding line with Linux grep (specified character search)
――I was developing with multiple people, and when I ran the shell pulled from git, this display often appeared.
$ ./test.sh
-bash: ./test.sh: /bin/bash^M:Wrong interpreter:There is no such file or directory
So do the same thing as the reference article
sed -i 's/\r//' test.sh
[Reference article] / bin / bash ^ M: bad interpreter: No such file or directory
I'm not sure if this article is the best solution. If you look it up, there may be an easier way.
Recommended Posts