Familiarizing with Bash

This blog contains some notes for using bash command. I’m writing them here in an “aid-sheet” style. This could also be a concise, elementary tutorial into bash. I’ll group them into following sections.
Survival

• File system navigation (cd, ls, cat, du, df)
• Playing around with variables

Remote working

• Connect to remote computer (ssh, sftp, wget, curl)
• Compress / uncompress files (dpkg, tar, zip)
• Run programs remotely (chmod, nohup, &, &&)

Processing data

• For loop, sed, find, grep, piping

Survival

0. RTFM

Goal	Command
Read the manual of command command_name	man command_name
Find the command for a specific purpose	Search for it online

For any command mentioned below, you can refer to a comprehensive documentation at man <command_name>. This blog can only provide a very short description, for easiness of memorization. The man page is the textbook, while this blog is at most an aid-sheet.

1. Navigating your filesystem

Goal	Command
To navigate to your home folder	cd ~/
To navigate to the previous directory	cd -
Create a folder with name myname	mkdir myname
View all items in current folder	ls
View, and with information including permissions, file sizes	ls -l
View, while making the file sizes human readable	ls -lh
View the size of the current folder	du
View, while making the file sizes human readable	du -h
Check file system space usage	df
View the content of the file myfile.txt	cat myfile.txt
View the contents of files myfile1.txt … myfile3.txt	cat myfile[1-3].txt
View the contents of all files under current directory matching regular expression myfile*.txt	cat myfile*.txt
Write them to mylog.txt while overwriting this txt file	cat myfile*.txt > mylog.txt
Append them to the end of ~/.bashrc	cat myfile*.txt >> mylog.txt

2. Playing around with variables

Goal	Command
To view the value of variable JAVA_HOME	echo $JAVA_HOME
Evaluate the value of bash_command, and print it out	echo $bash_command
To change the value of variable JAVA_HOME to /usr/local/java	JAVA_HOME=/usr/local/java
To change it at startup (i.e., append to ~/.bashrc or ~/.profile)	echo "JAVA_HOME=/usr/local/java" >> ~/.bashrc
To change it via a script setup_env.sh	source setup_env.sh (Shouldn’t use sh since that creates another shell, which has its own set of environment variables.)

Remote working

3. Working with a remote computer: ssh, sftp, wget, curl

Goal	Command
To establish an ssh connection	ssh <username>@<address> -p <port_number>
To establish an sftp connection to your directory	sftp -P <port_number> <username>@<address>:<directory>
(during an sftp connection) Download txt files with file name starting with test_	get test_*.txt
To download an object with wget from url https://somewhere/something.tar.gz	wget https://somewhere/something.tar.gz
To submit GET request with curl	curl https://somwehre
To submit a POST request with curl	curl --data "name1=value1&name2=value2" https://somewhere

4. Zipping with tar, gzip, zip

Goal	Command
To extract (-x) the .tar.gz file here, verbose (-v)	tar -xzvf something.tar.gz
To create (-c) .tar.gz file	tar -czvf something.tar.gz myfile1.txt
To extract the tar file	tar -xvf something.tar.gz
To create a file into tar file	tar -cvf something.tar myfile1.txt
To unzip the .zip file	unzip something.zip
To zip files matching regexp myfile[1-3].txt into myzip	zip myzip myfile[1-3].txt

5. Run a program

Goal	Command
Make the file tmp.sh runnable	chmod +x tmp.sh
Run command in the current process	./tmp.sh
Run it as a child process of current process (i.e., will be killed when you log out)	./tmp.sh &
Run it, but without it being killed, and write stdout to your_log.out	nohup ./tmp.sh > your_log.out &
Run multiple commands using AND operator (i.e., &&)	command1 && command2
Input prepared stdin data	tmp.sh < prepared_stdin.in

Processing data

For loop

Goal	Command
Loop over an array and print the values	for i in {1,3,5}; do echo $i; done
Loop over a range and print the values	for i in {1..5}; do echo $i; done

grep, pipe, sed, find

Goal	Command
Display all .txt files under current directory	ls | grep *.txt
Display files, but applying multiple grep filters	ls | grep | grep
Count the number of txt files	ls | grep *.txt | wc -l
Display only (-o) the numerical values from the variable output	echo $output | grep -oE [0-9]
Replace, in-place (-i) <regex> to “Content” of the file readme.md	sed -i "s/<regex>/Content/" readme.md
Find a file under current directory by filename	find . -name <regex>
Find content in directory dirname recursively (-R), with line number (-n), case-insensitive (-i) containing CrossEntropy	grep CrossEntropy dirname -iRn