Basic BSD/Linux/Unix commands

This module is designed to show you basic commands for everyday terminal usage.

Introduction

Typically if a command returns nothing it has performed it’s operation correctly—this is a unix philosophy—however, some commands are expected to return an output for the user or the verbose option is turned on. Some terminals include a symbol to indicate that commands can be typed on the prompt; I will be using $ to denote this.

the directory layout for all commands assumes the following files and directories: data/experimental.csv, script.py, readme.md

Command	Description
`man`	display manual pages
`ls`	list files and directories
`cd`	change directory
`cp`	copy files and directories
`mv`	move/rename a file or directory
`mkdir`	make directory
`pwd`	present working directory
`rm`	remove file(s)
`rmdir`	remove directory
`tar`	create archive of files and folders
`cat`	concatenate files
`echo`	write arguments to the standard output
`head`	display the first few lines of a file
`tail`	display the last few lines of a file

man - display manual pages

This is a crucial command to learn about options about command line programs and utilities. It describes how to use the command and options that can be passed to it and sometimes examples on how to use it. To learn more about the man command type

$ man man

Pressing q will quit the man page, where up, down, PgUp and PgDn for navigating. The man pages can be searched by typing /<word or phrase you want>, with n and N to find next and previous result.

ls - list files and directories

Required for looking inside a directory. Calling ls with no options or arguments displays the content of the current directory. Passing any path will display the contents of that path.

$ ls
data       readme.md   script.py

If you do not have syntax highlighting for files or folders this is tricky to read as data is a directory. One solution is to use the option -F.

$ ls -F
data/       readme.md   script.py

Another option that is very useful is -l for listing details and -h for file sizes to be reported in Kilobyte, Megabyte etc…

$ ls -lh
drwxr-xr-x  2 dave  dave   512B Dec  9 20:38 data
-rw-r--r--  1 dave  dave     0B Dec  9 20:36 readme.md
-rw-r--r--  1 dave  dave     0B Dec  9 20:35 script.py

What you see here are the columns: file permissions, number of links, owner, group, size, date last modified and pathname. The file permissions here in the first column describe the type of file, d or -, which is a directory or file respectively. The next 3 groups of 3 characters describe the owner, group and other permissons; with r, w, x meaning read, write and execute respectively. To learn more read the man page:

$ man ls

cd - change directory

This is used to traverse the folder structure of the operating system. Some shortcuts have special meaning such as, . and .., which correspond to the current directory and one parent directory up if possible

$ cd data
$ ls
experimental.csv
$ cd ..
$ ls
data/       readme.md   script.py

Another shortcut is ~ which is the users home directory that expands to /home/user where user is the users name.

cp - copy files and directories

A simple tool to copy one file

$ cp script.py script.bck
$ ls -F
data/            readme.md        script.py        script.py.bck

or a directory recursively -r

$ cp -r data data.old
$ ls  -F
data/            data.old/        readme.md        script.py        script.py.bck

You could pass -v to turn to make cp more verbose

$ cp -vr data data.new
data/experiment.csv -> data.new/data/experiment.csv
$ ls -F
data/            data.new/        readme.md        script.py        script.py.bck

For more information read the man pages.

mv - move/rename a file or directory

When you need to restructure your files and folders

$ mv data files
$ ls -F
files/        readme.md        script.py

mkdir - make directory

When you need to create a directory for files

$ mkdir images
$ ls -F
data/       images/     readme.md   script.py

a useful option -p will create intermediate directories if they don’t exist

$ mkdir -p files/conf
$ ls -F
data/       files/      images/     readme.md   script.py
$ ls -F files
conf/

pwd - present working directory

On the commandline some shells list the current directory within the prompt (Bash or Zsh), however this is extra noise on the terminal, and can easily be found by issuing the command pwd

$ pwd
/home/dave/cli-example

rm - remove file(s)

This is necessary to delete files, however, there is no undo when this is done, i.e., no trash bin exists like on standard desktops (KDE, Gnome, MacOS or Windows, etc…)

$ rm readme.md script.py
$ ls -F
data/

To remove a directory with files inside of it use -rf

$ rm -rf data
$ ls -F
readme.md   script.py

WARNING rm -rf IS A VERY DANGEROUS COMMAND IT CAN CAUSE IRREVERSIBLE DAMAGE, USE WITH CAUTION

rmdir - remove directory

Removes an empty directory

$ rmdir data
rmdir: data: Directory not empty
$ rm data/experiment.csv
$ rmdir data
$ ls -F
script.py   readme.md

tar - create archive of files and folders

Similiar to the zipped files on other operating systems, tar stands for tape archive, originally for old magnetic tape reels. It is now a tool that archives data and files which can be then compressed.

To create (-c) an archive, you must specify the output file (-f) first, followed by a list of files and folders to be archived

$ tar -cf archive.tar data readme.md script.py
$ ls -F
archive.tar  data/        readme.md    script.py

If you want to see what files are being archived, then make it verbose -v

$ tar -cvf archive.tar data/ readme.md  script.py
data
data/experiment.csv
readme.md
script.py
$ ls
archive.tar  data/        readme.md    script.py

You can use external tools to compress the tar archive, such as gzip, bzip2, and xz. Variants of tar have these features built-in, such as gzip -z or bzip2 -j. Convention of the tar command expects you to append the file type for each compression for gzip (.gz) and bzip2 (.bz2). The compression can be compared as well

$ tar -czf archive.tar.gz data/ readme.md  script.py
$ tar -cjf archive.tar.bz2 data/ readme.md script.py
$ ls -lh archive.tar*
-rw-r--r--  1 dave  dave  10.0K Dec 10 12:14 archive.tar
-rw-r--r--  1 dave  dave   208B Dec 10 12:20 archive.tar.bz2
-rw-r--r--  1 dave  dave   191B Dec 10 12:20 archive.tar.gz

Typically bz2 has higher compression than gzip in more realistic cases but gzip is faster. Note that archive.tar is 10 KB almost 5 times as big compared to the compressed tar archives.

The next important feature is extracting -x a tar archive. If the tar has any compression such as gzip or bzip2 then the required feature option must be passed -z or -j

$ mkdir ~/new
$ cd ~/new
$ tar -xzf ~/cli-example/archive.tar.gz
$ pwd
/home/dave/new
$ ls
data/        readme.md    script.py

echo, cat, head and tail

Some other useful commands are:

cat - concanate text files and print contents to terminal
echo - print characters to terminal, e.g., echo "Hello World!"
head - display the first few lines of a file
tail - display the last few lines of a file

Read the man pages and experiment on how they work. These are useful for redirecting output into the input of other programs, see a future module.