Lab No. 2: The File System and Working with Files

After completing this lab, students will be able to:

  • Inspect and navigate the linux filesystem
  • Create, remove and alter files and directories

The File System

A file system is a method for organizing and storing data. Without a file sytem, the operating sytem (and the users) would have a hard time locating files that are needed to perform tasks. The file system consists of files and their attributes and relationships between files.

The Linux file system is composed of files and directories. A file is a collection of data that is logically related, and it is the smallest unit of storage in the Unix file system. Directories are also file, but they are special since they may contain other files and other directories (a physical analogy isa folder or envelope where you can use to organize and store papers, or other envelopes). In this way, the files sytem is organized in a hierarchical structure that resembles a tree, typically represented upside down (in the same fashion as a family tree). The top directory in a Linux file system is called the root directory, or simply, the root, and it is represented by the / (slash) character.

../../_images/fhs.svg

The standard hierarchy of Linux Filesystems is published by the Linux Foundation as a specification (available under https://refspecs.linuxfoundation.org/fhs.shtml). The following directories are required by the standard (Refer to Table 3-4 in TLCL for more details)

   
Directory Description
/ root directory
/bin Essential command that must be present for boot/recovery
/boot Contains the boot loader and kernel files
/dev Device files.
/etc System configuration
/home Users directories and files
/lib Essential shared libraries and kernel modules
/media Mount point for removable media (USB Drives, CD-ROMS)
/mnt Mount point for mounting a filesystem manually
/opt Add-on application software packages (“optional”)
/proc Process information pseudo-file system (contains runtime information).
/root Home directory for the “root” user.
/run Data required by running processes
/sbin System binaries.
/tmp Temporary files
/usr All the programs and support files to be used by regular users
/var Variable data (logs, user mail, spool files, etc).

File system navigation

In Linux all files must have a name. The following rules apply when naming a file:

  1. File names are case sensitive
  2. File names can only contain uppercase and lowercase characters, numbers, the dot and the underscore symbols.
  3. File names are limited to 255 characters

To be able to work with a file, we need to know it’s file name, but we also need to know the exact location of a file within the file system hierarchy. A pathname is the trail through the directory hierarchy to a file or directory. A pathname can be:

  1. absolute when it traces a path from the root directory (/). This is complete and unambiguous. Absolute pathnames always begin with /.
  2. relative when it traces a path from the user’s current position or location on the filesystem (known as the current working directory). The beginning (left-most) part of a relative path name is either an ordinary file, a directory or a reference to the parent of the current directory.

The following characters have special meaning when used in pathname:

  1. A single dot .: refers to the current directory
  2. A double dot ..: which refers to a parent directory.
  3. The tilde character (~) which is expanded to the current user’s home directoriy.

There are several commands that help you to navigate the file system.

Unix systems have the concept of the working directory (also referred as current directory), which determines the directory within the tree that your session is currently associated with. The pwd command prints your current directory.

[user@blue ~]$ pwd
/home/student/user

When you login into Unix systems, your current directory is set to your home directory by default. The ~ in the prompt means that your current working directory is your home directory. As mentioned before, home directories for most users are typically located under /home (On MacOS, home directories are located in /Users.)

You can change your current directory with the cd command, which takes as an argument the pathname of the desired directory. If no argument is provided, then cd will change the working directory to the user’s home directory.

Execute the following command to change your current working directory to the root directory:

[user@blue ~]$ cd /
[user@blue /]$

The cd command does not provide any output. However, you can see that the prompt changed to / instead of ~. Let’s see what is the output of the pwd command:

[user@blue /]$ pwd
/
[user@blue /]$

Let’s check the content of the root directory in blue, so we can compare it with the standard list of directories under root. In order to do this, we can use the ls commands which lists the contents of a directory.

[user@blue /]$ ls
bin   boot  etc   kirby   lib    lost+found  media  netatalk  proc  run   srv  target   tmp  var
blue  dev   home  kodiak  lib64  mamer       mnt    opt       root  sbin  sys  thekeep  usr

When ls is executed without any arguments, it lists the contents of the current directory. You can also specify a directory to lists its contents without having to change directory. Let’s check the contents of the /usr directory:

[user@blue /]$ ls /usr
arm64-linux-gnu    avr            etc               include  libexec     man                sbin   tmp
arm-linux-gnu      bin            games             lib      local       powerpc-linux-gnu  share  x86_64-w64-mingw32
arm-linux-gnueabi  @DATADIRNAME@  i686-w64-mingw32  lib64    lost+found  s390-linux-gnu     src    x86-linux-gnu

Sometimes it is useful to have a listing where each item is listed on it’s one line. Let’s see how that looks:

[user@blue /]$ ls -l /usr
total 604
drwxr-xr-x.   3 root root   4096 Jul  2  2018 arm64-linux-gnu
drwxr-xr-x.   4 root root   4096 Jul  2  2018 arm-linux-gnu
drwxr-xr-x    3 root root   4096 Sep 19  2018 arm-linux-gnueabi
drwxr-xr-x.   6 root root   4096 Jul 12  2018 avr
dr-xr-xr-x.   2 root root 143360 Nov  4 11:46 bin
drwxr-xr-x    3 root root   4096 Aug  7  2018 @DATADIRNAME@
drwxr-xr-x    2 root root   4096 Dec 26  2018 etc
drwxr-xr-x.   2 root root   4096 Aug  2  2017 games
drwxr-xr-x    4 root root   4096 Aug  9  2018 i686-w64-mingw32
drwxr-xr-x. 333 root root  20480 Aug  7 12:00 include
dr-xr-xr-x.  82 root root  20480 Mar 15  2019 lib
dr-xr-xr-x. 382 root root 270336 Aug  7 12:00 lib64
drwxr-xr-x.  88 root root  20480 May  7  2019 libexec
drwxr-xr-x.  16 root root   4096 Dec 13 11:14 local
drwx------.   2 root root  16384 Jun 26  2018 lost+found
drwxr-xr-x    3 root root   4096 Feb 22  2019 man
drwxr-xr-x.   3 root root   4096 Jul  2  2018 powerpc-linux-gnu
drwxr-xr-x.   3 root root   4096 Jul  2  2018 s390-linux-gnu
dr-xr-xr-x.   2 root root  36864 Aug  7 11:28 sbin
drwxr-xr-x. 691 root root  20480 May 23  2019 share
drwxr-xr-x.   8 root root   4096 Aug  7 11:29 src
lrwxrwxrwx.   1 root root     10 Aug  2  2017 tmp -> ../var/tmp
drwxr-xr-x    3 root root   4096 Aug  9  2018 x86_64-w64-mingw32
drwxr-xr-x.   3 root root   4096 Jul  2  2018 x86-linux-gnu

Notice that adding the -l (which stands for long listing format) changes the behavior of the command. (There is a lot of useful information packed in the output of that command, we will learn about those in later exercises) As we saw in Lab 01, these additional instructions are called options and almost every command allows users to specify different options to control the behavior of the command. As a refresher from Lab 01, let’s use the man utility (which opens the manual pages, or normally referred as the man pages) for the ls command:

[user@blue /]$ man ls
LS(1)                                                  User Commands                                                 LS(1)

NAME
       ls - list directory contents

SYNOPSIS
       ls [OPTION]... [FILE]...

DESCRIPTION
       List information about the FILEs (the current directory by default).  Sort entries alphabetically if none of -cftu‐
       vSUX nor --sort is specified.

       Mandatory arguments to long options are mandatory for short options too.

       -a, --all
              do not ignore entries starting with .

Scroll down the man page for the ls command and find the -l option. Also, review the description of the -1 and the -s options. Exit by pressing q. Command options can usually be combined. Let’s see this in action by invoking ls with the -1s option:

[user@blue /]$ ls /usr -s1
total 604
  4 arm64-linux-gnu
  4 arm-linux-gnu
  4 arm-linux-gnueabi
  4 avr
144 bin
  4 @DATADIRNAME@
  4 etc
  4 games
  4 i686-w64-mingw32
 20 include
 20 lib
268 lib64
 20 libexec
  4 local
 16 lost+found
  4 man
  4 powerpc-linux-gnu
  4 s390-linux-gnu
 36 sbin
 20 share
  4 src
  0 tmp
  4 x86_64-w64-mingw32
  4 x86-linux-gnu

Let’s try now changing to the /usr/local/bin directory:

[user@blue /]$ cd /usr/local/bin
[user@blue bin]$ ls
istatserver  n  node  npm  npx  update_rubygems

If we wanted to change directory to /usr/local which is the parent directory of /usr/local/bin we could change directory by using the absolute path, which is what we have done so far. However, we can use the special characters that we mentioned before. The parent directory can be referenced by using the .. (dot-dot) character:

[user@blue bin]$ cd ..
[user@blue local]$ pwd
/usr/local

Suppose we wanted to look at the contents of the /usr/local/bin directory that we just visited. We could again use the absolute path. But, since we are on the parent (/usr/local) it is a lot easier to use a relative path:

[user@blue local]$ ls bin
istatserver  n  node  npm  npx  update_rubygems

An alternative, more explicit way to refer to the current directory is using the . (dot) character:

[user@blue local]$ ls ./bin
istatserver  n  node  npm  npx  update_rubygems

In this case, the . character does not make a difference. You will learn later (when we talk about wildcards) how the . character can become quite useful.

Let’s change the directory back to your home directory, using relative paths (replace user with your username)

[user@blue local]$ cd ../../home/student/user
[user@blue ~]$

You can also use the - option of the cd command to change directory back to the previous working directory:

[user@blue ~]$ cd -
[user@blue local]$ pwd
/usr/local

And if you execute it again, it will take you back to your previous working directory:

[user@blue local]$ cd -
[user@blue ~]$

Creating directories and files

The mkdir utility creates new directories. It requires as arguments the pathnames of the directories to be created. The following example shows the commands required to generate the directory structure shown below (the shown output assumes that your home directory is empty. If that is not the case, then existing files will show in the output of the ls command.)

../../_images/cars.svg
[user@blue ~]$ ls
[user@blue ~]$ mkdir vehicles
[user@blue ~]$ ls
vehicles
[user@blue ~]$ mkdir -p vehicles/cars/sedan
[user@blue ~]$ ls vehicles/cars
sedan
[user@blue ~]$ mkdir -p vehicles/cars/pickup vehicles/cars/suv
[user@blue ~]$ ls vehicles/cars
pickup  sedan  suv

Notice that we did not create the cars directory directly, and instead we used the -p option to instruct the mkdir command to make any non-existing intermeditate directories. Note also that you can create more than one directory at a time.

The tree command is useful to have a directory listing in a tree like structure:

[user@blue ~]$ tree vehicles
vehicles
└── cars
    ├── pickup
    ├── sedan
    └── suv

 4 directories, 0 files

To remove directories you can use the rmdir command. Just as mkdir, it requires the pathnames of the directories to be removed. However, rmdir has the limitation that it only works with empty directories. To remove directories that are not empty, you can use the rm -r command.

Let’s remove the ./vehicles/cars/suv directory:

[user@blue ~]$ rmdir vehicles/cars/suv
[user@blue ~]$ rm -r vehicles/cars/pickup
[user@blue ~]$ tree vehicles
vehicles
└── cars
    └── sedan

2 directories, 0 files

Moving files

The mv command can be used for two purposes: rename a file/directory, or move the file/directory to another path. Let’s rename the sedan directory to coupe:

[user@blue ~]$ mv vehicles/cars/sedan vehicles/cars/coupe
[user@blue ~]$ tree vehicles
vehicles
└── cars
    └── coupe

 2 directories, 0 files

Noticed that the first argument is the source file or directory and the second is the destination directory or file. In this case, since the destination did not exist, then it perfoms a “rename”

To see how the mv command can be used to move files, let’s recreate the sedan directory and and empty file under it:

[user@blue ~]$ mkdir vehicles/cars/sedan
[user@blue ~]$ touch vehicles/cars/sedan/shelby
[user@blue ~]$ tree vehicles
vehicles
└── cars
    ├── coupe
    └── sedan
        └── shelby

3 directories, 1 file

Let’s move shelby from the sedan directory to the coupe directory (where it belongs, obviously).

[user@blue ~]$ mv vehicles/cars/sedan/shelby vehicles/cars/coupe
[user@blue ~]$ tree vehicles
vehicles/
└── cars
    ├── coupe
    │   └── shelby
    └── sedan

3 directories, 1 file

Notice that in this case, the destination exists, and it is a directory, so the result of the command is that the file is move to the destination directory.

Copying files

The cp command is used to copy files or directories. In the following example, a file is created by redirecting the output of the echo command into a file called original. The cat command reads that file and prints that on the screen, this is done as a verification step. original is then copied into a file called clone. Notice how the contents of clone are the same as the contents of original.

[user@blue ~]$ echo "This text is the content" > original
[user@blue ~]$ ls
original vehicles
[user@blue ~]$ cat original
This text is the content
[user@blue ~]$ cp original clone
[user@blue ~]$ ls
clone  original vehicles
[user@blue ~]$ cat clone
This text is the content

In order to copy directories, you need to provide the -r option to the cp command. This option stands for recursive, and basically it means that it will traverse the whole tree under the directory that you want to copy.

Lab Report

  1. There is a directory named mail inside the /var/log directory. What is the relative path to that directory if you are in your home directory?
  2. Absolute and relative pathnames can be mixed. Consider the following path:/usr/include/curl/./././///../boost/random/../math. Simplify that path so it does not use relative references (hint: try it with the cd command).

The following diagram corresponds to a portion of the filesystem in blue. Fill out the empty elements in A, B, C and D.

../../_images/navfill.svg
  1. A
  2. B
  3. C
  4. D
  5. What is the oldest file in the /usr/bin directory?
  6. What is the newest kernel file in blue (kernel files are named with the suffix vmlinuz)?
  7. We want to make a backup of the vehicles directory into a directory called vehicles_backup. What command will serve this purpose?
  8. Provide a series of commands to create the following directory structure in your current working directory?
transport
├── bicycles
│   └── mountain
├── cars
│   ├── pickup
│   ├── sedan
│   │   └── compact
│   └── suv
└── jet
    ├── cargo
    └── fighter
  1. Make your home the working directory. Run the following command: tar -xf /tmp/vehicles.tar. This will create three files with names of transportation vehicles in your working directory. Provide a list of commands that will result in moving the files to the corresponding directory within the transport directory (durango is an SUV, prius a compact sedan, and stumpjumper is a mountain bike).
  2. The three files should contain the Wikipedia link that corresponds to the name of the file. However, the file named prius is incorrect. Provide a single command that will make the file name reflect the content of the file (e.g. the file name should be fseries instead of prius) and will place the file under the correct directory (~/transport/cars/pickup)