:orphan: ************************************************* Lab No. 2: The File System and Working with Files ************************************************* After completing this lab, students will be able to: * Inspect and navigate the linux filesystem * Create, remove and alter files and directories The File System =============== A file system is a method for organizing and storing data. Without a file sytem, the operating sytem (and the users) would have a hard time locating files that are needed to perform tasks. The file system consists of files and their attributes and relationships between files. The Linux file system is composed of **files** and **directories**. A file is a collection of data that is logically related, and it is the smallest unit of storage in the Unix file system. Directories are also file, but they are special since they may contain other files and other directories (a physical analogy isa folder or envelope where you can use to organize and store papers, or other envelopes). In this way, the files sytem is organized in a hierarchical structure that resembles a tree, typically represented upside down (in the same fashion as a *family tree*). The top directory in a Linux file system is called the **root directory**, or simply, the **root**, and it is represented by the ``/`` (slash) character. .. image:: fhs.svg The standard hierarchy of Linux Filesystems is published by the Linux Foundation as a specification (available under https://refspecs.linuxfoundation.org/fhs.shtml). The following directories are required by the standard (Refer to Table 3-4 in TLCL for more details) ========= ========================================================================= ========= ========================================================================= Directory Description / root directory /bin Essential command that must be present for boot/recovery /boot Contains the boot loader and kernel files /dev Device files. /etc System configuration /home Users directories and files /lib Essential shared libraries and kernel modules /media Mount point for removable media (USB Drives, CD-ROMS) /mnt Mount point for mounting a filesystem manually /opt Add-on application software packages ("optional") /proc Process information pseudo-file system (contains runtime information). /root Home directory for the "root" user. /run Data required by running processes /sbin System binaries. /tmp Temporary files /usr All the programs and support files to be used by regular users /var Variable data (logs, user mail, spool files, etc). ========= ========================================================================= File system navigation ---------------------- In Linux all files must have a name. The following rules apply when naming a file: #. File names are case sensitive #. File names can only contain uppercase and lowercase characters, numbers, the dot and the underscore symbols. #. File names are limited to 255 characters To be able to work with a file, we need to know it's file name, but we also need to know the exact location of a file within the file system hierarchy. A *pathname* is the trail through the directory hierarchy to a file or directory. A *pathname* can be: #. *absolute* when it traces a path from the *root directory* (``/``). This is complete and unambiguous. Absolute pathnames **always** begin with ``/``. #. *relative* when it traces a path from the user's current position or location on the filesystem (known as the *current working directory*). The beginning (left-most) part of a relative path name is either an ordinary file, a directory or a reference to the parent of the current directory. The following characters have special meaning when used in pathname: #. A single dot ``.``: refers to the current directory #. A double dot ``..``: which refers to a parent directory. #. The tilde character (``~``) which is expanded to the current user's home directoriy. There are several commands that help you to navigate the file system. Unix systems have the concept of the *working directory* (also referred as *current directory*), which determines the directory within the tree that your session is currently associated with. The ``pwd`` command prints your *current directory*. .. parsed-literal:: [user\@blue ~]$ :command:`pwd` /home/student/user When you login into Unix systems, your *current directory* is set to your *home directory* by default. The ``~`` in the prompt means that your current working directory is your *home directory*. As mentioned before, home directories for most users are typically located under ``/home`` (On MacOS, home directories are located in ``/Users``.) You can change your current directory with the ``cd`` command, which takes as an argument the *pathname* of the desired directory. If no argument is provided, then ``cd`` will change the working directory to the user's *home directory*. Execute the following command to change your current working directory to the root directory: .. parsed-literal:: [user\@blue ~]$ :command:`cd /` [user\@blue /]$ The cd command does not provide any output. However, you can see that the prompt changed to ``/`` instead of ``~``. Let's see what is the output of the ``pwd`` command: .. parsed-literal:: [user\@blue /]$ :command:`pwd` / [user\@blue /]$ Let's check the content of the root directory in **blue**, so we can compare it with the standard list of directories under root. In order to do this, we can use the ``ls`` commands which lists the contents of a directory. .. parsed-literal:: [user\@blue /]$ :command:`ls` bin boot etc kirby lib lost+found media netatalk proc run srv target tmp var blue dev home kodiak lib64 mamer mnt opt root sbin sys thekeep usr When ``ls`` is executed without any arguments, it lists the contents of the current directory. You can also specify a directory to lists its contents without having to change directory. Let's check the contents of the ``/usr`` directory: .. parsed-literal:: [user\@blue /]$ :command:`ls /usr` arm64-linux-gnu avr etc include libexec man sbin tmp arm-linux-gnu bin games lib local powerpc-linux-gnu share x86_64-w64-mingw32 arm-linux-gnueabi @DATADIRNAME@ i686-w64-mingw32 lib64 lost+found s390-linux-gnu src x86-linux-gnu Sometimes it is useful to have a listing where each item is listed on it's one line. Let's see how that looks: .. parsed-literal:: [user\@blue /]$ :command:`ls -l /usr` total 604 drwxr-xr-x. 3 root root 4096 Jul 2 2018 arm64-linux-gnu drwxr-xr-x. 4 root root 4096 Jul 2 2018 arm-linux-gnu drwxr-xr-x 3 root root 4096 Sep 19 2018 arm-linux-gnueabi drwxr-xr-x. 6 root root 4096 Jul 12 2018 avr dr-xr-xr-x. 2 root root 143360 Nov 4 11:46 bin drwxr-xr-x 3 root root 4096 Aug 7 2018 @DATADIRNAME@ drwxr-xr-x 2 root root 4096 Dec 26 2018 etc drwxr-xr-x. 2 root root 4096 Aug 2 2017 games drwxr-xr-x 4 root root 4096 Aug 9 2018 i686-w64-mingw32 drwxr-xr-x. 333 root root 20480 Aug 7 12:00 include dr-xr-xr-x. 82 root root 20480 Mar 15 2019 lib dr-xr-xr-x. 382 root root 270336 Aug 7 12:00 lib64 drwxr-xr-x. 88 root root 20480 May 7 2019 libexec drwxr-xr-x. 16 root root 4096 Dec 13 11:14 local drwx------. 2 root root 16384 Jun 26 2018 lost+found drwxr-xr-x 3 root root 4096 Feb 22 2019 man drwxr-xr-x. 3 root root 4096 Jul 2 2018 powerpc-linux-gnu drwxr-xr-x. 3 root root 4096 Jul 2 2018 s390-linux-gnu dr-xr-xr-x. 2 root root 36864 Aug 7 11:28 sbin drwxr-xr-x. 691 root root 20480 May 23 2019 share drwxr-xr-x. 8 root root 4096 Aug 7 11:29 src lrwxrwxrwx. 1 root root 10 Aug 2 2017 tmp -> ../var/tmp drwxr-xr-x 3 root root 4096 Aug 9 2018 x86_64-w64-mingw32 drwxr-xr-x. 3 root root 4096 Jul 2 2018 x86-linux-gnu Notice that adding the ``-l`` (which stands for long listing format) changes the behavior of the command. (There is a lot of useful information packed in the output of that command, we will learn about those in later exercises) As we saw in Lab 01, these additional instructions are called **options** and almost every command allows users to specify different options to control the behavior of the command. As a refresher from Lab 01, let's use the ``man`` utility (which opens the manual pages, or normally referred as the **man pages**) for the ``ls`` command: .. parsed-literal:: [user\@blue /]$ :command:`man ls` .. parsed-literal:: LS(1) User Commands LS(1) NAME ls - list directory contents SYNOPSIS ls [OPTION]... [FILE]... DESCRIPTION List information about the FILEs (the current directory by default). Sort entries alphabetically if none of -cftu‐ vSUX nor --sort is specified. Mandatory arguments to long options are mandatory for short options too. -a, --all do not ignore entries starting with . Scroll down the *man page* for the ``ls`` command and find the ``-l`` option. Also, review the description of the ``-1`` and the ``-s`` options. Exit by pressing :kbd:`q`. Command options can usually be combined. Let's see this in action by invoking ``ls`` with the ``-1s`` option: .. parsed-literal:: [user\@blue /]$ :command:`ls /usr -s1` total 604 4 arm64-linux-gnu 4 arm-linux-gnu 4 arm-linux-gnueabi 4 avr 144 bin 4 @DATADIRNAME@ 4 etc 4 games 4 i686-w64-mingw32 20 include 20 lib 268 lib64 20 libexec 4 local 16 lost+found 4 man 4 powerpc-linux-gnu 4 s390-linux-gnu 36 sbin 20 share 4 src 0 tmp 4 x86_64-w64-mingw32 4 x86-linux-gnu Let's try now changing to the `/usr/local/bin` directory: .. parsed-literal:: [user\@blue /]$ :command:`cd /usr/local/bin` [user\@blue bin]$ :command:`ls` istatserver n node npm npx update_rubygems If we wanted to change directory to ``/usr/local`` which is the parent directory of ``/usr/local/bin`` we could change directory by using the absolute path, which is what we have done so far. However, we can use the special characters that we mentioned before. The parent directory can be referenced by using the ``..`` (dot-dot) character: .. parsed-literal:: [user\@blue bin]$ :command:`cd ..` [user\@blue local]$ :command:`pwd` /usr/local Suppose we wanted to look at the contents of the ``/usr/local/bin`` directory that we just visited. We could again use the absolute path. But, since we are on the parent (``/usr/local``) it is a lot easier to use a relative path: .. parsed-literal:: [user\@blue local]$ :command:`ls bin` istatserver n node npm npx update_rubygems An alternative, more explicit way to refer to the current directory is using the ``.`` (dot) character: .. parsed-literal:: [user\@blue local]$ :command:`ls ./bin` istatserver n node npm npx update_rubygems In this case, the ``.`` character does not make a difference. You will learn later (when we talk about **wildcards**) how the ``.`` character can become quite useful. Let's change the directory back to your home directory, using relative paths (replace ``user`` with your username) .. parsed-literal:: [user\@blue local]$ :command:`cd ../../home/student/user` [user\@blue ~]$ You can also use the ``-`` option of the ``cd`` command to change directory back to the previous working directory: .. parsed-literal:: [user\@blue ~]$ :command:`cd -` [user\@blue local]$ :command:`pwd` /usr/local And if you execute it again, it will take you back to your previous working directory: .. parsed-literal:: [user\@blue local]$ :command:`cd -` [user\@blue ~]$ Creating directories and files ------------------------------ The ``mkdir`` utility creates new directories. It requires as arguments the pathnames of the directories to be created. The following example shows the commands required to generate the directory structure shown below (the shown output assumes that your home directory is empty. If that is not the case, then existing files will show in the output of the ``ls`` command.) .. image:: cars.svg .. parsed-literal:: [user\@blue ~]$ :command:`ls` [user\@blue ~]$ :command:`mkdir vehicles` [user\@blue ~]$ :command:`ls` vehicles [user\@blue ~]$ :command:`mkdir -p vehicles/cars/sedan` [user\@blue ~]$ :command:`ls vehicles/cars` sedan [user\@blue ~]$ :command:`mkdir -p vehicles/cars/pickup vehicles/cars/suv` [user\@blue ~]$ :command:`ls vehicles/cars` pickup sedan suv Notice that we did not create the ``cars`` directory directly, and instead we used the ``-p`` option to instruct the ``mkdir`` command to make any non-existing intermeditate directories. Note also that you can create more than one directory at a time. The ``tree`` command is useful to have a directory listing in a tree like structure: .. parsed-literal:: [user\@blue ~]$ :command:`tree vehicles` vehicles └── cars ├── pickup ├── sedan └── suv 4 directories, 0 files To remove directories you can use the ``rmdir`` command. Just as ``mkdir``, it requires the pathnames of the directories to be removed. However, ``rmdir`` has the limitation that it only works with empty directories. To remove directories that are not empty, you can use the ``rm -r`` command. Let's remove the ``./vehicles/cars/suv`` directory: .. parsed-literal:: [user\@blue ~]$ :command:`rmdir vehicles/cars/suv` [user\@blue ~]$ :command:`rm -r vehicles/cars/pickup` [user\@blue ~]$ :command:`tree vehicles` vehicles └── cars └── sedan 2 directories, 0 files Moving files ------------ The ``mv`` command can be used for two purposes: rename a file/directory, or move the file/directory to another path. Let's rename the ``sedan`` directory to ``coupe``: .. parsed-literal:: [user\@blue ~]$ :command:`mv vehicles/cars/sedan vehicles/cars/coupe` [user\@blue ~]$ :command:`tree vehicles` vehicles └── cars └── coupe 2 directories, 0 files Noticed that the first argument is the source file or directory and the second is the destination directory or file. In this case, since the destination did not exist, then it perfoms a "rename" To see how the ``mv`` command can be used to move files, let's recreate the ``sedan`` directory and and empty file under it: .. parsed-literal:: [user\@blue ~]$ :command:`mkdir vehicles/cars/sedan` [user\@blue ~]$ :command:`touch vehicles/cars/sedan/shelby` [user\@blue ~]$ :command:`tree vehicles` vehicles └── cars ├── coupe └── sedan └── shelby 3 directories, 1 file Let's move ``shelby`` from the ``sedan`` directory to the ``coupe`` directory (where it belongs, obviously). .. parsed-literal:: [user\@blue ~]$ :command:`mv vehicles/cars/sedan/shelby vehicles/cars/coupe` [user\@blue ~]$ :command:`tree vehicles` vehicles/ └── cars ├── coupe │   └── shelby └── sedan 3 directories, 1 file Notice that in this case, the destination exists, and it is a directory, so the result of the command is that the file is move to the destination directory. Copying files ------------- The ``cp`` command is used to copy files or directories. In the following example, a file is created by redirecting the output of the ``echo`` command into a file called ``original``. The ``cat`` command reads that file and prints that on the screen, this is done as a verification step. ``original`` is then copied into a file called ``clone``. Notice how the contents of ``clone`` are the same as the contents of ``original``. .. parsed-literal:: [user\@blue ~]$ :command:`echo "This text is the content" > original` [user\@blue ~]$ :command:`ls` original vehicles [user\@blue ~]$ :command:`cat original` This text is the content [user\@blue ~]$ :command:`cp original clone` [user\@blue ~]$ :command:`ls` clone original vehicles [user\@blue ~]$ :command:`cat clone` This text is the content In order to copy directories, you need to provide the ``-r`` option to the ``cp`` command. This option stands for *recursive*, and basically it means that it will traverse the whole tree under the directory that you want to copy. .. admonition:: **Lab Report** :class: worksheet 1. There is a directory named ``mail`` inside the ``/var/log`` directory. What is the relative path to that directory if you are in your home directory? 2. Absolute and relative pathnames can be mixed. Consider the following path:``/usr/include/curl/./././///../boost/random/../math``. Simplify that path so it does not use relative references (hint: try it with the ``cd`` command). The following diagram corresponds to a portion of the filesystem in ``blue``. Fill out the empty elements in ``A``, ``B``, ``C`` and ``D``. .. image:: navfill.svg 3. A 4. B 5. C 6. D 7. What is the oldest file in the ``/usr/bin`` directory? 8. What is the newest kernel file in **blue** (kernel files are named with the suffix ``vmlinuz``)? 9. We want to make a backup of the ``vehicles`` directory into a directory called ``vehicles_backup``. What command will serve this purpose? 10. Provide a series of commands to create the following directory structure in your current working directory? .. parsed-literal:: transport ├── bicycles │   └── mountain ├── cars │   ├── pickup │   ├── sedan │   │   └── compact │   └── suv └── jet ├── cargo └── fighter 11. Make your *home* the working directory. Run the following command: :command:`tar -xf /tmp/vehicles.tar`. This will create three files with names of transportation vehicles in your working directory. Provide a list of commands that will result in moving the files to the corresponding directory within the ``transport`` directory (durango is an SUV, prius a compact sedan, and stumpjumper is a mountain bike). 12. The three files should contain the *Wikipedia* link that corresponds to the name of the file. However, the file named ``prius`` is incorrect. Provide a single command that will make the file name reflect the content of the file (e.g. the file name should be ``fseries`` instead of ``prius``) and will place the file under the correct directory (``~/transport/cars/pickup``)