The Environment

Environment Variables

In the last chapter, we talked about how your "current directory" provides context for commands you run. Another way of providing context is through something called environment variables. In programming, variables are used to store data and to be able to reference and retrieve that data at a later point using a name. In the command cd $HOME, the $HOME part is a reference to the HOME variable, and is replaced by the path to your home directory when the command is run. In other words, running cd $HOME is the same as running cd /home/ubuntu, assuming your home directory is /home/ubuntu.

When you log in to the command line, a variety of environment variables are automatically set. You can see exactly what variables have been set, along with their values, by running env at the command line. Type env, hit enter, and find the value for HOME. It should say something like /home/ubuntu, where ubuntu will be replaced by your username. If you're doing this on a Mac, the value will probably be something like /Users/bob. This is the path to your home directory.

How to Change your Command Line Environment

While there are several environment variables that are set for you automatically, you can also set your own or modify existing variables. You can do this on the fly, so that your changes only affect the current command or the current session, or you can make the changes more permanent so that they stick between sessions.

Note: The term "session" refers to the state of being logged in to a computer's command line interface. When you log in, you start a new session, in which your commands will be recorded and other contextual information will be maintained. When you close Terminal or type "exit", your session is closed and that context and data is lost.

Setting Environment Variables on the Fly

There are two ways to set an environment variable on the fly:

  1. Set the variable on its own line, then use it anywhere:

    $ SOMETHING="some value"
    $ echo $SOMETHING
    some value
    
  2. Set the variable before a command, on the same line:

    $ SOMETHING="a value" env
    ...
    SOMETHING=a value
    ...
    

Note: You cannot (very easily) use a value on the same line that you set it. That's because variables are evaluated before the setting occurs:

$ SOMETHING="something else" echo $SOMETHING
# no output

Did you notice that when you set a variable you don't prepend the dollar sign, but when you reference it, you do? Also note that there should be no spaces between the variable and the equal sign or the equal sign and the value. Lastly, it's usually best to use quotations around the value that you are assigning to the variable, but you don't have to when the value doesn't have any special characters.

Let's try changing our current session's environment. Maybe you'd like to simplify your prompt. To change your prompt, adjust the PS1 variable to whatever you'd like it to be:

$ PS1="(testprompt)> "
(testprompt)>

As you can see, the prompt is now (testprompt)>, and every time a command finishes, it will show up again. If you want a more complicated prompt, try the following:

$ PS1="\n\[\e[0;37m\][\h] \e[0;35m\]\d\e[0m\]\n\[\e[0;31m\]\u\[\e[0;34m\] in \[\e[1;33m\]\w\[\e[m\]\[\e[0;31m\]\n\[\033[35m\]$\[\033[00m\] "

[chopin] Wed Apr 08
ubuntu in ~
$

The new prompt is multi-line and has color-coding. If you want to revert to your old prompt, just close your session and start a new one. Since we made the changes to the environment variable PS1 on the fly, they won't be used in future sessions.

Making More Permanent Changes

It is possible to make more permanent changes to the command line environment. When you start a command line session by opening a new Terminal window, one or more environment files are executed. These files can be used to modify or create environment variables. They are usually located in your home directory and include the following files: .bashrc and .bash_profile in Bash, and .zshrc and .zprofile in Zsh. Because they start with a ., they are considered to be "hidden" files, and using the ls command alone won't show them. Type ls -a ~ to see them listed along with other files in your home directory. Remember, the ls is the command, and the -a and ~ are arguments to the ls command. The -a flag tells the ls command to include files that start with . in its output, while the ~ is the directory that ls should inspect (recall that ~ means your "home" directory).

Notes for Bash

Create a .bashrc and a .bash_profile file with the following command:

touch .bashrc .bash_profile      # Be careful about the spaces

If the files already exist, this command won't change them. You should edit the .bash_profile file and add the following line somewhere near the top of the file:

[ -r ~/.bashrc ] && . ~/.bashrc  # be careful with the spaces

If you can, open a new Terminal window to make sure it still works.

Once you've created these files, you can make most configuration changes in the .bashrc file. You can generally ignore the .bash_profile file.

Notes for Zsh

Create a .zshrc file with the following command:

touch .zshrc                    # Be careful about the spaces

If you can, open a new Terminal window to make sure it still works.

Once you've created the .zshrc file, you can make most configuration changes there. You can generally ignore the .zprofile file if you have one.

$ ls -a ~
.              .cache          .sudo_as_admin_successful
..             .mysql_history  .vbox_version
.bash_history  postinstall.sh  .veewee_version
.bash_logout   .profile        .vim
.bashrc        .ssh            .viminfo

The rules behind which environment file is read for a new session are complicated and depend on your shell and how the session is created. For our purposes, using .bashrc for Bash, and .zshrc for Zsh, should be sufficient. If your edits aren't working, try updating .bash_profile or .zprofile instead.

Log in to your console and type the following command:

cat ~/.bashrc                 # If using bash
cat ~/.zshrc                  # If using zsh

The cat command reads the file and displays its contents. You may see a bunch of output from these commands, or nothing at all, depending on the current contents of these files.

Bash only: If you're ready to customize your prompt a bit more permanently, open the .bashrc file in a code editor, and add your custom prompt to the bottom of your .bashrc file:

export PS1="[your custom prompt goes here] "

Note: Editing hidden files can be a bit tricky if you've never done it before. The following steps will guide you through editing your .bashrc file.

  1. Open .bashrc with Nano by entering the following command:

    nano ~/.bashrc
    
  2. Place your cursor on the very last line of the file (make sure it's a new, blank line) and paste the following:

    export PS1="[your custom prompt goes here] "
    
  3. Save .bashrc by pressing <Ctrl> + o then <Enter>. Exit Nano by pressing <Ctrl> + x.

  4. Re-run .bashrc by entering the following command:

    source ~/.bashrc
    

The export isn't needed because the variable is already available globally. However, it doesn't hurt to specify it. Use the following pieces along with any custom text to make your prompt:

Special Characters Description
\h Hostname
\u User name
\w Current directory
\W Basename of current directory
\d Current date
\n Newline

Did you notice that just editing and saving your .bashrc file didn't do anything? The file is only evaluated, or run, when you first log in. If you want to re-run a particular environment file like .bashrc or .bash_profile, use the source command:

source ~/.bashrc

To revert back to your old prompt, edit the same file and remove your PS1 setting. Then run source on that file:

[my custom prompt]$ source ~/.bashrc
$

Using Environment Variables

Let's look for a moment at the different ways we can use environment variables.

1. As Parts of Commands

First, variables can be used as arguments to commands. Take a look at the following example:

$ MESSAGE="Hello, world!"
$ echo $MESSAGE
Hello, world!

This is a very simple example, but you can see that the $MESSAGE variable is used as the first (and only) argument to the echo command. You can actually use variables as commands as well:

$ MESSAGE="Hello, world!"
$ COMMAND="echo"
$ $COMMAND $MESSAGE
Hello, world!

2. Interpolated in Strings

Variables can also be interpolated, or included, in other strings. Take the following example:

$ MESSAGE1="This is message 1."
$ MESSAGE2="This is message 2."
$ MESSAGE="$MESSAGE1 $MESSAGE2"
$ echo $MESSAGE
This is message 1. This is message 2.

To ensure that variables will be interpolated, you must use double quotation marks ("), not single quotes ('). Try the following example in your command line to see the difference:

$ MESSAGE='$MESSAGE1 $MESSAGE2'
$ echo $MESSAGE
$MESSAGE1 $MESSAGE2

The difference between single and double quotes is primarily an issue of interpolation. With double quotes, a $ is treated as the beginning of a value that will be interpolated into the string; with single quotes, a $ is an ordinary character with no special meaning.

Single and double quotes also play a part with "escape literals" such as the \h and \u characters that you can use in the PS1 environment variable (see above). Use single quotes to treat the \ as an ordinary character; use double quotes to treat the \ and the following character as an escape literal.

Finally, you can use double quotes to quote a string that contains single quotes, or single quotes to quote a string that contains double quotes:

$ echo "What's up, Doc?"
$ echo 'My name is "Pete".'

The situation gets much more complicated if you have a string with both single and double quotes as well as \ and $ characters. We won't get into that here.

3. Behind the Scenes

Environment variables can be used by commands (programs) behind the scenes. In other words, you can set a variable, then run a command without passing the variable as an explicit argument to that command, and the command could use that variable. The PWD variable is automatically used by any command that tries to get the user's current directory. The HOME variable is automatically used by cd when you don't pass any arguments to it. If you make up a custom variable (like PI=3.14), only programs that know about it will be able to use it without explicitly using it as an argument.

If you want to temporarily change a variable before it gets used in a command behind the scenes, you can set the variable immediately preceding the command on the same line:

# Set home to root directory and change to home.
$ HOME=/ cd
$ pwd
/

# Change to home directory.
$ cd
$ pwd
/home/ubuntu

Note how the second cd takes you to your original home directory, whereas the first cd takes you to the root directory because that's what you set HOME to.

$PATH and Executables

One of the most important environment variables you'll work with on the command line is PATH. In the last chapter, we discussed how commands are really just files, but we didn't talk about how the command line knew which file to execute for commands like cd or echo or other built-in or installed programs. The PATH variable provides the additional context that the command line needs to figure out which particular file to execute. Let's look at a PATH variable's value:

$ echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games

If you examine the output of the echo $PATH command above, you'll see that it is a bunch of paths connected by colons. You may have noticed that most of the paths end in /bin. This is because bin is short for "binary", and bin is a standard directory name for executable files, or programs.

Let's look at what resides in one of the directories listed in the PATH variable:

$ ls /usr/bin
[                               mysqlanalyze
2to3                            mysqlbinlog
2to3-2.7                        mysqlbug
a2p                             mysqlcheck
aclocal                         mysql_client_test
aclocal-1.11                    mysql_convert_table_format
...
mysql                           zdump
mysqlaccess                     zsoelim
mysqladmin

Depending on your computer, different types of files may have different colors. On many computers, for example, executables will probably be colored green. If you look at the files located in your home directory, however, they are probably white and blue, which tells us that they are not executable.

One of the items in the /usr/bin directory is man. If I type man on the command line and hit enter, it will execute that file. How can I be sure that it will execute that file, and not some other file that happens to be named man on my server?

When you type a word into the command line, and it doesn't start with a /, ~, or a . (because those would indicate a path to an actual file), the command line will search each of the directories listed in the PATH environment variable for that command. Thus, when we type man and hit enter, the CLI searches /usr/local/sbin, then /usr/local/bin since those are first in the list. It then moves on through the list of directories until it finds a file named "man" in one of them. It then stops searching and executes the file. We can verify which file is getting executed by using the which command:

$ which man
/usr/bin/man

The which searches the paths for the named command (man, in this case) and displays the first one it finds. (Some versions of which will locate all occurrences of the command in the path, but most only report one,) As you can see, the path that is output by the which command corresponds with the man command we found earlier.

You can create or install executables. To make it so that a custom executable can be used like a built-in command, all you have to do is make sure it has the correct permissions (discussed in the next chapter), and add the path to the directory it is contained in to the PATH variable in ~/.bashrc or ~/.zshrc:

export PATH="/path/to/my/executables-directory:$PATH"

Note how we added the executables directory, then the colon, then the $PATH variable again. This preserves the old PATH locations while making your directory the highest priority. We're adding our directory to the top of the list.

Note that if this path you're adding to the $PATH environment variable contains an executable file called "man", then when you type man from the command line, this new file will be executed instead of the correct one in /usr/bin. The path lookup rules for all commands relies heavily on managing this $PATH environment variable carefully.

To sum up:

  • The PATH variable determines which directories are searched when a command is entered
  • PATH is an ordered, colon-delimited, list of directories that contain executables
  • The order of the directories in the PATH variable is first-found-first-execute
  • If you use a /, ., or ~ before your command, the command line will interpret that as an actual path to a file, and will not use the PATH variable
  • You can add to PATH to make more commands available without having to memorize their exact path
  • Modifications to PATH, or any environment variable, on the fly will not be permanent; permanent modifications should be done in an environment file, like .bashrc

Exercises

  1. Run the following commands to experiment with altering your command line environment:

    $ cd
    $ PS1="\u@\w$ "
    ubuntu@~$ echo "Hello world"
    Hello world
    ubuntu@~$
    

    Exit out of Terminal (make sure to close each tab and window if you are on a Mac) and open it up again. What does your prompt look like now? The value you set PS1 to above should no longer be in effect.

    Set your prompt in your ~/.bashrc (or ~/.zshrc if appropriate) file:

    echo 'export PS1="this is a test$ "' >> ~/.bashrc
    

    Note that we are using single quotes in this command to surround the argument export PS1="this is a test$ ". We do that since part of the argument includes double quotes -- if we used double quotes around the entire argument, bash would have trouble with the command.

    We'll discuss the differences between single and double quotes in more detaillater.

    The redirection operator (>>) is used to append text to a file. If the target file doesn't exist, then it will be created.

    Exit and open up a new terminal. You should see something like this:

    this is a test$
    

    Edit the ~/.bashrc file (or ~/.zshrc if appropriate), remove the last line, and run source ~/.bashrc (or source ~/.zshrc) to return your prompt to its previous state.

  2. As discussed previously in this chapter, the PATH variable determines which files can be executed without specifying their path explicitly. Run through the commands in this exercise to see this principle in action.

    Set the current PATH variable to another variable so we can revert to it later:

    $ OLDPATH=$PATH
    $ echo $OLDPATH
    /usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/opt/aws/bin:/home/ubuntu/bin
    

    Temporarily change PATH and try to run a program that is no longer located in the directories specified in PATH:

    $ cd /usr/local
    $ ls
    bin etc games include lib man sbin share src
    $ PATH="" ls
    -bash: ls: No such file or directory
    $ ls
    bin etc games include lib man sbin share src
    

    Note: The contents of local folder may be different in your case.

    Now, create an executable in your home directory by entering the following:

    cd ~
    echo '#!/bin/bash' > test.sh
    echo 'echo "Hello world"' >> test.sh
    chmod +x test.sh # Make sure test.sh has the executable permission
    

    Run the executable, first by specifying the path, then by running it like a command:

    $ ./test.sh
    Hello world
    $ test.sh
    -bash: test.sh: command not found
    

    Now, change PATH more permanently for your current session, and try to run test.sh again:

    $ PATH=$PATH:$HOME
    $ echo $PATH
    /usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/opt/aws/bin:/home/ubuntu/bin:/home/ubuntu
    $ test.sh
    Hello world
    $ cd /
    $ test.sh
    Hello world
    

    Exit out of Terminal and then open it up again. Try running test.sh again:

    $ test.sh
    -bash: test.sh: command not found