I work at a software consultancy and am currently staffed as part of a DevOps team. We’re focused on operations: making it easy for developers to build and deploy applications. This involves lots of traditional system administration tasks, so I spend most of my days spelunking around the Linux command line.
When I joined the team I knew only the very basics of Linux, enough to navigate around the filesystem, manage files, and run scripts. In the past few months I’ve learned a great deal (though I still have a lot to learn), and I’ve noticed myself and my team coming back to the same subset of utilities and commands again and again.
Linux is a beast, and there’s a great deal you can do with it. But for most developers, and for most purposes, you can go far simply by learning the ten things listed in this article. Just like a builder may have hundreds of tools, but keep returning to her nail-gun again and again; the utilities and commands in this article are a fantastic starting point for growing more comfortable with Linux, and being able to achieve most things you’ll need to do on a regular basis.
In writing this, I’ve been inspired by my colleague Cam Jackson’s article on 9 Things Every React.js Beginner Should Know.
1. Navigating the file system
As a developer, you’ll need to be comfortable navigating around the Linux file system. After opening up a terminal multiplexer like Terminator, you’ll be dropped into the file system. You can use the pwd (Print Working Directory) command to see where you are. From your current location, you can move anywhere else with the cd (Change Directory) command. For example:
$ cd ~/.ssh/
In the above example the tilde (~) represents your user’s home directory. If you’re inside a sub-directory and want to go up a level, you can do so with .., for example:
$ cd ../Documents
Many Linux beginners don’t realize that you can also use the tab key to auto-complete file and directory names.
To quickly create a file you can use the touch command.
$ touch hello.txt
To copy a file or directory, use the cp command. In example below, the left-most file is the original file, and the right-most file is the copy to be created.
$ cp hello.txt ciao.txt
Perhaps you made a mistake when naming your copied file and want to rename it. You can do this with the mv (Move) command. You can also use the mv command to move a file from one directory to another.
The below invocation of mv will rename the file hello.txt to bonjour.txt.
$ mv hello.txt bonjour.txt
Let’s say the creation of ciao.txt was a mistake. We can delete it with the rm (Remove) command.
$ rm ciao.txt
If we want to delete a directory and everything in it, you can pass the-rf flag to rm. -f will remove the files without confirmation, and -r will remove files inside the directory. Note: be careful with this command. Make sure you are not deleting something important.
$ rm -rf ~/Downloads
Let’s hold up a second. The Linux command line doesn’t show you what’s inside a particular directory unless you ask it to. It’s tough to be productive if you can’t see what you’re working with. This is where the ls command comes in useful.
$ ls ~/Downloads
ls (List Files) allows you to see the filenames of any files in the given directory. If the command is run without an argument it will default to the current directory, but you can also specify a path on the right-hand side, as seen in the above example. ls is the simplest form of this command, listing only filenames. Often, you’ll want to see more information about the files you’re interested in. You may also want to see hidden files, which sometimes hold important configuration.
The ls -l command allows you to see a more detailed view of each file. It’s so commonly used that ll is an alias that does the same thing. If you want to see hidden files as well, use ll -a or ls -la.
The ‘long list’ (-l) version of the ls command will show you the following information about each file you are inspecting:
- File owner
- File group
- File size
- Modification time
- Filename
With these few commands, you should be able to comfortably move around the Linux file system, and create, move and delete files.
2.cat, grep, and the wonders of piping
The cat or concatenate command writes files to the standard output (the text you see on your screen in your terminal).
$ cat hello.txt
cat is often the easiest way to quickly inspect the contents of a file. It becomes especially powerful when piped into grep:
$ cat guest_list.txt | grep Lucy
‘Piping’, i.e. the | character, allows you to chain commands together, using the output of the left-hand side command as input to the command on the right-hand side. It’s a useful technique that allows you to do complex output processing by combining simple commands together.
One of the most common uses of the pipe command is to grep the result of the left-hand side command. grep, a catchy acronym for the not-so-catchy name Global Regular Expressions Print, is a simple utility that searches input for a line that matches the given pattern, in this case, a line containing the word ‘Lucy’.
Another very common use of cat and grep together is searching for a specific event in a large log file, i.e. /var/log/messages.
$ cat /var/log/messages | grep '500 Internal Server Error'
grep can be used for searching any kind of output, not just file contents. Many Linux commands output dozens of lines packed with information. For example, if your Linux machine is running over a dozen Docker containers, you can use grep to zero-in on only the container you are interested in:
$ docker ps | grep my-awesome-container
You’ll learn more about the Linux ps command shortly.
You can also save the output of any command to a file by using redirection (>):
$ echo "Linux was created by Linus Torvalds" > bio.txt
The above command will create a new file, or overwrite the contents of an existing file. To append to an existing file, use >> instead of >.
3. find
You’ve dropped inside a directory with dozens of sub-directories. You know there’s a file inside this directory that you need, but you aren’t sure where it is. find can help!
$ find . -name CS101
The ‘find’ command lets you walk a file hierarchy (the first argument to the find command), and search it on several different dimensions. You can type man find into your terminal to see them all, but the example above uses the -name flag to search for filenames including the character sequence CS101. Instead of bumbling through directory after directory like someone searching for an orphaned sock under the bed, consider using find.
4. File permissions and ownership
Every file and directory in the Linux file system has permissions and an owner. Permissions are who is allowed to do what with the file. To see the permissions on a file, use the command ls -l <filename>. You’ll see something like this in the left-most column:
-rw-r--r--
This is a little hard to read, so let’s break it down in the example below:
..own grp oth
-|---|---|---
The dash on the far left will be replaced with a d if the file is a directory. The next three groups of three dashes represent permissions for the owner of the file, the group of the file, and all others. The ‘owner’ of a file is the user who initially created it, though ownership can be changed (more on that shortly). The ‘group’ that owns a file will be the group that its owner belongs to, though this can also be changed. The permissions for ‘others’ apply to any user who is not the owner of the file and not in the group that owns the file. One exception is the ‘root’ user, which has full access to every file on the system.
Here is an example of a file where the owner has full permissions but nobody else can read, write or execute the file:
-rwx------
You may occasionally get a ‘Permission denied’ or ‘Username is not in the sudoers file’ error when trying to do something with a file or directory. This generally means your user does not have the correct permissions for what you are trying to do. You will need to change to a user who does, for example:
$ su sudo
To re-run your previous command as root, you can use sudo !!, where the two exclamation marks will be replaced with your previous command.
You will occasionally need to change the permissions on a file:
$ chmod u=rwx,g=rx,o=r hello.txt
In the above example, we set read, write and execute permissions for the user, read and execute permissions for the group, and read permissions for other users.
If you’re game to learn it, there’s an even simpler shorthand for setting permissions:
$ chmod 766 hello.txt
The 7 represents owner permissions, the 6 represents group permissions, and the last 6 represents permissions for the group. But where do these numbers come from?
Each permission is represented by a digit. The permissions for each user type are added together to form the final number.
- 4 is “read”,
- 2 is “write”,
- 1 is “execute”
- 0 is “no permissions”
So, 7 represents 4 (read) + 2 (write) + 1 (execute). 6 represents 4 (read) + 2 (write), and so on.
You’ll need read permissions to inspect the contents of a file, write permissions to make changes to the file, and execute permissions to run scripts or executables.
You can change the owner and group of a file with the chmod command. For example, let’s say you have a file with the following permissions:
drwxr-xr-x 32 root root 4096 16 Jul 17:48 cowsay.sh
You decide that you want your user account to be the owner of the file, and its group to be your group. As root, you can run the following command to change the file’s owner and group:
$ chown <your_user>:<your_group> hello.txt
When you run ls -l on the file, you’ll see that its owner and group have changed:
drwxr-xr-x 32 your_user your_group 4096 16 Jul 17:48 cowsay.sh
5. reverse-i-search
reverse-i-search is a handy utility to search back through your command history and re-run a previous command. You may know that hitting the up arrow allows you to cycle through your command history, but what if you want to re-run a command you ran 5 minutes ago, and have run dozens of commands since then? reverse-i-search is the perfect tool for these situations.
To initiate a reverse-i-search, type ctrl + r. You can then start typing in characters that feature in your target command. The search is fuzzy, so these characters can occur anywhere in the command string. In cases where there are multiple matches, you can cycle through them by hitting ctrl + r again.
6. Watching, Tailing, and Following
You’ll often want to re-run a command to check for changes in the output at regular intervals. For example, you may want to see the rate that memory usage on your machine is changing over time:
$ watch -n 5 free -m
The above command will run the free command to show memory usage in megabytes, every 5 seconds.
What about when you want to see the latest changes to a file, for example, a log file?
$ tail /var/log/messages
By default, tail will print the last 10 lines of a given file to standard output. You can change the number of lines printed with the -n flag, i.e. -n 1000. One limitation of this method is that it prints the file at the time the command was run. The output won’t be updated as new lines are written to the file. We can fix this by adding the -f flag as follows:
$ tail -f -n 100 /var/log/messages
This will show you a live updated output of the last 100 lines of /var/log/messages.
7. man pages and getting help
Many Linux commands have optional parameters that refine their behavior. These optional parameters or flags are often prefixed with a dash, i.e. -l. It can be difficult to remember all the possible options you can pass to a command. This is one of the many reasons why manpages are useful. By typing man <command> you’ll get access to a description of what the command does, as well as a list of all possible options and what they do. man in this instance stands for manual, though I do chortle occasionally when running commands like man cat.
8. Checking and monitoring system resource usage
It’s surprising how many operating system issues are caused by a shortage of system resources: memory, CPU, or disk space. Luckily, Linux provides us with several tools we can use to quickly diagnose these issues.
The top command presents you with lots of information about all the different processes (essentially, applications) running on your machine, including memory consumption and CPU utilization. This command is useful for identifying rogue processes that are consuming too many resources, or processes that should not be running, but are.
The free command allows you to see current memory usage on a machine. This is useful for checking that a machine has enough memory to do what you require of it, for example, running a batch process that consumes several GB of memory.
The df -h command shows disk usage on your machine in a human readable format (megabytes and gigabytes rather than bytes). You can see at a glance which of your disks is reaching close to 100% used, and free up space if necessary. A full disk can cause all kinds of problems on a machine.
9. Managing processes
You can see all the processes running on your machine with the following command:
$ ps aux
The aux options tell ps to show processes owned by all users. This is a useful diagnostic tool when you think a process might be running when it shouldn’t be, or that it might be consuming too many resources. You may also want to start a new process and find that you get an error because the same process is already running.
This command has a lot of output, and is often paired with grep when you want to see information for a single process.
$ ps aux | grep ruby
Often, you’ll want to kill the offending process. An easy way to do this is with the following command, which allows you to kill processes by name rather than by their PID (Process ID):
$ pkill -9 Slack
You can also kill every instance of a process with killall. An example usage is killall ruby, to kill all ruby processes. However, be careful that you’re killing the correct process, and use these commands with care.
10. Vi
Vi is an ancient, powerful text editor that is installed on all Linux machines by default. First released in 1978, it has since spawned a more feature rich variant called Vim. Though it’s an old tool, many programmers swear by Vim. It’s older and lighter-weight variant Vi is the text editor most likely to be installed on any machine you may need to SSH on to. For this reason, having a basic understanding of how to use Vi can help you to quickly edit and manipulate the contents of files on almost any machine without leaving the comfort of your terminal.
Becoming a Vi/Vim expert is a multi-year journey, but you can go quite far with only a little bit of knowledge. Vim Adventures is a cute and creative way to get familiar with some basic commands. This tutorial will also help you configure Linux for development with Vim.
Beyond the basics
If you’ve enjoyed learning these Linux tips and tricks, there are some great books you can read to take your knowledge further:
- How Linux Works by Brian Ward
- The Linux Command Line by William E. Shotts
- Linux For Beginners by Jason Cannon
- Linux Command Line and Shell Scripting Bible by Richard Blum & Christine Bresnahan
I hope you’ve enjoyed learning more about Linux. Unlike many tools, operating systems and frameworks you’ll encounter in your programming career, Linux is relatively stable. Throughout changes and new versions, the fundamentals remain basically the same, and have done so for a while. The knowledge you’ve gained is likely to last you a long time.
