Archive for the 'Unix' Category

The diff command

If you're new here, you may want to subscribe to my RSS feed.

The diff command is a very useful command to see the differences between two files. An example of this command would be

diff file1.txt file2.txt

It takes file2.txt and compares it to file1.txt and displays the differences that are in file2.txt. You may also want to use the -b options in the command because that will allow it to ignore differences in white space. This command is very useful in programming when you have a backup version of a script that you just modified. If the script no longer works it can help you track down bugs, by looking at backup version of the script that does work.

The kill command

Today we are going to look at a command that you don’t use very much in a Unix environment. The kill command. It is used when an application is stuck or not responding. The basic form of the command is kill [option]... PID. The PID is the process ID, you can get it by using the top command that I talked about previously. The most common option for the kill command is the -9 option. It will kill the process immediately.

Example

kill -9 2342

If you type top again you should see that the process that was associated with that PID is no longer running.

Top

Top is a great command when you are trying to see all of your running processes. It also gives a great deal of other information. It is broken down into four categories - tasks, cpu, mem, and swap. Tasks tells you about the processes that are running. Cpu tells you your processor load. Mem tells you about the total memory you have, how much is being used, and how much is free. Swap Is just like mem except it deals with your swap partition.

Below, you will see all the processes that are running on your computer. It gives most of the info about the processes that you will ever need to know. The main columns I focus on are PID, USER, %CPU, and %MEM. They are to me the most useful. PID is the process ID which is very useful when a program freezes. USER is who is running the program. %CPU is the amount of processing power the program is using. %MEM IS the amount of memory the program is taking up.

Top by itself only gives you information on what is running, but when you combine it with other commands, it makes it very powerful. We will get into that next time.

Searching for a File with a Specific String in the File.

To accomplish this I am going to show you a new command and then use | and grep with it. The command is locate. Your computer has a database that has the path names and files that are publicly accessible on your computer. Locate searches that database for all the pathnames and finds the one you are looking for. It is probably the easiest command to remember and use. You just type locate then what you are looking for.

Example:

locate report

You can also search for all files with a certain extension.

Example:

locate '*.jpg'

You may be saying why would I want to do such a specific search? Well lets say you are a web designer that need to modify a template file for all your websites that contain a certain function. Say the function was called capcha and all your template files end in .tpl. You would use you the | command to combine the two function together to accomplish this.

Example:

locate '*.tpl' | grep -R 'capcha' *

This works by running the locate command and then using grep on just the file names that locate returns. Yes you could just type grep -R 'capcha' *, but you would have to be in the root directory of your hard drive and the grep command would search every file on our computer. That would take a long time to complete, and if you are running the search on a web server you could bring it to its knees.

Grep

Grep is a very powerful command in the Unix world. It allows you to search for words in documents. This is very useful if you are trying to find bugs in code or just tracking down certain files.

The basic structure of a grep statement is below.

grep -R "string you are looking for" *

The -R in the statement means you want to search recursively. Make sure you put the “” around your string or you will get an error when you try to execute the command. The * means that the preceding item will be matched zero or more times.

Their are many more options that can be added into the grep function to make it even more useful. To find them type the following into the command line.

man grep

Pipes

I mentioned previously that Unix uses lots of text files, which are easy to create and manipulate. Unix has a variety of commands to manipulate text files. The way to move text between these commands is with pipes. The pipe symbol is the vertical bar (|) above your Enter key.

For example, suppose you have a text file full of email addresses, with one email address per line. Suppose it’s named “emails.txt” and it’s on your Desktop.

To view the emails, type the following command in the Terminal:
cat ~/Desktop/emails.txt

To view the emails sorted alphabetically, type the following:
cat ~/Desktop/emails.txt | sort

The pipe causes the output of the cat command to be fed into the sort command, then outputted to the screen. You can tack on as many pipe commands as you need.

To view the emails sorted alphabetically and with a prompt between each page, type:
cat ~/Desktop/emails.txt | sort | more

The list of emails “flows” from the cat command to the sort command to the more command, as if going through water pipes.

Viewing text files: cat, head, tail, more, less

Many Unix files are text files. This may seem simplistic, but it’s actually very powerful. Text files are easy to create, read, and manipulate. Many Unix commands are for working with text files.

The first category of Unix commands for working with text files might be called “text viewers”. These includes commands such as cat, head, tail, more, and less. All of these commands are used by typing the command plus the name of a file.

cat — Short for “concatenate”, this command is used to temporarily combine files. But if you use it on only one file, it just displays the file to the screen.

head — This command shows the first 10 lines of a file. (You can adjust the number of lines using the flag -n.)

tail — This command shows the last 10 lines of a file. (You can also use -n here to show more lines.)

more — Like cat, this shows the entire contents of a file, but prompts you for “more?” between each page. Hit the space bar to go to the next page, or use the up and down arrows. Hit “q” to quit.

less — This is another “pager” like more, i.e. it shows one page at a time. It was meant to be a successor or alternative to more and was hence given the opposite name less. It had more features than more (sounds like an Abbott and Costello routine) when it first came out, but now the two commands are about the same.

Here are some examples:

How to view your hosts file, where you can map host names to IP addresses:
cat /etc/hosts

If you have Personal Web Sharing turned on, how to view the last 10 lines of the Apache log file:
tail /var/log/httpd/access_log

How to read the Apache configuration file, one page at a time:
more /etc/httpd/httpd.conf

Invisible files

Files that begin with a dot . are invisible to Mac OS X. They’re also not typically shown in Unix, but it’s easy to change that. For example, if you open Terminal and type ls you’ll see Desktop, Documents, etc. But if you type ls -a you’ll also see additional hidden files that begin with a dot.

The dot is useful for hiding configuration files that might otherwise clutter your view. On a web server like Apache, files beginning with a dot can’t be accessed over the web. That’s why you can safely put configuration options for each directory in an .htaccess file. (If you have a web site, you’ve almost certainly seen .htaccess files.)

In addition to the Unix method of hiding files by beginning them with a dot, Mac OS X also includes its own unique way of hiding files and folders. If you install the Developer Tools CD that came with your Mac, you’ll have access to a SetFile command. You can type SetFile -aV and the name of a file to make the file invisible. You can use SetFile -av (lowercase “v”) to make the file visible again.

In addition, you can see ALL the files on a computer by using Applications like Smultron that have an “Open Hidden” option.

Command history

Besides tab completion, another way to cut down on the amount of typing you do in Unix is the history command. Open Terminal, type history and press Enter. You’ll see a list of Unix commands you previously typed. If you’d like to rerun a command you previously used, you could of course Copy and Paste, or you can type an exclamation point (!) and the number of the command. For instance, if you see that the 10th command in your history is ls -al and you want to run it again, you can type !10 and that command will be re-executed.

If you want to simply rerun the last command, you can type !!.

You can also scroll through the command history by pressing the Up and Down arrows at a blank prompt. As soon as you find a command you want to rerun, you can edit it as needed or simply press Enter.

Tab completion, part 2

In the last post we discussed how tab completion makes typing Unix commands much easier. The downside, as Janssen pointed out, is that tab completion is case-sensitive, so typing cd doc and hitting tab won’t complete cd Documents/. But there’s a way to change this.

In your home folder you’ll find a file called .profile that contains settings for working in the bash shell (which is what you do every time you use Terminal.) It’s sort of the “preferences” file for the command line. The .profile file can do a lot of things, one of which is making tab completion case-insensitive. (We’ll cover more uses of .profile later.)

Because .profile begins with a dot, it’s invisible to most Mac OS X applications. A notable exception is Smultron, a free text editor that has an “Open Hidden” option. You can either use Smultron to open .profile in your home folder, or open Terminal and type open .profile, which should open the file in Apple’s TextEdit. If you don’t have this file, you’ll just create a new empty file by the same name and save it.

The line you’ll add to .profile is the following:

bind "set completion-ignore-case on"

If your .profile contains other text, simply put this at the end on its own line. If you’re creating .profile from scratch, this is the only line you’ll need. Save and close .profile.

After closing and reopening Terminal, your tab completion should now be case-insensitive. If you type cd doc and hit tab, it should fill in cd Documents/.