From HPC

Contents 1 Linux Commands 1.1 Files and Directories 1.1.1 Listing files in a directory 1.1.2 Creating a new directory 1.1.3 Copy files 1.1.4 Copy directories 1.1.5 Moving Files/Directories 1.1.6 Renaming Files/Directories 1.1.7 Delete Files 1.1.8 Delete Directories 1.2 Getting help/documentation about a command 1.2.1 Manual command 1.2.2 Quick help/showing command options 1.3 File commands (viewing/editing/creating) 1.3.1 dos2unix (convert text files created in MS Windows to unix/linux format) 1.3.2 Touch 1.3.3 Cat 1.3.4 Grep 1.3.5 Tail 1.3.6 Pipe output of one program into another 1.3.7 Redirect output to a file (new/append) 1.4 Finding/locating Files and Applications 1.4.1 Which 1.4.2 Find 1.4.2.1 Clever Stuff

Linux Commands

Remember Linux is Case Sensitive. myscript is different to MyScript.

Files and Directories

Listing files in a directory

ls 

Provides compact list of files and directories.

ls -l

Provides expanded list of file and directories including modify dates, file sizes, etc

Creating a new directory

mkdir directoryname

Eample

mkdir mydir

mkdir mydir/newdir

Copy files

cp from to

Examples

cp * mywork/

Copies all files in current directory to a sub directory called mywork.

cp myfile.txt myfile.txt.backup

Copies a file in the same directory and renames that file.

Copy directories

cp -R directory destinantion

The -R switch means recursive, meaning it will copy the contents of subdirectories as well.

Examples

cp -R dir1 dir2

This copy dir1 and it's contents to dir2. The end result is dir2/dir1/files

cp -R dir1/ dir2/

This copies the contents of dir1 (but not actually the direcory it's self) to dir2. Note dir2 must already exist. The end result is dir2/files

Moving Files/Directories

Move file to a different directory

mv myfile dir/to/move/to/

Move files using wild cards

mv file* dir/ (would move any file starting "file" i.e. file1, file2, file-that-should-be-moved)

Move the contents of a directory (files + sub directories) to another directory.

mv mydir/* mydir2/

As above but move to current directory

mv mydir/* ./

Renaming Files/Directories

To rename a file or directory, use the mv (move) command

mv myfile myfile-newname

mv dir dirnewname

Delete Files

To delete a file

rm <filename>
rm myjob.log

To delete multiple files

rm <partfilename>*
rm mywork* (delete all files starting with mywork)
rm *.log (delete all files ending with .log)

Delete Directories

To delete a directory (note the directory should be empty)

rmdir <directory>
rmdir myworkdir

Getting help/documentation about a command

Manual command

Most (if not all) commands will have an associated manual document

man <command>
man ls (get manual page on ls command)
man qsub (get manual page on qsub command)

Quick help/showing command options

Most commands you can run will have a command option that will show you a list of available options and summary help. This is often the command switch --help

<command> --help
ls --help
qsub --help
rm --help

File commands (viewing/editing/creating)

These are tools that allow you to create/modify text files via the command line or in scripts.

dos2unix (convert text files created in MS Windows to unix/linux format)

dos2unix myfile.txt

Touch

Creates empty files ready for writing to

touch myfile.txt

Cat

Reads files and dumps them to the console, you can redirect to a file via >

//read single file
cat myfile

//read multiple files (myfiles1, myfiles2, myfiles3)
cat myfiles*

//read multiple files and dump in a file
cat myfiles* > combined.txt

//append one file to another
cat file1 >> file2

Grep

Provides serach/filter operations on input (case sensitive) use regular expression. See "man grep" for more details

//read file, search for lines containing the word debug 
cat mylog | grep debug

//read file, search for "results returned" and output to new file
cat mywork.txt | grep results\ returned > myresults.txt

Tail

To view the tail/bottom of a file. Very usefull to see the end of a log file.

//show the bottom lines of a file
tail mywork.log

//show the bottom 150 lines of a file
tail -n 150 myfile

//Show the bottom of a file and stays runing to show you new additional lines to the file. This will hold the terminal (i.e. it waits till you press crtl+C before you can run another command)
tail -f mywork.log

Pipe output of one program into another

Using a pipe character ( | ) you can redirect the console output of one file into another.

command1 | command2

Redirect output to a file (new/append)

Using the characters > < >> << you can redirect console out put to files.

//Redirect the output of the ls command to a new file (it will over write and exsiting file of the same name.
ls > mydirectorylisting.txt

//Redirect the output of a cat command and append it to an existing command
cat file2 >> file1

Finding/locating Files and Applications

Which

The which command will show the location in the filesystem of a command. This searches all paths listed in the $PATH enviroment variable. It will also show if the command is available in multiple locations.( If this is the case the top one is the one that would be run if you just typed and excuted that command with out the path)

which gcc
/usr/bin/gcc

which icc
/opt/intel/compiler101/x86_64/bin/icc

Find

The find command allows you to search for files and directories. This is a very powerful command. you can use you search results to execute command. For example deleting the files found, combining the files, etc

//Find any file/directory whose name starts with mywork, searching from current directory
find . -name 'mywork*'

//find all files with the extension .log
find . -name '*.log'

Clever Stuff

Delete files older than 7 days

find /path/to/files* -mtime +7 -exec rm {} \;