BashScriptingAndShellProgramming

01 Bash Programming Course Overview and Downloads

01 Course Overview.mp4

Shell Scripting Course Overview

Course Overview

* Exactly how to get started writing shell scripts;

* What the shebang is and why every one of your scripts need ones;

* How to create and use variables;

* Testing and decision making.

* Command line argument processing;

* Input and output, include user input;

* If statements;

* Exit statuses;

* Functions.

* Wildcards;

* For loops;

* Case statements;

* Logging;

* Debugging tips;

* Bash shell options;

* More.

02 Course Downloads.mp4

Download Layout

shell-scripting-course/debugging/lessons

shell-scripting-course/debugging/practice-exercises

02 Shell Scripting in a Nutshell

01 Section 1 Introduction.mp4

Introduction to Shell Scripting

* The next 2 videos are really important!!!;

* Later, you'll learn more about each topic;

* You'll also learn about other topics;

* Rewatch the next 2 videos.

02 Shell Scripting Part I.mp4

Shell Scripting (Part One)

What You Will Learn

* What scripts are;

* The components that make up a script;

* How to use variables in your scripts;

* How to perform tests and make decisions;

* How to accept command line arguments;

* How to accept input from a user.

Scripts

* Contain a series of commands;

* An interpreter executes commands in the script.

* Anything you can type at the command line, you can put in a script.

* Great for automating tasks.

script.sh

#!/bin/bash

echo "Scripting is fun!"

The interpreter executes the script

$ chmod 755 script.sh

$ ./script.sh

> Scripting is fun!

Shebang

#!/bin/csh

echo "This script uses csh as interpreter"

#!/bin/ksh

echo "This script uses ksh as interpreter"

#!/bin/zsh

echo "This script uses zsh as interpreter"

sleepy.sh

#!/bin/bash

sleep 90

The interpreter executes the script

$ chmod 755 sleepy.sh

$ ./sleepy.sh

$ ./sleepy.sh &

>[1] 12685

$ ps -fp 12685

>UID PID PPID C STIME TTY TIME CMD

>admin1 12685 7273 0 13:32 pts/0 00:00:00 /bin/bash ./sleepy.sh

$ ps -ef | grep 12685 | grep -v grep

>admin1 12685 7273 0 13:32 pts/0 00:00:00 /bin/bash ./sleepy.sh

>admin1 12686 12685 0 13:32 pts/0 00:00:00 sleep 90

$ pstree -p 12685

>sleepy.sh(12685)───sleep(12686)

Shebang or Not to Shebang

* If a script does not contain a shebang the commands are executed using your shell;

* You might get lucky. Maybe. Hopefully.;

* Different shells have slightly varying syntax.

More than just shell scripts

hi.py

#!/usr/bin/python

print "This is a Python script."

The interpreter executes the script

$ chmod 755 hi.py

$ ./hi.py

>This is a Python script.

Variables

* Storage locations that have a name;

* Name-value pairs;

* Syntax:

VARIABLE_NAME='Value'

* Variables are case sensitive;

* By convention variables are uppercase.

Variable Usage

variable1.sh

#!/bin/bash

MY_SHELL="bash"

echo "I like the $MY_SHELL shell."

echo "I like the ${MY_SHELL} shell."

The interpreter executes the script

$ chmod 755 variable1.sh

$ ./variable1.sh

>I like the bash shell.

>I like the bash shell.

variable2.sh

#!/bin/bash

MY_SHELL="bash"

echo "I am ${MY_SHELL}ing on my keyboard."

The interpreter executes the script

$ chmod 755 variable2.sh

$ ./variable2.sh

>I am bashing on my keyboard.

variable3.sh

#!/bin/bash

MY_SHELL="bash"

echo "I am $MY_SHELLing on my keyboard."

The interpreter executes the script

$ chmod 755 variable3.sh

$ ./variable3.sh

>I am bashing on my keyboard.

Assign command output to a variable

variable4.sh

#!/bin/bash

SERVER_NAME=$(hostname)

echo "You are running this script on ${SERVER_NAME}."

The interpreter executes the script

$ chmod 755 variable4.sh

$ ./variable4.sh

>You are running this script on ubuntu18.

Variable Names

Valid:

FIRST3LETTERS="ABC"

FIRST_THREE_LETTERS="ABC"

firstThreeLetters="ABC"

Invalid:

3LETERS="ABC"

first-three-letters="ABC"

first@Tree@Letters="ABC"

Tests

Syntax:

[ condition-to-test-for ]

Example:

[ -e /etc/password ]

File operators (tests)

-d FILE True if file is a directory.

-e FILE True if file exists.

-f FILE True if file exists and is a regular file.

-r FILE True if file readable by you.

-s FILE True if file exists and is not empty.

-w FILE True if file is writeable by you.

-x FILE True if file is executable by you.

String operators (tests)

-z STRING True if string is empty.

-n STRING True if string is not empty.

STRING1 = STRING2

True if the strings are equal.

STRING1 != STRING2

True if the strings are not equal.

Arithmetic operators (test)

arg1 -eq arg2 True if arg1 is equal to arg2.

arg1 -ne arg2 True if arg1 is not equal to arg2.

arg1 -lt arg2 True if arg1 is less than arg2.

arg1 -le arg2 True if arg1 is less or equal to arg2.

arg1 -gt arg2 True if arg1 is greater than arg2.

arg1 -ge arg2 True if arg1 is greater or equal to arg2.

03 Shell Scripting Part 2.mp4

Shell Scripting Part Two.

Making Decisions - The if statement

if [ condition-is-true ]

then

command 1

command 2

command N

fi

condition1.sh

#!/bin/bash

MY_SHELL="bash"

if [ "$MY_SHELL" = "bash" ]

then

echo "You seem to like the bash shell."

fi

The interpreter executes the script

$ chmod 755 condition1.sh

$ ./condition1.sh

>You seem to like the bash shell.

if/else

if [ condition-is-true ]

then

command N1

else

command N2

fi

condition2.sh

#!/bin/bash

MY_SHELL="csh"

if [ "$MY_SHELL" = "bash" ]

then

echo "You seem to like the bash shell."

else

echo "You don't seem to like the bash shell."

fi

The interpreter executes the script

$ chmod 755 condition2.sh

$ ./condition2.sh

>You don't seem to like the bash shell.

if/elif/else

if [ condition-is-true ]

then

command N1

elif [ condition-is-true ]

then

command N2

else

command N3

fi

condition3.sh

#!/bin/bash

MY_SHELL="csh"

if [ "$MY_SHELL" = "bash" ]

then

echo "You seem to like the bash shell."

else

echo "You don't seem to like the bash shell."

fi

The interpreter executes the script

$ chmod 755 condition3.sh

$ ./condition3.sh

>You seem to like the csh shell.

For loop

for VARIABLE_NAME in ITEM_1 ITEM_N

do

command 1

command 2

command N

done

loop1.sh

#!/bin/bash

for COLOR in red green blue

do

echo "COLOR: $COLOR"

done

The interpreter executes the script

$ chmod 755 loop1.sh

$ ./loop1.sh

>COLOR: red

>COLOR: green

>COLOR: blue

loop2.sh

#!/bin/bash

COLORS="red green blue"

for COLOR in $COLORS

do

echo "COLOR: $COLOR"

done

The interpreter executes the script

$ chmod 755 loop2.sh

$ ./loop2.sh

>COLOR: red

>COLOR: green

>COLOR: blue

loop3.sh

#!/bin/bash

# Attention:

# Необходимо внимательно следить за регистром:

"ls *jpg" в данном примере не будут работать, т.к. все имена файлов формата "jpg"

записанны в в верхнем регистре, будет работать команда "ls *JPG".

PICTURES=$(ls *JPG)

DATE=$(date +%F)

for PICTURE in $PICTURES

do

echo "Renaming ${PICTURE} to ${DATE} - ${PICTURE}"

mv ${PICTURE} ${DATE}-${PICTURE}

done

The interpreter executes the script

$ls

>condition1.sh condition2.sh condition3.sh loop1.sh loop2.sh loop3.sh \

P8090001.JPG P8090002.JPG P8090003.JPG P8090004.JPG P8090005.JPG P8090006.JPG

$ ls *JPG

>P8090001.JPG P8090002.JPG P8090003.JPG P8090004.JPG P8090005.JPG P8090006.JPG

$ chmod 755 loop3.sh

$ ./loop3.sh

>Renaming P8090001.JPG to 2020-06-04 - P8090001.JPG

>Renaming P8090002.JPG to 2020-06-04 - P8090002.JPG

>Renaming P8090003.JPG to 2020-06-04 - P8090003.JPG

>Renaming P8090004.JPG to 2020-06-04 - P8090004.JPG

>Renaming P8090005.JPG to 2020-06-04 - P8090005.JPG

>Renaming P8090006.JPG to 2020-06-04 - P8090006.JPG

Positional Parameters

$script.sh parameter.1 parameter.2 parameter.3

$0: "script.sh"

$1: "parameter1"

$2: "parameter2"

$3: "parameter3"

archive_user1.sh

#!/bin/bash

echo "Executing script: $0"

echo "Archiving user: $1"

# Lock the account

passwd -l $1

#Create an archive of the home directory.

tar cf /archive/${1}.tar.gz /home/${1}

The interpreter executes the script

$./archive_user1.sh elvis

>Executing script: ./archive_user1.sh

>Archiving user: elvis

>passwd: password expiry information changed.

>tar: Removing leading '/' from member names

archive_user2.sh

#!/bin/bash

USER=$1 # The first parameter is the user.

echo "Executing script: $0"

echo "Archiving user: $USER"

# Lock the account

passwd -l $USER

#Create an archive of the home directory.

tar cf /archive/${USER}.tar.gz /home/${USER}

The interpreter executes the script

$./archive_user1.sh elvis

>Executing script: ./archive_user1.sh

>Archiving user: elvis

>passwd: password expiry information changed.

>tar: Removing leading '/' from member names

archive_user3.sh

#!/bin/bash

echo "Executing script: $0"

for USER in $0

do

echo "Archiving user: $USER"

# Lock the account

passwd -l $USER

#Create an archive of the home directory.

tar cf /archive/${USER}.tar.gz /home/${USER}

done

The interpreter executes the script

$./archive_user1.sh chet joe

>Executing script: ./archive_user1.sh

>Archiving user: chet

>passwd: password expiry information changed.

>tar: Removing leading '/' from member names

>Archiving user: joe

>passwd: password expiry information changed.

>tar: Removing leading '/' from member names

Accepting User Input(STDIN)

The read command accepts STDIN.

Syntax:

read -p "PROMPT" VARIABLE

archive_user4.sh

#!/bin/bash

read -p "Enter a user name: " USER

echo "Archiving user: $USER"

# Lock the account

passwd -l $USER

#Create an archive of the home directory.

tar cf /archives/${USER}.tar.gz /home/${USER}

The interpreter executes the script

$./archive_user1.sh chet joe

>Executing script: ./archive_user4.sh

Enter a user name: Joe

>Archiving user: Joe

>passwd: password expiry information changed.

>tar: Removing leading '/' from member names

Summary

#!/path/to/interpreter

VARIABLE_NAME="Value"

$VARIBLE_NAME

${VARIBLE_NAME}

VARIBLE_NAME=$(command)

if [ condition-is-true ]

then

commands

elif [ condition-is-true ]

then

commands

else

commands

fi

for VARIABLE_NABE in ITEM_1 ITEM_N

do

command 1

command 2

command N

done

Positional Parameters:

$0, $1, $2, ... $9

$@

Comments start with #.

Use read to accept input.

03 Return Codes and Exit Statuses

01 Exit Statuses and Return Codes.mp4

Exit Statuses and Return Codes.

What You Will Learn.

* How to check the exit status of a command;

* How to make decisions based on the status;

* How to use exit statuses in your own scripts.

Exit Status/Return Code

* Every command returns an exit status;

* range from 0 to 255;

* 0 = success;

* Other than 0 = error condition;

* Use for error checking;

* Use man or info to find meaning of exit status.

Checking the Exit Status

* $? contains the return code of the previously executed command.

$ ls /not/here

> ls: невозможно получить доступ к '/not/here': Нет такого файла или каталога

$ echo "$?"

> 2

ping1.sh

#!/bin/bash

HOST="google.com"

ping -c 1 $HOST

# shellcheck disable=SC2181

if [ "$?" -ne "0" ]

then

echo "$HOST unreachable."

fi

$ chmod 755 ping1.sh

$ ./ping1.sh

> ping: google.com: Неизвестное имя или служба

> google.com unreachable.

ping2.sh

#!/bin/bash

HOST="google.com"

ping -c 1 $HOST

RETURN_CODE=$?

if [ "$RETURN_CODE" -ne "0" ]

then

echo "$HOST unreachable."

fi

$ chmod 755 ping2.sh

$ ./ping2.sh

> ping: google.com: Неизвестное имя или служба

> google.com unreachable.

&&=AND

* &&=AND

$ mkdir /tmp/bak $$ cp test.txt /tmp/bak # Not work

* ||=OR

$ cp test.txt /tmp/bak/ || cp test.txt /tmp

ping3.sh

#!/bin/bash

HOST="google.com"

ping -c 1 $HOST && echo "$HOST reachable."

$ chmod 755 ping3.sh

$ ./ping3.sh

> ping: google.com: Неизвестное имя или служба

ping4.sh

#!/bin/bash

HOST="google.com"

ping -c 1 $HOST || echo "$HOST unreachable."

$ chmod 755 ping4.sh

$ ./ping4.sh

> ping: google.com: Неизвестное имя или служба

> google.com unreachable.

The semicolon

* Separate commands with a semicolon to ensure they all get executed.

$ cp test.txt /tmp/bak/ ; cp test.txt /tmp

# Same as:

$ cp test.txt /tmp/bak/

$ cp test.txt /tmp

Exit Command

* Explicitly define the return code

- exit 0

- exit 1

- exit 2

- exit 255

- etc..

* The default value is that of the last command executed.

ping5.sh

#!/bin/bash

HOST="google.com"

ping -c 1 $HOST

# shellcheck disable=SC2181

if [ "$?" -ne "0" ]

then

echo "$HOST unreachable."

exit 1

fi

exit 0

$ chmod 755 ping5.sh

$ ./ping5.sh

> ping: google.com: Неизвестное имя или служба

> google.com unreachable.

Summary

* All command return an exit status;

* 0 - 255;

* 0 = success;

* Other than 0 = error condition;

* $? contains the exit status;

* Decision making -if, &&, ||;

* exit.

Exit Status Demo.

$ ping -c 1 google.com

> PING google.com(lg-in-x71.1e100.net (2a00:1450:4010:c08::71)) 56 data bytes

> 64 bytes from lg-in-x71.1e100.net (2a00:1450:4010:c08::71): icmp_seq=1 ttl=107 time=78.2 ms

>

> --- google.com ping statistics ---

> 1 packets transmitted, 1 received, 0% packet loss, time 0ms

> rtt min/avg/max/mdev = 78.291/78.291/78.291/0.000 ms

> admin1@ubuntu18:~$ echo $?

> 0

> admin1@ubuntu18:~$ ping -c 1 -W 1 amazon.com

> PING amazon.com (176.32.103.205) 56(84) bytes of data.

> 64 bytes from 176.32.103.205: icmp_seq=1 ttl=230 time=214 ms

>

> --- amazon.com ping statistics ---

> 1 packets transmitted, 1 received, 0% packet loss, time 0ms

> rtt min/avg/max/mdev = 214.762/214.762/214.762/0.000 ms

> admin1@ubuntu18:~$ echo $?

> 0

$ ping -c 1 -W 1 amazon.com.blah

> ping: amazon.com.blah: Неизвестное имя или служба

$ echo $?

> 2

$ man ping # See when maybe this exit code possible

$ mkdir /tmp/jason/bak && cp -v /etc/hosts /tmp/jason/bak

> mkdir: невозможно создать каталог «/tmp/jason/bak»: Нет такого файла или каталога

$ echo $?

> 1

$ mkdir -p /tmp/jason/bak && cp -v /etc/hosts /tmp/jason/bak

> '/etc/hosts' -> '/tmp/jason/bak/hosts'

$ cp -v /etc/hosts /tmp/bak/ || cp -v /etc/hosts /tmp/

> '/etc/hosts' -> '/tmp/bak/hosts'

> $ cp -v /etc/hosts /tmp/ || cp -v /etc/hosts /tmp/bak/

> '/etc/hosts' -> '/tmp/hosts'

$ ls /etc/linux-release ; hostname

> ls: невозможно получить доступ к '/etc/linux-release': Нет такого файла или каталога

> ubuntu18

$ ls /etc/hosts ; hostname

> /etc/hosts

> ubuntu18

$ ls /etc/hosts ; hostname ; uptime

> /etc/hosts

> ubuntu18

> 17:40:06 up 7:15, 1 user, load average: 0,26, 0,22, 0,20

04 Shell Functions

01 Functions, Part I

What You Will Learn

* Why to use functions;

* How to create them;

* How to use them;

* Variable scope;

* Function Parameters;

* Exit statuses and return codes.

Why use functions? (Keep it DRY!)

* Don't repeat yourself! Don't repeat yourself!

* Write once, use many times;

* Reduces script length;

* Single place to edit and troubleshoot;

* Easier to maintain.

Functions

* If you're repeating yourself, use a function;

* Reusable code;

* Must be defined before use;

* Has parameter support.

Creating a function

function function-name() {

# Code goes here.

}

function-name() {

# Code goes here.

}

Calling a function

function1.sh

#!/bin/bash

function hello() {

echo "Hello!"

}

hello

$ chmod 755 function1.sh

$ ./function1.sh

>Hello!

Functions can call other functions.

function2.sh

#!/bin/bash

function hello() {

echo "Hello!"

now

}

function now() {

echo "It's $(date)"

}

hello

$ chmod 755 function2.sh

$ ./function2.sh

>Hello!

>It's Ср июн 10 10:01:27 MSK 2020

DoNot do this...

function3.sh

#!/bin/bash

function hello() {

echo "Hello!"

now

}

hello

function now() {

echo "It's $(date)"

}

$ chmod 755 function3.sh

$ ./function3.sh

>Hello!

>./function3.sh: строка 5: now: команда не найдена

Position Parameters

* Functions can accept parameters;

* The first parameter is stor ed in $1;

* The second parameter is stored in $2, etc.

* $@ contains all of the parameters;

* Just like shell scripts.

$0 = the script itself, not function name.

function4.sh

#!/bin/bash

function hello() {

echo "Hello! $1"

}

$ chmod 755 function4.sh

$ ./function4.sh

>Hello! Jason

function5.sh

#!/bin/bash

function hello()

{

# shellcheck disable=SC2068

for NAME in $@

do

echo "Hello! $NAME"

done

}

hello Jason Dan Ryan

>Hello! Jason

>Hello! Dan

>Hello! Ryan

Variable Scope

* By default, variables are global;

* Variables have to be defined before used.

###########################################

GLOBAL_VAR=1

# GLOBAL_VAR is available in the function.

my_function

###########################################

# GLOBAL_VAR is NOT available in the function.

my_function

GLOBAL_VAR=1

###########################################

function6.sh

#!/bin/bash

my_function()

{

GLOBAL_VAR=1

}

# GLOBAL_VAR not available yet.

echo $GLOBAL_VAR

my_function

# GLOBAL_VAR is now available.

echo $GLOBAL_VAR

>$ chmod 755 function6.sh

>$ ./function6.sh

>

>1

Local Variables

* Can only be access within the function;

* Create using the local keyword.

- local LOCAL_VAR=1;

* Only function can have local variables;

* Best practice to keep variables local in functions.

02. Functions Part 2.

Exit Status(Return Codes)

* Functions have an exit status;

* Explicitly:

- return <RETURN_CODE>.

* Implicitly:

- The exit status of the last command executed in the function.

* Valid exit codes range from 0 to 255;

* 0 = success;

* $? = the exit status.

my_function

echo $?

##################################################################

$# - общее количество параметров переданных скрипту

$* - все аргументы переданыне скрипту(выводятся в строку)

$@ - тоже самое, что и предыдущий, но параметры выводятся в столбик

$! - PID последнего запущенного в фоне процесса

$$ - PID самого скрипта

##################################################################

function1.sh

#!/bin/bash

function backup_file ()

{

if [ -f $1 ]

then

BACK="/tmp/$(basename ${1}).$(date +%F).$$"

echo "Backing up $1 to ${BACK}"

cp $1 $BACK

fi

}

backup_file /etc/hosts

if [ $? -eq 0 ]

then

echo "Backup succeeded!"

fi

>$ chmod 755 function1.sh

>$ ./function1.sh

>Backing up /etc/hosts to /tmp/hosts.2020-06-10.9200

>Backup succeeded!

function2.sh

#!/bin/bash

function backup_file ()

{

if [ -f $1 ]

then

local BACK="/tmp/$(basename ${1}).$(date +%F).$$"

echo "Backing up $1 to ${BACK}"

# The exit status of the function will

# be the exit status of the cp command.

cp $1 $BACK

else

# The file does not exist.

return 1

fi

}

>$ chmod 755 function1.sh

>$ ./function1.sh

>Backing up /etc/hosts to /tmp/hosts.2020-06-10.10308

function3.sh

#!/bin/bash

function backup_file ()

{

if [ -f $1 ]

then

local BACK="/tmp/$(basename ${1}).$(date +%F).$$"

echo "Backing up $1 to ${BACK}"

# The exit status of the function will

# be the exit status of the cp command.

cp $1 $BACK

else

# The file does not exist.

return 1

fi

}

backup_file /etc/hosts

# Make a decision based on the exit status.

if [ $? -eq 0 ]

then

echo "Backup succeeded!"

else

echo "Backup failed!"

# About the script and return a non-zero exit status.

exit 1

fi

>$ chmod 755 function3.sh

>$ ./function3.sh

>Backing up /etc/hosts to /tmp/hosts.2020-06-10.10551

>Backup succeeded!

Summary

* DRY;