Kunal Gohrani

Mastering Shell Scripting in Linux: Advanced Concepts and Automation

From Loops to Functions: Mastering Advanced Shell Scripting in Linux for Efficient Automation

Kunal Gohrani's photo
Kunal Gohrani
·

9 min read

Mastering Shell Scripting in Linux: Advanced Concepts and Automation

Table of contents

Welcome back to the continuation of our two-part series on shell scripting in Linux. In the previous blog, we covered the basics of shell scripting, variables, reading input from users, and conditional statements. If you haven't read it, I highly recommend checking it out here to establish a strong foundation for the advanced concepts we'll explore in this blog.

Introduction to Advanced Shell Scripting Concepts

In this section, we'll explore advanced shell scripting concepts, including loops and functions, to streamline tasks and enhance productivity. Get ready for hands-on examples and practical applications as we tackle two real-world automation tasks: creating directories based on a range and automating backups. With these powerful tools at your disposal, you'll be well-equipped to unleash the full potential of shell scripting and elevate your skills as a system administrator or DevOps engineer. Let's dive in and embark on this exciting journey of automation in Linux! 🚀💻

Loops in Shell Scripting

Loops allow you to execute a block of code repeatedly. Let's explore the syntax of each loop with code snippets:

For Loop

While there are other forms of for loop in shell scripting, in this post, we will learn what is called the C-style for loop.

In shell scripting, the C-style for loop is a powerful construct that allows you to perform repetitive tasks efficiently. Modelled after the for loop syntax in the C programming language, it provides a familiar structure for iterating over a range of values or performing a specific number of iterations.

The C-style for loop consists of three main components: initialization, condition, and increment/decrement. These components work together to control the flow of the loop and execute a block of code repeatedly until a specific condition is met.

Syntax of the C-Style For Loop in Bash:

for (( initialization; condition; increment/decrement ))
do
    # Code to be executed in the loop
done

Components of for loop:

  1. Initialization: The initialization is an expression that initializes the loop control variable or any other variables used in the loop.

  2. Condition: The condition is an expression that is evaluated before each iteration. If the condition evaluates to true, the loop body is executed. If the condition evaluates to false, the loop terminates, and program execution continues after the loop.

  3. Increment/Decrement: Increment/decrement is an expression that modifies the loop control variable or any other variables used in the loop.

  4. Loop Body: The loop body contains the code to be executed within the loop. It can consist of one or more commands or statements.

Example Usage:

#!/bin/bash

for (( i=1; i<=5; i++ ))
do
    echo "Iteration $i"
done

If you run this script, the output will be like:

The iteration starts from 1 and continues till 5 as given in the condition of for loop.

While Loop

In shell scripting, the while loop allows you to repeat a block of code as long as a specified condition is true. It's like a "do this while that is true" scenario. Here's how it works:

  1. Condition: You set a condition that is checked before each iteration of the loop. If the condition is true, the loop continues executing. If it's false, the loop ends, and the script moves on.

  2. Loop Body: Inside the loop, you write the code you want to repeat. It can be a single command or multiple commands.

The while loop is useful when you need to repeat a task until a certain condition is met. It's often used when the number of iterations is not fixed or when you want to keep running a block of code until a specific event occurs.

Example of while loop:

#!/bin/bash

counter=1

while [ $counter -le 5 ]
do
    echo "Iteration $counter"
    counter=$((counter + 1))
done

In this example, we start with a counter set to 1. The loop continues executing as long as the counter value is less than or equal to 5. Inside the loop, we print the current iteration number and increment the counter by 1 using counter=$((counter + 1)).

When you run the above script, it will output:

Loop Control: Break and Continue Statements:

In addition to the basic loop structures, shell scripting provides loop control statements to modify the flow of the loop execution.

break statement

The break statement allows you to terminate the loop prematurely. When encountered, it immediately exits the loop and continues with the next line of code after the loop.

Example of break statement:

#!/bin/bash

for (( i=1; i<=5; i++ ))
do
    if [[ "$i" -eq 3 ]]; then
        break # The loop will stop at 3 and wont print 3,4,5
    fi
    echo "Iteration $i"

done

When you run the script, you will see this as output:

continue statement

The continue statement allows you to skip the rest of the current iteration and proceed to the next iteration of the loop.

Example of continue statement:

#!/bin/bash

counter=1

while [ $counter -le 5 ]
do
    # The loop will be skipped at 3 and will move to 4
    if [[ $counter -eq 3 ]]; then
        counter=$((counter + 1))
        continue
    fi
    echo "Iteration $counter"
    counter=$((counter + 1))
done

When you run the script, the output will be:

As you can see, Iteration 3 was not printed because we skipped 3 in the code.

The continue statement is helpful when you want to skip specific iterations based on a condition, without terminating the entire loop.

Automation Task 1

Ok, now we are ready to face our first automation task. Imagine that you have been asked to create multiple directories with the same starting name and with a pattern such as day0, day5, day10, day15, day20, upto day1000. Now obviously you wouldn't want to sit and keep creating directories one by one right?

Now, someone who doesn't know Linux shell would take hours to create these directories. But do you know that this task can be done in minutes?

So let's write a script that does exactly this.

Script:

#!/bin/bash

DirectoryName=$1
StartRange=$2
EndRange=$3
Increment=$4

# Validate inputs: check if DirectoryName is a valid string and if StartRange and EndRange are integers
# ^[0-9]+$ is a regular expression which is used to check if the variable contains only integers, if no,
# then it's a invalid input
if [[ -z $DirectoryName || !( "$StartRange" =~ ^[0-9]+$ ) || !( "$EndRange" =~ ^[0-9]+$ ) || !( "$Increment" =~ ^[0-9]+$ ) ||  "$StartRange" -gt "$EndRange" ]]; then
    echo "Error: Invalid inputs. Please provide a valid DirectoryName, StartRange, EndRange, and Increment."
    exit 1
fi

# Create directories in the current folder
currentDirectory=$(pwd)
for (( i=$StartRange; i<=$EndRange; i+=$Increment )); do
    directory="${DirectoryName}${i}"
    if [[ -d "${currentDirectory}/${directory}" ]]; then
            continue
    fi

    mkdir "${currentDirectory}/${directory}"
done

echo "Directories created successfully."
exit 0

When you try to run this script with inputs: day 0 100 5, it creates the folders from 0 to 100 with an increment of 5 like this:

Congratulations, you have now written the first shell script that automates your work and reduces the manual work required. 🤩🥳

You can also modify the shell script to create text files instead of directories. Try it out and let me know on LinkedIn if you face any trouble.

BONUS

Did you know there's a one-line command that can do the exactly same thing? Try out mkdir day{0..100..5} 😎

Check your directory after running this command, you will see all 20 folders were created from day0 to day 100.

Functions in Shell Scripting

In shell scripting, a function is a named block of code that performs a specific task. Functions help you avoid code duplication and make your scripts more modular and readable. By defining functions, you can reuse the same piece of code in different parts of your script.

Syntax of Functions in Shell Scripting:

To create a function, you need to follow this syntax:

function_name() {
    # Function code goes here
}

Let's make a Hello function which prints out "Hello User!" once it's called.

Hello() {
        echo "Hello User!"
}

Seems simple enough right? But wait, how are you gonna use call this function? That's what we will be seeing in the next section.

Calling a Function

Once you've defined a function, calling it is as simple as using its name followed by the arguments (if any). Here's how you can call the Hello function in the script:

#!/bin/bash
Hello() {
        echo "Hello User!"
}

Hello

That's it! You just need to use the name of the function along with any arguments if needed. When the script is invoked, this will be printed on the terminal:

Passing Arguments to Functions

Functions can accept arguments to perform tasks based on different input values. We use special variables ($1, $2, etc.) to access the arguments passed to the function. These variables represent the values of the first, second, and subsequent arguments. Let's write a script that takes two numbers as input, and sends the numbers to add function, which takes in two numbers as arguments, adds them, and echo's back the results.

add(){
        firstNumber=$1
        secondNumber=$2
        sum=$(( $firstNumber + $secondNumber ))
        echo "Sum of $firstNumber and $secondNumber is $sum"
}

read -p "Enter First Number:" number1
read -p "Enter Second Number:" number2
add $number1 $number2

When the script is run, the output looks like this:

Easy isn't it? Now you can also try to include other functions in the script such as subtraction, multiplication, and division and try to run it.

Automation Task 2

Congratulations on reaching this part of the blog! 🥳🔥 Now, let's explore a practical use case of shell scripting: creating backups. This example showcases the power of shell scripting in creating backups.

We will be writing a script that takes in two arguments, source_directory, and target_directory. The files in the source_directory will be then compressed bytar command, and the file name of the compressed file will be in the format <year><month><day><hour><minute><second>.tar.gz. Finally, the compressed file will be stored in the target directory.

Please try to write this script on your own before looking at the code below.

#!/bin/bash

source_directory="$1"
target_directory="$2"
timestamp=$(date +%Y%m%d%H%M%S)
tar_file="${target_directory}/${timestamp}.tar.gz"

# Validate input: check if source directory exists
if [ ! -d "$source_directory" ]; then
    echo "Error: Source directory '$source_directory' does not exist."
    exit 1
fi

# Create target directory if not present
if [ ! -d "$target_directory" ]; then
    mkdir -p "$target_directory"
    if [ $? -ne 0 ]; then
        echo "Error: Failed to create target directory '$target_directory'."
        exit 1
    fi
fi

# Create tar compressed file of source directory contents
tar -czf "$tar_file" -C "$source_directory" .

if [ $? -eq 0 ]; then
    echo "Backup created successfully at: $tar_file"
else
    echo "Error: Failed to create backup."
    exit 1
fi

exit 0

I have a scripts folder, which I want to back up to script_backup folder, when I run the script the output will be like this:

Do you see? After the script backup.sh was run, a new folder called "script_backup" was created, inside which, we had our backup tar file.

Congratulations, You have now completed 2 automation tasks using shell scripting, pat yourself on the back! 🔥

via GIPHY

Conclusion

Congratulations on completing this two-part journey of mastering shell scripting in Linux! 🎉 Throughout this blog, we explored powerful concepts like loops, and functions, and completed two real-world automation tasks. From efficiently processing data with loops to creating modular and reusable code with functions, you now have the tools to streamline tasks and boost productivity.

We even ventured into the world of real-world scripting, creating directories with a range and creating backup scripts like seasoned Linux experts.

Stay tuned for more exciting content on my blog. If you have any questions or need assistance, feel free to connect with me on LinkedIn. Happy scripting! 🚀💻

Devops#90daysofdevopsLinuxlinux for beginnersshell