Learning Scripting the Hard Way

February 15, 2017 5 minutes read

linux • bash • regex • command-line

In my first semester, I dipped my toe in the water and started learning programming via different courses, blogs and other online sources, along with the “Introduction to Programming” course obviously. I also installed Linux for the first time - ubuntu obviously (via Pclub guide). I had no idea how to install anything else at this point. For exploring what all things we could do with Linux, I was learning via a few sources:

This semester started with a blast. From dipping my toe into the water, I am now swimming into the deep end. I have finally realized what people mean when they say “Learn X the hard way”. Well, however difficult it is to learn something the hard way by just delving into it and reverse-engineer it, it is still very effective and I am pretty sure I am going to keep doing things the same way for the rest of my life :P

I have taken a project offered by Pallav Agarwal under Association of Computing Activities (A club under Computer Science Department). The title of the project is “Linux From Scratch”, sounds simple enough right? Believe me, it’s not!!

There are so many pre-requisites to actually accomplish the goal of this project, that it does not seem like a semester long project. Also, being a rookie programmer, it was a big challenge for to fulfil all these. My first obstacle was to learn scripting, Linux utilities and command-line.

Setting up

To kick-off the project, Pallav set up VMs for us and instructed that we ensure all the project related work is kept updated on our assigned VMs. He also asked us to setup a simple Node.js server (running forever) to keep a record of what we had done and learnt so far.

Logging in to VM and setting it up was not very hard but installing node and setting up a Node.js server was an initial challenge (since I had never worked with any similar technologies). Although, I was exploring web programming via MDN Docs in my first semester. I don’t know whether this was directly helpful or not but I was able to figure out the details and setup the server.

Next question that we had to answer was - “Which editor should I pick?” - Vim or Emacs. I went with Vim, I was already using it and I liked it very much.

I went through the Vim documentation in detail and setup my initial Vimrc. My vim at this point looks something like this:

Stimulating Command Line Session

After setting everything up, Pallav gave us the task to start reading about loops and conditionals in bash and the time he gave us was 30 minutes for it (yes, you heard me right!).

I got onto Google and read up the bash docs + multiple blogs for the session that we were going to have soon. After 30 minutes had elapsed, all of us set up a meeting and went on a hangouts call for an interesting command-line quiz/session.

Pallav gave un interesting tasks to accomplish via command line, some of them were:

Create a recursive directory structure where there are ‘x’ directories at ‘x’ depth, e.g. (if x = 3)

1
├── 1
└── 2
    ├── 1
    ├── 2
    └── 3

# Answer
$ x=3; for i in `seq 1 $x`; do for j in `seq 1 $i`; do mkdir $j; done; cd $i; done

Given a directory, recursively change the filenames of each file from <name> to a<name>a

# Answer
$ for i in $(find 1 -type f); do d=$(echo $i | rev | cut -d '/' -f 2- | rev); name=$(echo $i | rev | cut -d '/' -f 1 | rev); mv $i "$d/a${name}a"; done

We went through multiple questions like the above, all based on a 30 minute reading :)

Regular Expressions + Grep/Sed

The next task was to study regular expressions, starting with Regexone and then moving to advanced tutorials. Later moving to grep and sed. We had about 24 hours to go through all this, next day would be another quiz/session.

During this time, I learnt the value of manpages. They are a trove of information for learning the command line. Each command has many flags and utilities, all of which is detailed in the manpages.

Also, while exploring command line, I came across a very interesting Github repository, one should definitely read this - The Art of Command Line.

Xargs + Parallelism

Once we were comfortable with command line in general (in just 3 days), our next task was to read and understand xargs, its features and utilities, also how we can use it for parallelism and removing loops from our code.

Our initial task to demonstrate our understanding was to write a script to fetch the students data from Student Search portal (based on list of roll numbers) via curl command. And we should not use loops any where in this script, it should be done using xargs (and in parallel).

Student Search in Bash

Our final task to demonstrate our understanding of Linux command line, scripting and utilities, Pallav gave us the task to write a complete student search in bash. He gave us 3 days to complete this assignment.

I was able to write the complete student in about 200 lines of bash code with complete search functionality. The code for the same can be found here.

Some screenshots, please don’t judge the UI (it is still a terminal).

All the tasks shown within this page were completed in about a week (so, you can guess why the topic is - “Learning Scripting the Hard Way”).

We continued forward to Python after this, i.e. Learning Python the Hard Way.