If you are following me on Twitter, you have already seen a lot of (re)tweets related to Aadhaar. For the people first time hearing this term, it is a 12 digit unique identification number provided by the Unique Identification Authority of India (UIDAI). It is also the world’s largest bio-metric ID system. It is supposed to be a voluntary service.

From the very beginning, this project tried to hide the details from the Indian citizens. Let it be privacy advocates or security researchers or human rights activists, everyone predicted that this will become a monster, a mass surveillance system, a tool of choice of the power hungry dictators.

Like any other complex system, the majority of the people only see the advertisements from the government and completely miss all the problems and horror stories this project is creating. Here are a few links below for the interested people to read.

• IRC session log from Kiran Jonnalagadda
• Karana (detailed posts related to various parts of the project) Must read site.
• Aadhaar fail Details of scams/failures/deaths/other issues from Aadhaar project

Neither my wife, nor our daughter has an Aadhaar (I also don’t have one), that means Py (our daughter) did not get admission to any school last year.

Whenever security researchers or journalists tried to report on the project, the UIDAI tried to hide behind denials and police complaints against the journalists or researchers. There are various reports on how one can get access (both read/write) to the actual production database with as little as $10-30. We now have examples of terrorist organizations having access to the same database. The UIDAI kept telling how this is an unhackable technology and for security they have a 13 feet wall outside of the data center which in turn will keep all hackers away.

They have already build 360 degree databases on top of Aadhaar, and now they are trying to link DNA to the same system.

The current government of India tried their level best to argue in the Supreme Court of India to tell that Indians don’t have any rights to privacy. But, thankfully they failed in this effort, and the Supreme Court ruled privacy as a fundamental right. We are now waiting for the judgment on the Aadhaar (which will hopefully come out in the next few weeks).

Meanwhile, the evil nexus is pushing down Aadhaar to the throats of the Indian citizens and Pakistani spies and gods.

A few days ago, in an event in Jaipur, they asked Edward Snowden the following question.

How big of an issue is privacy?

The answer started with from where that argument comes from.

The answer is that Nazi Germany. The nazi minister of propaganda Joseph Goebbels did this. Because he was trying to change the conversation away from “What are your rights?” and “What evidences must the government show?” to violet them, to intrude into your private life and instead said “Why do you need your rights?”, “How can you justify your rights?”, “Isn’t strange that you are invoking your rights? Isn’t that unusual?”. But, in a free society this is the opposite of the way it is supposed to work. We don’t need to explain why you have a right. You don’t need to explain why it is valuable, why you need it. It is for the government to explain why you don’t deserve it. They go to a court, they show that you are a criminal. This is increasingly falling out of favor, because the governments and companies think that it is inefficient. It is too much work. Life would be easier, life would be more convenient for them, life would be more profitable for them if we didn’t have any rights at all.

But, privacy isn’t about something to hide, privacy is about something to protect. And that is the very concept of liberty. It is the idea that there can be some part of you, of your life, of your ideas that belong to you, not to society. And you get to make the decision about who you share that with. -- Edward Snowden

Why are we reading this in your blog?

This might a question for many of you. Why are reading this in a blog post or in a planet? Because we, the people with the knowledge of technology are also part of these evil plans. We now know about many private companies taking part with their local government to build 360 degree profiles, to track the citizens and to run the mass surveillance systems. For example, related to Aadhaar, for the last 4 years, Google silently pushed the Aadhaar support phone number (which now UIDAI is trying to stay away from) to every Google Android phone in India. When they got caught red handed, they claimed that they did it inadvertently. Finacle software by Infosys denies creation of bank accounts without Aadhaar. Microsoft is working to link Skype with Aadhaar. Bill Gates is trying to push the idea that Aadhaar is all good, and does not have any issues.

What can you do?

You can start by educating yourself first. Read more about the technologies which controls our lives. Have doubt about the things and try to understand how they actually work. Write about them, ask questions to the people in power. Talk about the issues to your friends and family.

This is not gong to be an easy task, but, we all should keep fighting back to make sure of a better future for our next generation.

I'm quite fond about having a google-free phone. I run a lineage OS without g-apps, use my own caldav and carddav server to sync my contacts and calendars, can find every software I need in the F-Droid app and live happy ever since.

However, what has bugged me was synchronization of personal photos and videos. I did several attempts to solve that using webdav (owncloud/nextcloud), but my phone was not happy with the available clients and the battery would not last very long.

I now have a setup that allows me secure synchronization that I'm quite happy with, using rsync from my phone to server. It's simple and slim and feels just too familiar to be a bad idea.

Termux - Terminal Emulator and Linux Environment

Termux is what I've been looking for on Android for a long time. Native Linux on my phone would be priceless, but the ease of use and installation makes Termux a more than good compromise. It comes with a nifty packet manager and allows me to run small Python and R scripts including web-scraping etc. on the fly.

You can get Termux for free in the Google Playstore, the extensions however will cost around 2\$ (which is okay, you'll support some really good work!). Using the F-Droid app you can install the extensions for free.

I also installed the extension Termux:Widget that allows to start scripts via single tap from home screen - a very handy addition, I use it to reboot my raspberry pi or wake-on-lan my computer that's in the basement.

Set up rsync

To set up Termux for this task, install rsync first:

pkg install rsync

Generate a key-pair (I just use the defaults):

ssh-keygen

Per default this will create the files id_rsa and id_rsa.pub in your ~/.ssh directory. Make sure to put the contents of id_rsa.pub into the ~/.ssh/authorized_keys file on your target server.

And set up the Termux storage:

termux-setup-storage

Create a folder for the Termux:Widget shortcuts and create the script:

mkdir .shortcuts
echo "rsync -h -r --info=progress2 ~/storage/dcim/* robin@<myserver>:/<path_to_my_rsync_folder>" > .shortcuts/sync

Conclusion

With the Termux widget this simple setup allows stable and quick sync of my data that leaves me with a few worries less. In fact I set this up on my wife's phone as well, since it's working so nicely.

I have been working on a pet project to write a File Indexer, which is a utility that helps me to search a directory for a given word or phrase.

The motivation behind to build this utility was so that we could search the chat log files for dgplug. We have a lot of online classes and guest session and at time we just remember the name or a phrase used in the class, backtracking the files using these are not possible as of now. I thought I will give stab at this problem and since I am trying to learn golang I implemented my solution in it. I implemented this solution over a span of two weeks where I spent time to upskill on certain aspects and also to come up with a clean solution.

Exploration

This started with exploring a similar solution because why not? It is always better to improve an existing solution than to write your own. I didn’t find any which suits our need so I ended up writing my own. The exploration to find a solution led me to discover few of the libraries that can be useful to us. I discovered fulltext and Bleve.

I found bleve to have better documentation and really beautiful thought behind it. They have a very minimal yet effective thought process with which they designed the library. At the end of it I was sure I am going to use it and there is no going back.

Working On the Solution

After all the exploration I tried to break the problem I have into smaller problems and then to follow and solve each one of them. So first one was to understand how bleve works, I found out that bleve creates an index first for which we need to give it the list of files. The way the index is formed is basically a map structure behind the back where you give the id and content to be indexed. So what could be a unique constraint for a file in a filesystem? The path of the file I used it as the id to my structure and the content of my file as the value.

After figuring this out I wrote a function which takes the directory as the argument and gives back the path of each file and the content of each file. After few iteration of improvement it diverged into two functions one is responsible to get the path of all the files and the other just reads the file and get the content out.

func fileNameContentMap() []FileIndexer {
	var ROOTPATH = config.RootDirectory
	var files []string
	var filesIndex FileIndexer
	var fileIndexer []FileIndexer

	err := filepath.Walk(ROOTPATH, func(path string, info os.FileInfo, err error) error {
		if !info.IsDir() {
			files = append(files, path)
		}
		return nil
	})
	checkerr(err)
	for _, filename := range files {
		content := getContent(filename)
		filesIndex = FileIndexer{Filename: filename, FileContent: content}
		fileIndexer = append(fileIndexer, filesIndex)
	}
	return fileIndexer
}

This forms a struct which stores the name of the file and the content of the file. And since I can have many files I need to have a array of the struct. This is how the transition of moving from a simple data structure evolves into complex one.

Now I have the utility of getting all files, getting content of the file and making an index.

This forms a crucial step of what we are going to achieve next.

How Do I Search?

Now since I am able to do the part which prepares my data the next logical stem was to retrieve the searched results. The way we search something is by passing a query so I duck-typed a function which accepts a string and then went on a spree of documentation to find out how do I search in bleve, I found a simple implementation which returns me the id of the file which is the path and match score.

 func searchResults(indexFilename string, searchWord string) *bleve.SearchResult {
	index, _ := bleve.Open(indexFilename)
	defer index.Close()
	query := bleve.NewQueryStringQuery(searchWord)
	searchRequest := bleve.NewSearchRequest(query)
	searchResult, _ := index.Search(searchRequest)
	return searchResult
}

This function opens the index and search for the term and returns back the information.

Let’s Serve It

After all that is done I need to have a service which does this on demand so I wrote a simple API server which has two endpoints index and search.  The way mux works is you give the enpoint to the handler and which function has to be mapped with it. I had to restructure the code in order to make this work. I faced a very crazy bug which when I narrowed down came to a point of a memory leak and yes it was because I left the file read stream open so remember when you Open always defer Close.

I used Postman to heavily test it and it war returning me good responses. A dummy response looks like this:

 [{"index":"irclogs.bleve","id":"logs/some/hey.txt","score":0.6912244671221862,"sort":["_score"]}]

Missing Parts?

The missing part was I didn’t use any dependency manager which Kushal pointed out to me so I landed up using dep to do this for me. The next one was the best problem and that is how do auto-index a file, which suppose my service is running and I added one more file to the directory, this files content wouldn’t come up in the search because the indexer has not run on it. This was a beautiful problem I tried to approach it from many different angles first I thought I would re-run the service every time I add a file but that’s not a graceful solution then I thought I would write a cron which will ping /index at regular interval and yet again that was a bad option, finally I thought if I could detect the change in file. This led me to explore gin, modd and fresh.

Gin was not very compatible with mux so didn’t use it, modd was very nice but I need to kill the server to restart it since two service cannot run on a single port and every time I kill that service I kill the modd daemon too so that possibility also got ruled out.

Finally the best solution was fresh although I had to write a custom config file to suite the requirement this still has issues with nested repository indexing which I am thinking how to figure out.

What’s Next?

This project is yet to be containerised and there are missing test cases so I would be working on them as and when I get time.

I have learnt a lot of new things about filesystem and how it works because of this project, this helped me appreciate a lot of golang concepts and made me realise the power of static typing.

If you are interested you are welcome to contribute to file-indexer. Feel free to ping me.

Till then, Happy Hacking!

Couple of weeks ago, Jen pointed me to vcrpy. This is a Python implementation of Ruby’s library with same name.

What is vcrpy?

It is a Python module which helps to write faster and simple tests involving HTTP requests. It records all the HTTP interactions in plain text files (by default in a YAML file). This helps to write deterministic tests, and also to run them in offline.

It works well with the following Python modules.

• requests
• aiohttp
• urllib3
• tornado
• urllib2
• boto3

Usage example

Let us take a very simple test case.

import unittest
import requests

class TestExample(unittest.TestCase):

    def test_httpget(self):
        r = requests.get("https://httpbin.org/get?name=vcrpy&lang=Python")
        self.assertEqual(r.status_code, 200)
        data = r.json()
        self.assertEqual(data["args"]["name"], "vcrpy")
        self.assertEqual(data["args"]["lang"], "Python")


if __name__ == "__main__":
    unittest.main()

In the above code, we are making a HTTP GET request to the https://httpbin.org site and examining the returned JSON data. Running the test takes around 1.75 seconds in my computer.

$ python test_all.py
.
------------------------------------------------------------------
Ran 1 test in 1.752s

OK

Now, we can add vcrpy to this project.

import unittest
import vcr
import requests

class TestExample(unittest.TestCase):

    @vcr.use_cassette("test-httpget.yml")
    def test_httpget(self):
        r = requests.get("https://httpbin.org/get?name=vcrpy&lang=Python")
        self.assertEqual(r.status_code, 200)
        data = r.json()
        self.assertEqual(data["args"]["name"], "vcrpy")
        self.assertEqual(data["args"]["lang"], "Python")

if __name__ == "__main__":
    unittest.main()

We imported vcr module, and added a decorator vcr.use_cassette to our test function. Now, when we will execute the test again, vcrpy will record the HTTP call details in the mentioned YAML file, and use the same for the future test runs.

$ python test_all.py
.
------------------------------------------------------------------
Ran 1 test in 0.016s

OK

You all can also notice the time taken to run the test, around 0.2 second.

Read the project documentation for all the available options.

I have been trying to have regular monthly Emacs meetups online, starting from 2018.

The following are the meeting minutes and notes from the Jitsi meetings held online in the months of February and March 2018.

February 2018

The February 2018 meetup was primarily focussed on using Emacs for publishing.

Using Emacs with Hakyll to build websites and resumes were discussed. It is also possible to output in muliple formats (PDF, YAML, text) from the same source.

I shared my shakthimaan-blog sources that uses Hakyll to generate the web site. We also discussed the advantanges of using static site generators, especially when you have large user traffic to your web site.

I had created the xetex-book-template, for creating multilingual book PDFs. The required supported features in the same, and its usage were discussed in the meeting.

Kushal Das asked about keyboards use in Emacs, and in particular for Control and Alt, as he was using the Kinesis. The best Emacs keyboard options available at http://ergoemacs.org/emacs/emacs_best_keyboard.html was shared. The advantage of using thumb keys for Control and Alt was obvious with the bowl shaped keyboard layout in Kinesis.

We also talked about the Emacs Web Browser (eww), and suggested the use of mu4e for checking e-mails with Emacs.

March 2018

At the beginning of the meetup, the participants asked if there was a live stream available, but, we are not doing so at this point in time with Jitsi.

For drawing inside Emacs, I had suggested ASCII art using Artist Mode.

Emacs has support for rendering PDFs inside, as the following old blog post shows http://www.idryman.org/blog/2013/05/20/emacs-and-pdf/. nnnick then had a question on “Why Emacs?”:

nnnick 19:23:00
Can you tell me briefly why emacs is preferred over other text editors

The discussion then moved to the customization features and extensibility of Emacs that makes it well suited for your needs.

For people who want to start with a basic configuration for Emacs, the following repository was suggested https://github.com/technomancy/emacs-starter-kit.

I had also shared links on using Org mode and scrum-mode for project management:

• https://github.com/ianxm/emacs-scrum

• https://gitlab.com/shakthimaan/operation-blue-moon

I shared my Cask setup link https://gitlab.com/shakthimaan/cask-dot-emacs and mentioned that with a rolling distribution like Parabola GNU/Linux-libre, it was quite easy to re-run install.sh for newer Emacs versions, and get a consistent setup.

In order to SSH into local or remote systems (VMs), Tramp mode was suggested.

I also shared my presentation on “Literate DevOps” inspired by Howardism https://gitlab.com/shakthimaan/literate-devops-using-gnu-emacs/blob/master/literate-devops.org.

Org entries can also be used to keep track of personal journal entries. The Date Trees are helpful in this context as shown in the following web page http://members.optusnet.com.au/~charles57/GTD/datetree.html.

Tejas asked about using Org files for executing code in different programming languages. This can be done using Org Babel, and the same was discussed.

Tejas 19:38:23
can org mode files be used to keep executable code in other languages apart from elisp?

mbuf 19:38:42
Yes

mbuf 19:39:15
https://orgmode.org/worg/org-contrib/babel/languages.html

The other useful tools that were discussed for productivity are given below:

• Magit for VCS: https://github.com/magit/magit.

• Workgroups for workspace management: https://github.com/pashinin/workgroups2.

• Winner mode to switch between sessions in windows: https://www.emacswiki.org/emacs/WinnerMode.

• Emacs Code Browser is like a full flown IDE-like environment: http://ecb.sourceforge.net/.

Tejas said that he uses perspective-el, but it does not have the save option - just separate workspaces to switch between them - for different projects basically.

A screenshot of the session in progress is shown below:

Arun also suggested using Try for trying out Emacs packages without installation, and cycle-resize package for managing windows.

Tejas and Arun then shared their Emacs configuration files.

Arun 19:51:37
https://github.com/aruntakkar/emacs.d

Tejas 19:51:56
https://github.com/tejasbubane/dotemacs

We closed the session with few references on learning Emacs Lisp:

Tejas 20:02:42
before closing off, can you guys quickly point me to some resources for learning elisp?

mbuf 20:03:59
Writing GNU Emacs Extensions.

mbuf 20:04:10
Tejas: Emacs Lisp manual

Tejas 20:04:35
Thanks 

This is a continuation of the design pattern series.

I had blogged about Singleton once, when I was using it very frequently. This blog post is about the use of the Template Design Pattern. So let’s discuss the pattern and then we can dive into the code and its implementation and see a couple of use cases.

The Template Method Design Pattern is a actually a pattern to follow when there are a series of steps, which need to be followed in a particular order. Well, the next question that arises is, “Isn’t every program a series of steps that has to be followed in a particular order?”

The answer is Yes!

This pattern diverges when it becomes a series of functions that has to be executed in the given order. As the name suggests it is a Template Method Design pattern, with stress on the word method, because that is what makes it a different ball game all together.

Let’s understand this with an example of Eating in a Buffet. Most of us have follow a set of similar specific steps, when eating at a Buffet. We all go for the starters first, followed by main course and then finally, dessert. (Unless it is Barbeque Nation then it’s starters, starters and starters :))

So this is kind of a template for everyone Starters --> Main course --> Desserts.

Keep in mind that content in each category can be different depending on the person but the order doesn’t change which gives a way to have a template in the code. The primary use of any design pattern is to reduce duplicate code or solve a specific problem. Here this concept solves the problem of code duplication.

The concept of Template Method Design Pattern depends on, or rather  is very tightly coupled with Abstract Classes. Abstract Classes themselves are a template for derived classes to follow but Template Design Pattern takes it one notch higher, where you have a template in a template. Here’s an example of a BuffetHogger class.

from abc import ABC, abstractmethod

class BuffetHogger(ABC):

    @abstractmethod
    def starter_hogging(self):
        pass

    @abstractmethod
    def main_course_hogging(self):
        pass

    @abstractmethod
    def dessert_hogging(self):
        pass

    def template_hogging(self):
        self.starter_hogging()
        self.main_course_hogging()
        self.dessert_hogging()

So if you see here the starter_hogging, main_course_hogging and dessert_hogging are abstract class that means base class has to implement it while template_hogging uses these methods and will be same for all base class.

Let’s have a Farhaan class who is a BuffetHogger and see how it goes.

class Farhaan(BuffetHogger):
    def starter_hogging(self):
        print("Eat Chicken Tikka")
        print("Eat Kalmi Kebab")

    def __call__(self):
        self.template_hogging()

    def main_course_hogging(self):
        print("Eat Biryani")

    def dessert_hogging(self):
        print("Eat Phirni")

I had organized a hands-on scripting workshop using the Elixir programming language for the Computer Science and Engineering department, MVJ College of Engineering, Whitefield, Bengaluru on May 5, 2018.

The department were interested in organizing a scripting workshop, and I felt using a new programming language like Elixir with the power of the Erlang Virtual Machine (VM) will be a good choice. The syntax and semantics of the Elixir language were discussed along with the following topics:

• Basic types
• Basic operators
• Pattern matching
• case, cond and if
• Binaries, strings and char lists
• Keywords and maps
• Modules and functions

Students had setup Erlang and Elixir on their laptops, and tried the code snippets in the Elixir interpreter. The complete set of examples are available in the following repo:

https://gitlab.com/shakthimaan/elixir-scripting-workshop

A group photo was taken at the end of the workshop.

I would like to thank Prof. Karthik Myilvahanan J for working with me in organizing this workshop.

People that are browsing the web with ad-blockers often find themselves on websites that gray out, block scrolling and show a modal dialog that kindly suggests to switch off the ad-blocker or to whitelist that particular page.

Here's a little work-around how you can continue browsing most of those sites without whitelisting the page or turning off the ad-blocker, by live-editing the HTML.

So whenever you see a banner like this - I've come across this a dozen times now, this is an example from a German online news magazine - open the Web Inspector. There are multiple ways to do this; in Firefox you can open Tools -> Web Developer -> Inspector, or using Chromium it would be Menu -> More tools -> Developer tools, or just hit F12.

I like right-clicking the object I want to inspect and select Inspect element.

So when I inspect the modal dialog and follow the DOM a bit upwards, I find the corresponding <div> tag that describes the dialog. Note the style="display: block;" css rule.

Since we don't want to see this dialog at all, right-click that html element and simply delete the whole node.

Schlurps and the dialog is gone. However, we still have that gray veil. In this example, the responsible <div> tag is the one just right above the previous modal dialog tag. Again we find the style="display: block;" rule and again we simply delete that node.

Finally the website looks almost normal. But shoot! scrolling is deactivated. If you happen to use Vim keybindings for navigating in your browser, you might not even notice. However, to be able to scroll with your mouse or arrow keys, find the <body> tag way up the DOM.

You may have guessed it: Right we're not going to delete the <body> tag. Notice the css lines "overflow-y: hidden; height: 911px;". This hides the scroll bar and sets a fixed height to what seems to be my browser window height. You can either delete that css, or - if you want to - modify to something like "overflow-y: auto; height: 100%;" and you should be browsing that site without ads and annoying modals.