TL;DR

PyDelhi conf 2017 was a two-day conference which featured workshops, dev sprints, both full-length and lightning talks. There were workshop sessions without any extra charges. Delhiites should not miss the chance to attend this conference in future. I conducted a workshop titled “Tango with Django” helping beginners to understand the Django web framework.

Detailed Review

About the PyDelhi community

PyDelhi conf 2017 volunteers

The PyDelhi community was known as NCR Python Users Group before few years. This community is performing a role of an umbrella organization for other FLOSS communities across New Delhi, India. They are actively arranging monthly meetups on interesting topics. Last PyCon India which is a national level conference of Python programming language was impressively organized by this community. This year too they took the responsibility of managing it. I am very thankful to this community for their immense contribution to this society. If you are around New Delhi, India then you should not miss the chance to attend their meetups. This community has great people who are always happy to mentor.

PyDelhi conf 2017

Conference T-shirt

PyDelhi conf is a regional level conference of Python programming language organized by PyDelhi community. It is their second year organizing this conference. Last year it was located at JNU University. This year it happened at IIM, Lucknow campus based in Noida, New Delhi, India. I enjoyed various talks which I will mention later here, a workshops section because I was conducting one and some panel discussions because people involved were having a good level of experience. 80% of the time slot was divided equally between 30 minutes talk and 2-hour workshop section. 10% were given to panel discussions and 10% was reserved for lightning talks. The dev sprints were happening in parallel with the conference. The early slot was given to workshops for both the days. One large conference hall was located on a 2nd floor of the building and two halls at the ground floor. Food and beverages were served on the base floor.

Panel Discussion

Registration desk

Tea break

Keynote speakers

• Mr. Ricardo Rocha: Mr. Rocha is a software engineer at CERN. I got some time to talk with him post-conference. We discussed his responsibilities at CERN. I was impressed when he explained how he is managing infrastructure with his team. On inquiring opportunities available at CERN he mentioned that the organization is always looking for the talented developers. New grads can keep an eye on various Summer Internship Programs which are very similar to Google Summer of Code program.

• Mr. Chris Stucchio: Mr. Stucchio is director of Data Science at Wingify/ VWO. I found him physically fit compared to other software developers (mostly of India). I didn’t get much time to have a word with him.

Interesting Talks

Because I took the wrong metro train, I was late for the inaugural ceremony. I also missed a keynote given by Mr. Rocha. Below talks were impressively presented at the conference.

• Let’s talk about GIL by Mr. Amit Kumar: Mr. Kumar discussed various ways to trace threads first and then moved the track towards Global Interpreter Lock. He described why the GIL is important in CPython.

• Concurrency in Python 3.0 world - Oh my! by Mr. Anand Pillai: Mr. Pillai is well experienced in programming using Python language. I like getting his advices on various programming topics. He explained how async IO can be leveraged to boost your programs. I got few correct references on understanding latest API of the async library.

• Property based testing 101 by Aniket Maithani: I always enjoy chit chatting with Mr. Maithani during conferences. He has jolly nature and always prepared with one liner. He discussed various strategies for generating demo data for test cases. I was amazed by his references, tips and tricks for generating test data.

I love discussing with people rather than sit in on sessions. With that ace-reason, I always lose some important talks presented at the conference. I do not forget to watch them once they are publicly available. This year I missed following talks.

• Optimizing Django for building high-performance systems by Mr. Sanyam Khurana

• Mocking in Python by Mr. Saurabh Kumar

Volunteer Party

I got a warm invitation by the organizers to join the volunteer party, but I was little tensed about my session happening on the next day. So, I decided to go home and improve the slides. I heard from friends that the party was awesome!

My workshop session

Me conducting workshop

I conducted a workshop on Django web framework. “Tango with Django” was chosen as a title with a thought of attracting beginners. I believe this title is already a name of famous book solving the same purpose.

• Slides

• Video Youtube

Dev sprints

Me hacking at dev sprints section

The dev sprints were happening parallel with the conference. Mr. Pillai was representing Junction. I decided to test few issues of CPython but didn’t do much. There were a bunch of people hacking but didn’t find anything interesting. The quality of chairs was so an impressive that I have decided to buy the same for my home office.

Why attend this conference?

• Free Workshops: The conference has great slot of talks and workshops. Workshops are being conducted by field experts without expecting any other fees. This can be one of the great advantages you leverage from this conference.

• Student discounts: If you are a student then you will receive a discount on the conference ticket.

• Beginner friendly platform: If you are novice speaker than you will get mentorship from this community. You can conduct a session for beginners.

• Networking: You will find senior employees of tech giants, owner of innovative start-ups and professors from well-known universities participating in this conference. It can be a good opportunity for you to network with them.

What was missing?

• Lecture hall arrangement: It was difficult to frequently travel to the second floor and come back to the ground floor. I found most people were spending their time on the ground floor rather than attending talks going on upstairs.

• No corporate stalls: Despite having corporate sponsors like Microsoft I didn’t find any stall of any company.

• The venue for dev sprints: The rooms were designed for teleconference containing circularly arranged wooden tables. This was not creating a collaborative environment. Involved projects were not frequently promoted during the conference.

Thank you PyDelhi community!

I would like to thank all the known, unknown volunteers who performed their best in arranging this conference. I am encouraging PyDelhi community for keep organizing such an affable conference.

• Conference Photos

• Talk videos

Proofreaders: Mr. Daniel Foerster, Mr. Dhavan Vaidya, Mr. Sayan Chowdhury, Mr. Trent Buck

This post is for my younger colleagues on the student planet.
Yes kids, this is a long post. Suck it up and read it over the weekend.
This is important stuff.
And please go, read this on the blog.

Introduction

In The Intelligent Investor, published in 1949, Benjamin Graham¹ wrote:

In the old legend the wise men finally boiled down the history of mortal affairs into the single phrase, “This too will pass.”

Confronted with a like challenge to distill the secret of sound investment into three words, we venture the motto;

MARGIN OF SAFETY.

… the function of the margin of safety is, in essence, that of rendering unnecessary an accurate estimate of the future.

(emphasis mine)

What does that mean to me?

That in order to do anything with my life, in any area, I should have enough to face the worst that I think, could happen to me.

I’ll simplify it further.

Margin is having enough room to breathe;
and knowing what “enough” means in every sphere of life.

It has implications throughout life and across various societal disciplines even.
Ben Graham’s quote above comes from the field of finance.
What about engineering?

Consider a highly-engineered jet engine part. If the part were to fail, the engine would also fail, perhaps at the worst possible moment—while in flight with passengers on board … We’ve designed the part for 10,000 hours of average flying time. That brings us to a central question: After how many hours of service do we replace this critical part? 9,999 hours? Why replace it any sooner than we have to? Why waste money?

The first problem is, we know nothing of the composition of the 10,000 hours any individual part has gone through. Were they 10,000 particularly tough hours, filled with turbulent skies? Was it all relatively smooth sailing?
And, how confident are we, that the part will really last the full 10,000 hours? What if it had a slight flaw during manufacturing? What if we made an assumption about its reliability that was not conservative enough? What if the material degraded in bad weather to a degree we didn’t foresee?

The challenge is clear, and the implication obvious: we do not wait until the part has been in service for 9,999 hours. Perhaps at 7,000 hours, we seriously consider replacing the part, and we put a hard stop at 7,500 hours.

The difference between waiting until the last minute and replacing it comfortably early gives us a margin of safety. We leave ourselves a cushion. (Ever notice how your gas tank indicator goes on long before you’re really on empty? It’s the same idea.)

Go read the whole post. It’s awesome.

So, Why Do We Need Margin?

To paraphrase Dr. Richard Swenson, who’s literally written the book on Margin (the book has a moral, Christian bent. Don’t let that bother you);

• Progress gives us more and more, which leads to
• stress, change, complexity, speed, intensity, and overload.
• which is bad, because all humans have physical, mental, emotional, and financial limits that are relatively fixed
• Thus progress is on a collision course with human limits and once we can’t take progress any more, we flame and burn out. On the unsaturated side of our limits, humans can be open and expansive. Cross them however, and the rules of life totally change.

What would you think if this page had no margins?
What would you think of me if I crammed the print top to bottom and side to side so that every blank space was filled up?
The result would be aesthetically displeasing and chaotic.

Like some of our lives!

Marginless, according to Swenson, is being thirty minutes late to the doctors office because you were twenty minutes late getting out of the bank because you were ten minutes late dropping the kids off at school because the car ran out of fuel two klicks from the petrol pump and you forgot your wallet.
Margin, on the other hand, is having breath left at the top of the staircase, money left at the end of the month, and sanity left at the end of adolescence.

No Margin, causes hyperstress and overload. (terms that to folks like us, need no definition)

Here’s Dr. Swenson again,

To be healthy, we require margin in at least four areas:
- emotional energy,
- physical energy,
- time,
- and finances.

Yet, right now for the most part we are emotional wrecks, stressed, alone, and so exhausted in spirit.
Physically, we are overfed, underactive, and sleep-deprived.
We have no time, at all.
And when it comes to finances, our bank balances look as barren as the surface of Mars.

What Can We Do about It?

Thank Pareto, grab the low hanging fruit and do the twenty percent of the work that’ll yield the most margin in your life.

Remember, we live lives, full of intention now :)

Let’s intentionally tackle each domain and see what we can do.

Emotional:

• Bond with your family
• Make time for friends
• Switch off the TV! … and your other screens
• Get a pet or go play with doggies in the park
• Pray, meditate.

Physical:

• Eat less, move more
• Walk, Run!
• Cut out sugar
• Drink water.
• Have the discipline (or build it up) to keep at it.
  (I didn’t have any and I’m paying the price!)
• Most importantly, get enough sleep.

Time:

• You’ll never “find time”. Be intentional. Make time!
• Learn to plan your day and your week.
• Plan your day with plenty of spaces in the middle
• Did you cut out FB? Twitter? TV? :P
• Learn to say No. No is a complete sentence.

Financial:

• Stop eating out! Learn to make do.
• The only debt you have should be your home. (and no more than 40% of income at that)
• Have 6 months living expenses in the bank.
• Spend way less than you make. (The bigger the buffer, the more chaos you can handle.)
• Be comfortable with charity and giving now, when it’s hard to do. Like Dave Ramsey says, “If you will live like no one else, later you can live and give like no one else.”

And finally here’s James Clear, echoing Taleb
Leave Room for the Unexpected!

If your life is designed only to handle the expected challenges, then it will fall apart as soon as something unexpected happens to you.
Always be stronger than you need to be.
Always leave room for the unexpected.

Thank you for reading. Here’s a puppy.

This is the bonus section

Want to hear from an actual practitioner?

So what is the lesson?
Save your current income, even as you build complementary skills like frugality, adaptability, and the ability to earn money in more than one way.
Then you’ll be ready to retire much earlier, because the sum of your safety margins will be enough to make you feel comfortable taking the leap.
And most importantly, stop worrying!

That’s Mr. Money Mustache (real name, Pete Adeney), software developer who retired at age 30! (watch the video)

Resources:
(in the grand tradition of @mbuf feeding us well with knowledge)

• The eponymous book on the topic, by Dr. Richard Swenson
• Shane Parrish - A Margin of Safety
• Shawn Blanc’s Margin Series
• Be Strong - James Clear
• MMM - Levels of Safety
• How to be Rich, Happy & Save the World!

P.S. This post is part of the Life Outside Code series.

My life can be divided into major two parts.
Life before knowing “Libre Office” and after.
No, I won’t go on about how good the software is (which it is; fairly good).

I was fascinated by the name, the meaning.
Libre, means “free, at liberty.”
The name suggested a philosophy, a vision, an idea to me.

Libre != Gratis

Though I was introduced to the Free Software World briefly a while back at FOSS.in, 2012, a conference that altered the way I look at my soul-mate forever.

But the concept of open source, free software, the industry did not quite settle in clearly, in my mind. The past leads us to the present. So I became more interested in its history. The history of the movement of liberating cyber space. I asked Kushal, who happens to be a free software advocate, and a devout follower to tell me the story, and guide me through the path.

We embarked on a wonderfully meandering journey where I was visiting older, simpler times and Kushal was revisiting the movement with me. The era of sharing, the beginning of the complexities and the fight against it.

Our primary source of knowledge was the book, Hackers: Heroes of the Computer Revolution, apart from several old articles, newsletters etc.

I was astonished to know that there exists software that respects freedom; the freedom of end users.
There exists software that believes in cooperation and collective building.
By strict definition, the first one is known as free software and the second one is open source software.
To my utmost surprise there are people sacrificing their material wants, over the peace of mind, and easy fame to help others. To preach freedom and openness.

How did it help me?

It made me understand the present state of the world, a lot better.
I now see the words community, & collaboration in a different light altogether.
I now understand why people are so passionate about words, phrases or things.
Why these terms make all the difference.

Most importantly it gave me a refuge. I may not be someone with a coding background, but I can still very much be a part of this community. Coding may be a big part of the technical community but it’s not the only part. It gave me the power to ignore the cold shoulder and the occasional jibe (and laugh at their incomprehension about history), whenever someone treated me like an outsider. It gave myself the justification, and confidence to take the decision of quitting a fairly established career as mortgage attorney and start afresh at 30.

Community, collaboration, and freedom are the some my most used words (apart from “No Py”!) in a day since then.

And now I know why.
And it’s time for me to spread the word.

Coming to the present

July 2017, a tweet and its reply made us relive that journey. It was the discussion of the recent travel ban. A tweet came from @gnome about this. Totally appropriate given the primary idea of gnome. But many could not figure out why such statements are coming from @gnome.
I was taken aback. We were sad, surprised and somewhat angry. People were enjoying the fruits of so many people’s hard work and effort and were yet ignorant of the history and the toil behind it all.
Among many other replies, there was a reply from Miguel de Icaza that seemed to echo our (Kushal’s and mine) thoughts.

We understand why he said that. We thought it’s time for us too, to do our bit, to spread the word.
And so, we decided to do two things -

Firstly, to tell newer, younger lot about it.
For which Kushal & I have taken classes on
- the history of hacker ethics - licensing and how the choices we make affect us, the society, and the world at large, at DGPLUG Summer training session.
A similar session will follow in PyLadies Pune August meetup.

Secondly, to write an article on the Hacker Ethic and the Free Software movement, together.
We have published this on 12th July, 2017. Here is the link
. Please read it (though it is a bit lengthy) and be a part of our journey.

Encrypting hard drives should be a common step in our regular computer usage. If nothing else, this will help you sleep well, in case you lose your computer (theft) or that small USB disk you were carrying in your pocket. In this guide, I’ll explain how to encrypt your USB disks so that you have peace of mind, in case you lose them.

But, before we dig into the technical details, always remember the following from XKCD.

What is LUKS?

LUKS or Linux Unified Key Setup is a disk encryption specification, first introduced in 2004 by Clemens Fruhwirth. Notice the word specification; instead of trying to implement something of its own, LUKS is a standard way of doing drive encryption across tools and distributions. You can even use drives from Windows using the LibreCrypt application.

For the following example, I am going to use a standard 16 GB USB stick as my external drive.

Formatting the drive

Note: check the drive name/path twice before you press enter for any of the commands below. A mistake, might destroy your primary drive, and there is no way to recover the data. So, execute with caution.

In my case, the drive is detected as /dev/sdb. It is always a good idea to format the drive before you start using it. You can use wipefs tool to clean any signature from the device,

$ sudo wipefs -a /dev/sdb1

Then you can use fdisk tool to delete the old partitions , and create a new primary partition.

Next step is to create the LUKS partition.

$ sudo cryptsetup luksFormat /dev/sdb1

WARNING!
========
This will overwrite data on /dev/sdb1 irrevocably.

Are you sure? (Type uppercase yes): YES
Enter passphrase: 
Verify passphrase:

Opening up the encrypted drive and creating a filesystem

Next, we will open up the drive using the passphrase we just gave, and create a filesystem on the device.

$ sudo cryptsetup luksOpen /dev/sdb1 reddrive
Enter passphrase for /dev/sdb1
$ ls -l /dev/mapper/reddrive
lrwxrwxrwx. 1 root root 7 Jul 17 10:18 /dev/mapper/reddrive -> ../dm-5

I am going to create an EXT4 filesystem on here.
Feel free to create which ever filesystem you want.

$ sudo mkfs.ext4 /dev/mapper/reddrive -L reddrive
mke2fs 1.43.4 (31-Jan-2017)
Creating filesystem with 3815424 4k blocks and 954720 inodes
Filesystem UUID: b00be39d-4656-4022-92ea-6a518b08f1e1
Superblock backups stored on blocks: 
        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208

Allocating group tables: done                            
Writing inode tables: done                            
Creating journal (16384 blocks): done
Writing superblocks and filesystem accounting information: done   

Mounting, using, and unmounting the drive

The device is now ready to use. You can manually mount it with the mount command. Any of the modern desktops will ask you to unlock using the passphrase if you connect the device (or try to double click on the file browser).

I will show the command line option. I will create a file hello.txt as an example.

$ sudo mount /dev/mapper/reddrive /mnt/red
$ su -c "echo hello > /mnt/red/hello.txt"
Password:
$ ls -l /mnt/red
total 20
-rw-rw-r--. 1 root root     6 Jul 17 10:26 hello.txt
drwx------. 2 root root 16384 Jul 17 10:21 lost+found
$ sudo umount /mnt/red
$ sudo cryptsetup luksClose reddrive

When I attach the drive to my system, the file browser asks me to unlock it using the following dialog. Remember to choose forget immediately so that the file browser forgets the password.

On passphrases

The FAQ entry on cryptsetup page, give us hints and suggestions about passphrase creation.

If paranoid, add at least 20 bit. That is roughly four additional characters for random passphrases and roughly 32 characters for a random English sentence.

Key slots aka different passphrases

In LUKS, we get 8 different key slots (for passphrases) for each device(partition). You can see them using luksDump sub-command.

$ sudo cryptsetup luksDump /dev/sdb1 | grep Slot
Key Slot 0: ENABLED
Key Slot 1: DISABLED
Key Slot 2: DISABLED
Key Slot 3: DISABLED
Key Slot 4: DISABLED
Key Slot 5: DISABLED
Key Slot 6: DISABLED
Key Slot 7: DISABLED

Adding a new key

The following command adds a new key to the drive.

$ sudo cryptsetup luksAddKey /dev/sdb1 -S 5
Enter any existing passphrase: 
Enter new passphrase for key slot: 
Verify passphrase:

You will have to use any of the existing passphrases to add a new key.

$  sudo cryptsetup luksDump /dev/sdb1 | grep Slot
Key Slot 0: ENABLED
Key Slot 1: DISABLED
Key Slot 2: DISABLED
Key Slot 3: DISABLED
Key Slot 4: DISABLED
Key Slot 5: ENABLED
Key Slot 6: DISABLED
Key Slot 7: DISABLED

Removing a passphrase

Remember that removing a passphrase is based on the passphrase itself, not by the key slot number.

$ sudo cryptsetup luksRemoveKey /dev/sdb1
Enter passphrase to be deleted: 
$ sudo cryptsetup luksDump /dev/sdb1 | grep Slot
Key Slot 0: ENABLED
Key Slot 1: DISABLED
Key Slot 2: DISABLED
Key Slot 3: DISABLED
Key Slot 4: DISABLED
Key Slot 5: DISABLED
Key Slot 6: DISABLED
Key Slot 7: DISABLED

Now in case you don’t know the passphrase, then you can use luksKillSlot.

$ sudo cryptsetup luksKillSlot /dev/sdb1 3
Enter any remaining passphrase:

Overview of the disk layout

The disk layout looks like the following. The header or phdr contains various details like magic value, version, cipher name, following the 8 keyblocks (marked as kb1, kb2.. in the drawing), and then the encrypted bulk data block. We can see all of those details in the C structure.

struct luks_phdr {
        char            magic[LUKS_MAGIC_L];
        uint16_t        version;
        char            cipherName[LUKS_CIPHERNAME_L];
        char            cipherMode[LUKS_CIPHERMODE_L];
        char            hashSpec[LUKS_HASHSPEC_L];
        uint32_t        payloadOffset;
        uint32_t        keyBytes;
        char            mkDigest[LUKS_DIGESTSIZE];
        char            mkDigestSalt[LUKS_SALTSIZE];
        uint32_t        mkDigestIterations;
        char            uuid[UUID_STRING_L];

        struct {
                uint32_t active;

                /* parameters used for password processing */
                uint32_t passwordIterations;
                char     passwordSalt[LUKS_SALTSIZE];

                /* parameters used for AF store/load */
                uint32_t keyMaterialOffset;
                uint32_t stripes;
        } keyblock[LUKS_NUMKEYS];

        /* Align it to 512 sector size */
        char                _padding[432];
};

Each (active) keyblock contains an encrypted copy of the master key. When we enter the passphrase, it unlocks the master key, that in turn unlocks the encrypted data.

But, remember, all of this is of no use if you have a very simple passphrase. We have another XKCD to explain this.

I hope this post encourages you to use encrypted drives more. All of my computers have their drives encrypted; (I do that while installing the Operating System.) This means, without decrypting the drive you can not boot the system properly. On a related note, remember to turn off your computer completely, (not hibernation or suspend mode) when you’re traveling.

As I have mentioned in the dgplug summer training page, focusing on the Free Software movement is a big part of this year’s training program. A few weeks back there was a tweet from @gnome about travel ban, and many could not figure out why Gnome was writing about this topic. Amongst the many proper replies, Miguel de Icaza’s reply was to the point. This incident made Anwesha and me stop and think; and then made us rethink, about how we wanted to conduct the sessions on the Free Software movement and Software Licensing.

I was born in the beginning of the 80s and Anwesha even later. Our introduction to the movement was from the stories we heard (from many people); from Levy’s famous book, Hackers: Heroes of the Computer Revolution and the seminal Free as in Freedom.

My introduction to the FSF came through ilug-calcutta, and from Sayamindu. Later, at foss.in 2005, I made another friend (for life), Praveen A (he is from the same batch). And even later through out various conferences, I was introduced to other members of FSF India. In 2007, I was part of the 4th GPLv3 meet organizing team in Bangalore. That was my introduction to RMS, and his personality (I will write a blog post later about various incidents from that conference). That had a big impact on me.

Coming back to the story of the tweet, we also saw similar ignorance from newcomers, as they never got a chance to learn about the past, nor did get to meet the various people involved (distance and time). So, Anwesha and I, tried to write a brief history, including the hacker ethic, and beginning of the Free Software movement. A lot of stories mentioned in the article are from the books mentioned above. At the very end, I have written about how the different software we use everyday came about initially. I took the help of various FSF bulletins for the same.

This Monday I took a session on the same topic in the #dgplug IRC channel. When I made mention of the GNU C Library and the time Ronald McGrath started it, Siddhesh called attention to an announcement he (Ronald) made a few days ago (about stepping down from maintainership of that same GNU Library). I also pointed that Siddhesh is now one of the maintainer of Glibc. That gave the students a sense of impact and immediacy; a feeling of involvement and ownership.

Today evening from 13:30 UTC, Anwesha took a session on Software Licenses 101 in the #dgplug channel on Freenode. There will be more follow up sessions in the coming days.

Link to the article once again

While I was working on my project on software licensing, I noticed that there are several packages in PyPI which do not have a requirements.txt file (the file mentioning the dependencies). License files, though not a rare species but surely an endangered one in the Python World. These made my life so difficult you lazy programmers, grrr).

“One has to solve her problem.” So whenever I used to meet any developer I used to tell them the best practices for developers. Some of them used to give me not always a good vibe. (oh you irritating lady :(). Most of them were saddened by the long list of best practices. (me (grrr, grrr, grr))

I wanted to upload my new project, gitcen in PyPI. I shared my this thought with my eternal rival. He gave me quite smile.
He thinking : "Lady now you will understand". I smiled back.
Me thinking : "ohh man I hate you".

The code was all set and working

I used both PyCharm and jupyter notebook to write code.

The main job is done, now I will only have to upload my project at PyPI, the Pyhton Package Index.

requirements.txt

I created a requirements.txt file. One has to add the external dependencies. It helps to create the the similar development environment. Instead of manually typing the dependencies one can use pip freeze which will show the all modules installed in that particular virtual environment.

$ pip freeze
$ pip freeze > requirements.txt

This is easy, (Lazy coders).

setup.py

Now is the time for setup.py. It is a Python file which helps us to get the package. In the file, we call the setup function. The parameter inside the function are the different metadata of the project. The setup.py file of gitcen looks like this:

from setuptools import setup

setup(  
    name="gitcen",
    version='0.1.0',
    description="A project to find git information.",
    long_description="A project to find git information about authors' commits,most active days and time.",
    author="Anwesha Das.",
    author_email="anwesha@das.community",
    url="https://github.com/anweshadas/gitcen",
    license="GPLv3+",
    py_modules=['gitcen'],
    install_requires=[
        'Click',
        'pygit2==0.24'
    ],
    entry_points='''
        [console_scripts]
        gitcen=gitcen:main
    ''',
    classifiers=(
        'Development Status :: 3 - Alpha',
        'Intended Audience :: Developers',
        'License :: OSI Approved :: GNU General Public License v3 or later (GPLv3+)',
        'Programming Language :: Python :: 3.6'
    )
)

A little bite of history, previously there was package a named distutils which provided the setup function. But now we use the package setuptools.

It includes different information such as name, license, entry point etc. There is a parameter classifiers . Read more about it here.

This was lenghty but ok. May these coders are not that lazy (while coding, otherwise is. As per my experience with my man goes.)

Creating Source Distribution

The next was to create a compressed tarball for source distribution.

$python3 setup.py sdist

Output of the command is a tar.gz file.

Creating PyPI account

PyPI, the Python Package Index, the third party official software repository for Python ecosystem. One has to create an account in it.

test.pypi.org

If one wants to test the upload, test.pypi.org is the address to go to. Know more about the steps here.

When will this thing end?

Use twine

twine helps to upload packages in PyPI. It assembles the utilities to interact with PyPI. The first step was to install twine. I installed it from dnf package manager.

$ sudo dnf install python3-twine

I had to register my project. Using the following command

$ twine register dist/gitcen-0.1.0.tar.gz

Finally the moment had come! I uploaded gitcen in PyPI with the following command

 $ twine upload dist/gitcen-0.1.0.tar.gz

I had already put my authentication information in the ~/.pypirc

Now anyone can install gitcen from PyPI

$ pip install gitcen

Therefore going through this now I understand that it is very frustrating. You have a working code and you are not been able to share this with the community (easily, fast). If we give it a thought then we can realize that these are steps one has to memorize. As I was a first timer there were many new things to which I came across, so it made my life difficult. And yes,

Sorry coders, for not understanding your pain!

Note: This is for the Student Planet.
Please read this on the blog.

At the dawn of computing …

Last night, over at DGPLUG, Kushal gave quite a heart rousing talk on the history of Free Software, covering quite a bit about Richard Stallman and the events leading up to him (RMS, not Kushal) launching GNU & the FSF.

If you’re interested in reading more about that sort of thing, here’s a few more books & resources.

Free as in Freedom

The seminal book is of course, Free as in Freedom, on RMS’ life and the massive base he built, upon which we all stand today.
Reading this made me realise what a debt we owe to him.
So the next time we hear about how old & weird Stallman is, maybe we could cut him some slack.

The Groklaw Archives

Did you know, that the SCO Group once brought a case against IBM, suing them for using Linux?
It was a large, long drawn out affair, ¹ drawing old heavyweights such as Novell and new upcoming ones like Red Hat, into the fray.
If SCO had won, it would have been the end, of the just barely decade old Linux and our landscape would not have been as rich as it is today.

We know all this, because of the daring and intrepid, Pamela (PJ) Jones, who started up one of the earliest blogs on the internet.
It was called Groklaw, and it was …

a place where lawyers and geeks could explain things to each other and work together, so they'd understand each other's work better²

It brought awareness of the case to a wide swathe of people and Linux into the mainstream.
The bar on the left of the site, gives you access to a whole lot of cases threatening Linux, and the news and views of the people in the know

A Quarter Century of Unix

A short, really influential book on Unix History by Peter Salus.
I don’t quite know how you can lay hands on a copy, but if you do, it’s fascinating.
Nearly every article, every blog post, every book that needs something on the history of Unix, pulls a quote from this one.

The Daemon, the GNU & the Penguin

Salus’ follow up book, following Unix History on to Linux, is available to read on the Groklaw website.
You can think of it as an expanded version of Kushal’s talk last night.

Open Sources

Want to hear from the people involved in the free software movement?
From the horse’s mouths themselves?
Open Sources is a collection of essays from the folks who were there.
Marshall McKusick, author of the BSD filesystem narrates how BSD went on to be free from AT&T ownership and Free as in Freedom.
RMS tells us about GNU himself.
Bob Young, founder of Red Hat, expounds on how the company set itself on the path to becoming a business on the back of free software.
Linus, tells us of the edge, Linux had, to become successful.

Producing OSS

Karl Fogel’s book, on why we write free / open source software and how to pitch it in your organisation

This is all that comes to mind, right now.
If you know more, let me know or write about and I’ll update the page or link to yours.

Updates:

In the Beginning was the Command Line

Neal Stephenson’s essay on why Free Software would eventually win. (chock-full of history and analogy)
It’s dated and hasn’t quite panned out as he wished, but is still a fun read!

I had organized a one-day workshop based on my book i want 2 do project. tell me wat 2 do at reserved-bit, Pune on Saturday, June 3, 2017. Thanks to Nisha Poyarekar and Siddhesh Poyarekar for providing their makerspace as venue for the workshop.

The objective of the workshop is to share the methodology and best practices on working with Free and Open Source Software projects, and also to guide the participants on career options. Although there is plenty of free documentation (including my own) available on the subject, some people prefer a formal, systematic approach to learning in a private coaching environment. Hence, this workshop is tailored for the same and is made a paid workshop, similar to personal tutoring.

The book has been in circulation for many years. So, I had to give two options - pay only for the workshop, or pay for both the workshop and the book (if they have not already bought the book). I have also kept additional copies of my book at the reserved-bit makerspace if people are interested. I had covered the following topics:

1. Careers with F/OSS
2. “i want 2 do project. tell me wat 2 do” best practices
3. Introduction to Git
4. Hands-on problem solving exercises
5. Real-world project example
6. Goal-driven development

The feedback has been positive and I am sharing the same below:

If you are interested in attending such a workshop, do write to me: author at shakthimaan dot com.

My first step with FOSS was during the summer of 2014, where I got introduced to DGPLUG (Durgapur Linux Users Group). Every year, they conduct a training over IRC covering a wide range of topics. Since then, I've been contributing to different projects, primarily to Mozilla's Gecko Engine. Recently I also started contributing to CPython.

Over this journey as CuriousLearner over the past few years, I've been fortunate enough to collaborate with some of the best engineers, learnt a lot of skills & gained some nice perspective on different technologies by just contributing to Open Source. I owe my learning to them.

I've been always asked this question by my friends, colleagues, and strangers whom I often run into in local meet-ups and conferences.

• Why should I contribute to Open Source
• What's in it for me

So, I thought I would cover this topic from the perspective of a student as well as the perspective of a working professional.

I started contributing to CPython this year from PyCon Pune sprints. That was one of the best PyCon I've ever attended. Since then, I've spent some time in going through documentation, core dev-guide and now I'm still reading about internals of the Python programming language.

I recently contributed an enhancement patch to Python 3.7 which might be back-ported to Python 3.6. Let's understand the scenario and the enhancement:

New contributor to Python 3 might use old Python 2 syntax treating print as a statement rather than a function and then see an error like:

>>> print "hello"
  File "<stdin>", line 1
    print "hello"
                ^
SyntaxError: Missing parentheses in call to 'print'

The task was to improve the error message based on different kinds of user input. This shall ensure that we don't only show error to the user but also suggest the correct syntax to be used as a hint. So after the patch being applied, the error message shall say this:

>>> print "hello"
  File "<stdin>", line 1
    print "hello"
                ^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print("hello")?

Sounds easy? Right? No, we aren't done yet. This was a simple print statement dealing with just a string. Potentially someone can try to print anything. What if someone wants to print a tuple?

>>> print 4,
  File "<stdin>", line 1
    print 4,
          ^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print(4, end=" ")?

So, the correct syntax is suggested to the user according to Python3.

Along with this, there are cases when someone adds soft-spaces, or just use excessive whitespace like:

>>> print    "hello"
  File "<stdin>", line 1
    print    "hello"
                   ^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print("hello")?

This was the first patch where I had to deal with C code. It was more of reading and understanding what the current code does. I had to do a complete re-write of this patch two times in a span of just 1.5 days (Thanks to Nick and Serhiy for the quick reviews).

Here's why:

The first time I wrote the patch, I used raw C APIs, which means I used malloc and free for memory allocation and deallocation for manipulating the string input that was received from the user.

While this might work, but there is already a lot of work done by Python core-devs & they've created an entire C API layer. So, in the first review, I was pointed to use this API layer, so I had to re-write the patch. One of the learning I had is that virtually any API can fail. So, it is necessary to always check the results before using them.

Most of the methods from the C-API layer that were used in this patch were dealing with Unicode objects.

Another important learning was to deallocate the previous memory at each check point (if some of the API call fails and there is no memory allocated on heap then deallocate all the previous allocated memory).

Apart from handling the striping of characters that were input on the shell and detecting the tuple syntax, I also wrote test cases for the new feature. In CPython, most of the test cases are written in Python (except a few). Typically in this use case, we only need to check the functionality of the Python layer, so test cases were written in Python.

Let's see how the added feature does what it is intended to do:

// Static helper for setting legacy print error message
static int
_set_legacy_print_statement_msg(PySyntaxErrorObject *self, Py_ssize_t start)
{
    PyObject *strip_sep_obj = PyUnicode_FromString(" \t\r\n");
    if (strip_sep_obj == NULL)
        return -1;

    // PRINT_OFFSET is to remove `print ` word from the data.
    const int PRINT_OFFSET = 6;
    Py_ssize_t text_len = PyUnicode_GET_LENGTH(self->text);
    PyObject *data = PyUnicode_Substring(self->text, PRINT_OFFSET, text_len);

    if (data == NULL) {
        Py_DECREF(strip_sep_obj);
        return -1;
    }
    PyObject *new_data = _PyUnicode_XStrip(data, 2, strip_sep_obj);
    Py_DECREF(data);
    Py_DECREF(strip_sep_obj);

    if (new_data == NULL) {
        return -1;
    }
    // gets the modified text_len after stripping `print `
    text_len = PyUnicode_GET_LENGTH(new_data);
    const char *maybe_end_arg = "";
    if (text_len > 0 && PyUnicode_READ_CHAR(new_data, text_len-1) == ',') {
        maybe_end_arg = " end=\" \"";
    }
    PyObject *error_msg = PyUnicode_FromFormat(
        "Missing parentheses in call to 'print'. Did you mean print(%U%s)?",
        new_data, maybe_end_arg
    );
    Py_DECREF(new_data);
    if (error_msg == NULL)
        return -1;

    Py_XSETREF(self->msg, error_msg);
    return 1;
}

This function returns the following:

• 1 if the error message is successfully set.
• -1 in case any of the C-API fails.

So, first we use strip_sep_obj to define all the symbols that we need to strip from the input received from the user. Next, when we get the string, it would be of the form print "Hello".

PyObject *strip_sep_obj = PyUnicode_FromString(" \t\r\n");
if (strip_sep_obj == NULL)
    return -1;

// PRINT_OFFSET is to remove `print ` word from the data.
const int PRINT_OFFSET = 6;

What we're interested in is about the text after the print and space. That accounts for the PRINT_OFFSET that is set to constant 6.

Next, _PyUnicode_XStrip uses to strip strip_sep_obj from the data we assigned earlier. The 2 in this function is a constant value that implies stripping the input from both sides of the data.

PyObject *new_data = _PyUnicode_XStrip(data, 2, strip_sep_obj);

Next, we get the length of the text and check if the input meant to print a tuple by checking the last character from the Unicode buffer to be equal to string literal ,.

const char *maybe_end_arg = "";
if (text_len > 0 && PyUnicode_READ_CHAR(new_data, text_len-1) == ',') {
    maybe_end_arg = " end=\" \"";
}

After setting up the maybe_end_arg we format the new message and use Py_XSETREF to set the new error_msg to the PySyntaxErrorObject instance.

Py_XSETREF(self->msg, error_msg);

Here is the full patch if you want to see. The meaning of each of the methods used can be easily found from the official documentation.

I hope to write about my learning in more detail and also explain about different parts of the programming language in a future post.

Introduction

I attended Rootconf 2017 on May 11-12, 2017 at MLR Convention Centre, JP Nagar, Bengaluru.

The event had two parallel tracks, the Rootconf track at the main conference hall, and the DevConf.in track sponsored by Red Hat. A number of infrastructure hosting companies had sponsored the event. The selected talks had good diversity with topics ranging from monitoring, databases, networking, security etc.

Day I

The day’s proceedings began with the talk on the “State of open source monitoring landscape” by Bernd Erk. He gave a good overview of the available monitoring tools. He is involved in the Icinga2 project, which is re-written in C++ for performance. It is an alternative for the Nagios tool.

Aditya Patawari then provided an introduction to Kubernetes with a demo, as Spencer Krum could not make it to his talk on the “Anatomy of an alert”. After a short break, I attended “A little bot for big cause” session by Pooja Shah. The talk basically demonstrated how to create a bot by integrating various APIs.

I attended Manan Bharara’s talk on “Monitoring performance critical applications” where he had explained how they had used Clojure for monitoring at Otto in Germany. The presentation was very good, and the Oscillator Clojure code is available. He had provided good illustrations and code snippets in his presentation.

A number of stalls were available in the hallway, and I spent considerable time talking with the sponsors. At the Red Hat stall, I learnt some of the features and use cases for the Foreman provisioning tool. I was pictured with the Foreman helmet:

After lunch, I attended “SELinux: A deep dive” by Rejy M Cyriac. He gave an excellent introduction to SELinux, explaining the underlying concepts and rationale for the same. We also did basic hands-on exercises, but, I could not attend the last section as I had to move to the main conference hall for the flash talks. I sang the Nursery Rhymes version for DevOps. The video is available.

We then headed to the Rootconf party sponsored by Go-Jek!

Day II

The second day started with the talk by Kunal Grover on “Failure resilient architecture with microservice dependencies”. He presented the ways by which disruptions are performed in cloud infrastructure and how recovery mechanisms are tested. Ramya A then gave an interesting talk on “Asynchronous integration tests for microservices”. She had explained the pact.io family of frameworks to support Consumer Driven Contracts testing. Laxmi Nagarajan then gave a high level overview of “Capacity planning for AWS and configuring AWS autoscaling policies” sharing her Industry experience. After a short break, Colin Charles presented his story on “Capacity planning for your data stores” citing real world examples.

I then moved to the DevConf.in track to attend Ruchi Singh’s talk on “Migration of 300 microservices from AWS cloud to Snapdeal cloud”. The time was too short for such a talk, and it was an overview. I would have liked to see more details, given that they use Chef for provisioning and handle more than 16 TB of data as mentioned by Ruchi Singh.

After a quick lunch, I attended “Adventures in Postgres management” by Ramanan Balakrishnan. It was a good technical talk going over the failures and experiences learnt. After the talk I went to the hallway to listen to the Off-The-Record (OTR) session on mistakes to avoid when planning your infrastructure.

I returned to the main conference hall for doing some stretching exercises. In my opinion, all conferences should make this session mandatory, especially after you have been sitting for a long period of time. This session was then followed by the “Working with Secrets” talk by Shrey Agarwal, who gave an overview of using Hashicorp’s Vault for managing passwords. It was a very short introduction on the topic.

After a short beverage break, Toshaan Bharvani presented “Living with SELinux”, which was an excellent presentation on the subject. The initial slides had some material that Rejy M Cyriac had introduced the previous day, but, the content and presentation were good. With proper SELinux policies, he said that root in his production servers cannot do much. Rejy and Toshaan both asked people to use permissive mode instead of disabling SELinux altogether, so that you at least know what is being audited.

The last talk of the day was by Anurag Bhatia on “Understanding eyeball routing via RIPE Atlas”. He gave an overview of the RIPE Atlas project and how network metrics are measured and monitored wherever the RIPE device is installed. It is a TP-Link hardware whose firmware can be flashed using Libre software. Unfortunately, at this point the source of the firmware is not released as Free and Open Source Software. I was told that there is still an ongoing discussion on the same.

Conclusion

The talk by Manan Bharara and Rejy’s SELinux workshop were the highlights for day one for me. The content on the second day was much better and had greater depth in my opinion. Overall, it was a useful technical conference, and a good place to meet like-minded people.

I would like to thank Aerospike for sponsoring me to attend the conference.

The June Golang Bangalore Meetup was conducted on Saturday, June 17th, 2017. There were around 35-40 people who attended the meetup.

The meetup started at 10:15 with the first talk by Nurali Virani who works at SAP Labs, Bengaluru. He talked about “Understanding Type System In Go”.

Saifi gave an awesome talk about “Working with C code and plugins in Go”. The slides related to his talk can be found here

Umasankar Mukkara is the Co-Founder and CEO at CloudByte Inc. He talked about OpenEBS and their experience with Golang. The slides of his talk could be found here. OpenEBS is an open source project written in Golang. You can find the source code of OpenEBS here.

Satyam Zode who is also a fellow Golang programmer at OpenEBS presented a talk about “Package oriented design in Go”. The slides related to his talk could be found here

I thank Uma, OpenEBS and Nexus Ventures Partners for helping us to organize the June Golang Bangalore Meetup by providing venue and food. There were also a few goodies for the participants by OpenEBS. Fortuanately, this event was recorded and we will be sharing the recorded videos soon when they will be ready.

Back after a long time. It’s been a long time since I blogged. This was mostly because I was busy with different conferences. Yes,I attended three conferences PyCon Pune, FOSSASIA, and PyCon US since the last time I blogged. I would be back writing reports on all the three events and I have a pending third and the final blog post to be done on fedimg.

Now, let me rollback to the topic for which this blog post is meant to be.

11th June 2017, PyLadies Pune organized the meetup for the month of June. The meetup started a bit late with Shilpee Chamoli taking a 101 class on Data Science. The workshop focused on installing dependencies (Jupyter, Pandas, etc), and working on a dataset on ‘survival on the Titanic’. Since few people were having trouble with installing the dependencies, Kushal and a few other people used Azure Notebooks.

Anwesha and I had recorded videos during PyCon US which had messages from Lynn Root, Ewa Jodlowska, Carol Willing, and Jackie Kazil for the PyLadies Pune group members encouraging them on progressively working ahead and helping the community grow.

After lunch, a mini-dev-sprint was planned where the participants would come up with a project idea. Chandan, Praveen, Kushal and me were doing the mentoring. I was mentoring a group who are planning to build a recommendation system for restaurants (as of now). Based on my experience, I shared with them on which are the important things to focus first and follow the principle “release early and release often”. On the sidelines, I was also working on my project “invento”

The meetup ended at around 5 PM in the evening, though few people stayed up late working on the project.