Planet dgplug

April 22, 2019

Jason Braganza (Work)

Taking Notes While Reading

Riffing off last weeks mail on reading, Scott H Young has a really thorough, helpful post on how to take notes, while reading.

Starting with the why, and then onto various strategies on how to take notes, the post is well worth your time.

P.S. If you like my emails, forward it to your friends and ask them to subscribe too!


by Mario Jason Braganza at April 22, 2019 12:30 AM

April 18, 2019

Jason Braganza (Personal)

A Eulogy for Nana


Abby lost her grandmother this week.
This is her eulogy to her.

She was Aunty Matty to other people, mummy to her children and countless other fond names to who knew her.

But she was my Nana.

I have memories of her cradling me, and taking care of me as a baby.
Vacations at Nana’s were the highlight of my childhood years.
She was a tireless, hard working woman who raised her large family to the best of her abilities.
And not just her family, but also (to me it seemed) the whole neighbourhood.
She was loved and appreciated, just by about everyone whose life she touched.

As the years flew by, Nana seemed very out of place.
In our fast paced, always connected, no time for any one age, Nana was a slow, deliberate, thoughtful, kind, gentle and gracious woman, like someone from a different, more altruistic age.

And it was here in her shadow years, while i was grew up and was beginning to work and could make my own trips to see her, that i really began to see her for the strong willed, tireless, hard working that she was, beyond just my nana who cossetted me and made me nice things.

And after all these years, the only theme i see that has rung true throughout Nana’s life was, Nana was there.

  • when i was a young bawling baby, Nana was there.
  • throughout my growing up years, Nana was there.
  • to cook me what my heart desired, Nana was there.
  • for everyone in her life, Nana was there.
  • to crack jokes and lighten up any room, Nana was there.
  • to empathise and have a compassionate ear to whatever was ailing you, Nana was there.
  • to gently, yet firmly correct you, Nana was there.
  • to remember you on your birthdays and anniversaries, Nana was there.
  • to worry about you and pick you up when you were down, Nana was there.

Since the day before, when Nana left us, I feel distraught and left alone, that Nana wasn’t there.
And yet, as i read this little note, i realise that this is not quite true.
Like the Little Prince tells the author,

I cannot carry this body with me. It is too heavy.
But it will be like an old abandoned shell.
There is nothing sad about old shells …

Nana was in pain, and she moved on beyond her body to her rest.
But that does not mean, she isn’t there anymore.
I will always carry Nana with me.
We all do.
Nana is here.


by Mario Jason Braganza at April 18, 2019 04:19 PM

April 15, 2019

Jason Braganza (Work)

Want to Write Better? Become a Better Reader!

Busy with exams this month, so leaving you with this Austin Kleon post with tonnes of quotes on reading.

Here’s a few of my favourite ones …

“You can’t be a good writer without being a devoted reader.”
—J.K. Rowling

“Read, read, read everything — trash, classics, good and bad, and see how they do it. Just like a carpenter who works as an apprentice and studies the master. Read!”
—William Faulkner

“When I’m reading, I’m looking for something to steal. Readers ask me all the time the traditional question ‘Where do you get your ideas from?” I reply: ‘We are all having ideas all the time. But I’m on the lookout for them. You’re not.’”
—Philip Pullman

Go, read.

P.S. and forward this to your friends and ask them to subscribe!


by Mario Jason Braganza at April 15, 2019 12:30 AM

April 08, 2019

Subho

Increasing Postgres column name length

This blog is more like a bookmark for me, the solution was scavenged from internet. Recently I have been working on an analytics project where I had to generate pivot transpose tables from the data. Now this is the first time I faced the limitations set on postgres database. Since its a pivot, one of my column would be transposed and used as column names here, this is where things started breaking. Writing to postgres failed with error stating column names are not unique. After some digging I realized Postgres has a column name limitation of 63 bytes and anything more than that will be truncated hence post truncate multiple keys became the same causing this issue.

Next step was to look at the data in my column, it ranged from 20-300 characters long. I checked with redshift and Bigquery they had similar limitations too, 128 bytes. After looking for sometime found a solution, downloaded the postgres source, changed NAMEDATALEN to 301(remember column name length is always NAMEDATALEN – 1) src/include/pg_config_manual.h, followed the steps from postgres docs to compile the source and install and run postgres. This has been tested on Postgres 9.6 as of now and it works.

Next up I faced issues with maximum number columns, my pivot table had 1968 columns and postgres has a limitation of 1600 total columns. According to this answer I looked into the source comments and that looked quite overwhelming 😛 . Also I do not have a control over how many columns will be there post pivot so no matter whatever value i set , in future i might need more columns, so instead I handled the scenario in my application code to split the data across multiple tables and store them.

References:

  1. https://til.hashrocket.com/posts/8f87c65a0a-postgresqls-max-identifier-length-is-63-bytes
  2. https://stackoverflow.com/questions/6307317/how-to-change-postgres-table-field-name-limit
  3. https://www.postgresql.org/docs/9.6/install-short.html
  4. https://dba.stackexchange.com/questions/40137/in-postgresql-is-it-possible-to-change-the-maximum-number-of-columns-a-table-ca

by subho at April 08, 2019 09:25 AM

April 06, 2019

Tosin Damilare James Animashaun

To be Forearmed is to be Help-ready

I felt compelled to write this after my personal experience trying to get help with my code on IRC.

We all love to make the computer do things exactly the way we want, so some of us choose to take the bold step of learning to communicate with the machine. And it is not uncommon to find many of our burgeoning kind go from location to location on the web space trying to get help along the way. We are prompt to ask questions when we sight help.

When you learn to program, you are often encouraged to learn by doing.

The domain of computer programming or software development is a very practical one. Before now, I had carried this very principle everywhere with me -- in fact, preached it -- but hadn't really put it to use.

The thing about learning languages (or technologies) by reading big manuals is that, often times, beginners will approach the process like they would any other literature book. But that is clearly a wrong approach as empirical evidence has shown. You don't read these things to simply stomach them. Instead, you swallow and then post-process. In essence, you ruminate over stuff.


In truth, the only way you can really process what you read is to try things out and see results for yourself.

Weeks ago, while building an app, I visited IRC frequently to ask questions on just about everything that was unclear to me. While this mode of communication and seeking help is encouraged, abuse of it is strongly discouraged. The good guys over on the IRC channels get pissed off when it appears you're boycotting available resources like documentation, preferring to be spoonfed the whole time. (Remember this is not Quora, where the philosophy is for you to ask more and more questions).

This was the sort of thing that happened to me when I began flooding the channels with my persistent querying. Most of the time the IRC folks kept pointing me to the documentation, as workarounds for most of the issues I had were already documented. A lot of things became much clearer when I decided to finally read-the-docs.

What did I learn from that? "Do your own research!" It's so easy to skip this part, but if you make efforts at finding things out for yourself, you'll be surprised at how much you can dig out without having to bug people. However, this does not guarantee that even the few important questions you'll ask may not be met with hostility, but do not let that discourage you. The people who appear to be unwelcoming are doing so only as a way to discourage you from being over-dependent on the channel. Another advantage of finding things for yourself is that, you learn the why and not just the how.

I think it's fair to quote Armin Ronacher here,

"And it's not just asking questions; questioning other people, like what other people do or what you do yourself.

By far, the worst parts in all of my libraries are where I took the design from somewhere else. And it's not because I know better, it's because pretty much everything everybody does at any point in time has some sort of design decision behind it ... that major decision process.

So someone came up with a solution for a particular problem, and he thought about it and then wrote it down. But if you look at someone else's design, you might no longer understand why the decision was made in the first place. And ... in fact, if the original implementation is ten years old, who knows if the design ideas behind it are still entirely correct."


Personally, I like to answer questions and help put people on track. Nonetheless, if the queries got too overwhelming -- especially coming from the same person -- I would eventually lose interest in answering questions.


Let me remind you of some tidbits or etiquettes of IRC:

  • Construct your questions well (concise, well written and straight-to-the-point questions are more likely to attract help)

  • Don't EVER post code in a channel! Pastebin[1] it and share the link in the channel instead. While at it, don't post your entire code (unless you specifically need to). Post only the relevant portion -- the one you have an issue with. The only exception to this is if the snippet of code is considerably short, say one or two lines.

  • Don't be overly respectful. Yes, dont be too respectful -- cut all the 'Sirs'. Only be moderately polite.

  • Ensure you have and use a registered nick. This gives you an identity.

  • This last one is entirely my opinion but it's also based on what I have observed. Don't just be a leech, try to contribute to the community. Answer questions when you can.


So where do you look to before looking to IRC? There are three sources you may read from before turning to internet-relay-chat for help:

  • Read the documentation. * Documentation is the manual the creator or experts of a software product or tool provide their users with. So you want to know the ins and outs of a technology? That's the right place to look.

  • Read blog posts related to your topic-area. Blog posts are often based on people's experiences, so you're likely to find help from there, especially if the writer has faced the same issue. Remember to bookmark the really helpful ones as you go ;).

  • Last and very important. Read the source code!. This is two-fold: First is actually looking into your own code carefully and seeing what syntax or semantic errors you might have made. Secondly, you look into the original code of libraries/frameworks you are using if they are open source, otherwise revert to documentation. With this, you have it all stripped to its bare bones. Point blank! The source code reveals everything you need to know, once you know how to read it.


So why not arm yourself properly before going to post that question. That way, you would not only make it easier to get help [for yourself], but you would be better informed.



[1] Some Pastebin platforms I use:

Then Github provides a platform which supports version-controlled pastes: Github Gists.

Note: Because Hastebin heavily depends on Javascript, some people have complained of text-rendering issues possibly arising from browser-compatibility issues with it. So take caution using it. That said, I love its ease-of-use. It supports the use of keyboard shortcuts such as [Ctrl]+[S] to Save.

by Tosin Damilare James Animashaun at April 06, 2019 09:40 PM

March 28, 2019

Shakthi Kannan

"Opportunities in FLOSS" at Computer Society of India, Madras

I had given a talk on “Opportunities in Free (Libre) and Open Source Software”(FLOSS) on Saturday, March 2, 2019 at the Computer Society of India, Madras Chapter, jointly organized by the IEEE Computer Society, Madras Chapter and ACM India Chennai Professional Chapter. The Computer Society of India, Education Directorate is located opposite to the Institute of Mathematical Sciences in Taramani, close to the Tidel Park. Students and professors from IIT Madras, Anna University and engineering colleges in and around Chennai attended the event.

Session in progress

At around 6:00 p.m. people had assembled for networking, snacks and beverage. I started the “Building Careers with FLOSS” presentation at 6:30 p.m. Prof. Dr. D. Janakiram, CSE, IIT Madras also shared his insights on the benefits of learning from source code available under a FLOSS license. The session was very interactive and the audience asked a lot of good questions. Dr. B. Govindarajulu, author of the famous “IBM PC and Clones: Hardware, Troubleshooting and Maintenance” book then presented his views on creating a Computer History Museum in Chennai. Dinner was served around 8:00 p.m. at the venue.

Group photo

A review of my book has been published in Volume 14: No. 1, January-March 2019 IEEE India Council Newsletter. The excerpts of Chapter 4 of my book on “Project Guidelines” is also available. I had also written an article on “Seasons of Code” which is published in this edition of the IEEE newsletter.

Special thanks to Mr. H. R. Mohan, Editor, IEEE India for organizing the event and for the logistics support.

March 28, 2019 01:30 PM

March 27, 2019

Sayan Chowdhury

Kubernetes Day India 2019, Bangalore

Kubernetes Day India 2019, Bangalore

I returned to another conference with the jetlag of another conference, FOSSASIA but this conference was something I was very eagerly waiting to attend. Why? Well there are a couple of reasons, but the main reason for which I bought the ticket was to listen to Liz Rice in person.

The other reason was to see and meet the growing community of Kubernetes in Bangalore.

23rd March 2019, I woke up early morning and left along with Sayani, and Saptak to this place quite outside of Bangalore. Going to the venue almost felt like a weekend getaway trip.

Anyways we reached the venue at 8AM, and the Bangalore sun was already killing. CNCF had announced people would be eligible for KubeCon tickets if they came and collected their badges before 8:30AM. This was a nice trick because I could see a huge crowd in-front of me standing and collecting their badges. After collecting the badges, we headed back to the auditorium.

We grabbed some breakfast as soon as we reached the venue. The first talk/keynote did not delay much and started just after the breakfast.

Dan kicked off the event with an introduction to the conference, CNCF and a brief overview of the projects within CNCF.

Liz Rice, took the stage after Dan and talked on permission in Kubernetes. She gave a very nice ELI5 type analogy for permission/rbac in Kubernetes with file permission in Linux.

I moved outside after the talk and was mostly talking with people at the booths. Red Hat had it's booth also, so I spend some time talking to people about Fedora CoreOS and Silverblue

It was very nice to find the growing audience of Kubernetes, and I also happen to know about a couple of more interesting projects the community is building.


Photo by Cameron Venti on Unsplash

by Sayan Chowdhury at March 27, 2019 04:32 PM

March 26, 2019

Sayan Chowdhury

FOSSASIA 2019

FOSSASIA 2019

I was back to FOSSASIA this year after a gap of an year and nothing had changed other than the venue. The event had the same level of enthusiasm as before, though the crowd seemed to be less compared to the previous years.

This FOSSASIA was more special for the participants, organizers and the volunteers because this was the 10th year of the event. Kudos, to the organizing team for doing it successfully for last 10 years!

Day zero, the speakers informally gathered at the Lau Pa Sat street, where we shared some nice discussions along with sea food.

Sadly, the first day started out bad for me, with me waking up to bloodshot red eyes and burning sensation. I went back to have more some rest and later in the day headed to the conference after waking up.

The first day was a single-track event with the talks lined up one after another. Martin, from Debian, shared the varied differences and similarities within the open source communities. The panel on "What Opportunities Does "Open" Bring to Society?" was an interesting panel to sit through and listen to bunnie, Hong, Shanker, Dr. Graham, and Carsten share their opinions. Being there is the industry for long and embracing open source they shared their thoughts on how their employers embrace open source over time.

Recently, I've been looking IPFS (InterPlanetory File System) after we had a talk on the same during the Golang Bangalore Meetup. Jollen Chen in his talk shares how they are building distributed ledger/blochain from scratch for the IoT space. They use virtual blocks that provide a new blockchain design to ensure the real-time data transactions. This virtual blocks also add up with IPFS.

I lost track of time and could not attend the "Serverless with Knative".

The second day, I did not attend a lot of talks rather went around the booths, and made connections in the hallway tracks. Elastic, Upcloud, UI-licious, Indeed, Microsoft, IBM, and others. There were a couple of communities too who had put up their booth. CentOS also had a booth of it's own. We had our Fedora CoreOS stickers and Silverblue stickers up for display at the booth.

In talks, Rishav talked on the best practices to be followed while building Docker images. Manuel puts why/what problems is GraphQL  trying to solve in comparison to REST. The under/over fetching principle in GraphQL/REST seems to be interesting to me and I shall put some reading around it and try to get something implemented around it. Rohan took us through the magical land of babel and codemod. He explained how he refactored and migrated to a different library by writing custom codemod, and explaining JavaScript AST, babel parser on the way.

On the third day of the event, I went to talk by Harsha who was talking on building intelligent intrusion detection system, the first half of the talk mostly talks about the basics about IDS, and the second half on how he uses machine learning, and the fields to build this intelligent IDS. As expected, Jason Zaman gives an amazing talk on how to leverage SELinux to debug issues, for conditional policies, for policy types etc. He also demoed the various SELinux commands to use to tool effectively as an system administrator. Abhishek, from Swiggy, how Redis sits in-front of MySQL as a caching layer. He also demonstrated the plugin that he built and shared a demo on the performance of the tool.

The last day, I gave a talk on how I leverage the container-workflow tool on my primary workstation using Silverblue. I also shared why we built Silverblue in the first place, and what happens behind the scenes. The good thing was just after my talk Sinny's talk was scheduled which was on the upcoming  Fedora CoreOS.

I was actually happy that our talks had a good amount of audience, and there was good amount of enthusiasm from the crowd. Few people where interested in knowing the release cycle of the Silverblue/Fedora CoreOS and if it would be different from the one Container Linux followed which was maintaining channels. Also, an interesting question was if there was  limit to the number of package layering I can go while working with Silverblue.

Apart from the talk, I was reached out by a lot of people to know if Silverblue or the container-based workflow actually helps. I've to agree it's not an easy task to pitch a unconventional way to manage your system and something which is totally new. Still, at the end of the day, I hope I had good discussion around the topic with a lot of people.

I also happened to visit the Singapore Hackerspace which is one of the places I love to visit. On second visit we met a enthusiastic group of hackers and makers who are out on travelling South East Asia.

To end, I really had a good time at Singapore and FOSSASIA. I met a lot of people whom I did not meet for years, built connections to meet them in the future, and learnt quite a lot of things. And, I also happened to learn some really nice soldering technique at the very end.

À bientôt


Cover image by Michael Cannon (CC BY-SA 2.0)

by Sayan Chowdhury at March 26, 2019 09:06 AM

March 25, 2019

Jason Braganza (Personal)

Want to do Great Things?

Then be willing to look like an idiot.

(Claude) Shannon had courage. Who else but a man with almost infinite courage would ever think of averaging over all random codes and expect the average code would be good? He knew what he was doing was important and pursued it intensely. Courage, or confidence, is a property to develop in yourself. Look at your successes, and pay less attention to failures than you are usually advised to do in the expression, “Learn from your mistakes”. While playing chess Shannon would often advance his queen boldly into the fray and say, “I ain’t scaird of nothing”. I learned to repeat it to myself when stuck, and at times it has enabled me to go on to a success. I deliberately copied a part of the style of a great scientist. The courage to continue is essential since great research often has long periods with no success and many discouragements.

From Farnam Street’s article on doing great things.
There’s plenty more to do, than just be willing to look like an idiot though.
Go read the entire article.


If you like what I share, subscribe to the newsletter


by Mario Jason Braganza at March 25, 2019 12:30 AM

March 11, 2019

Saptak Sengupta

What to expect from GSoC?

If you were searching for a post about how to get selected in GSoC, and landed on this blog, you might be a little confused by the blog title. Because, well, this blog isn't about how to get selected in GSoC, or as many have sadly started saying, how to "crack" GSoC. There are plenty of blogs out there addressing that.

In this blog, I am going to write more about what you should expect from GSoC. And given that the applications for GSoC are about to start, it is high time you get your expectations right if you actually want to get the most out of it.
So let's get started.

1. Don't "Crack" GSoC

Unlike many other computer science and engineering programs, there is no pill that you take and you magically get selected overnight. Nor is there a particular curriculum or book that you can read over and over and practice and get selected.

Getting selected in GSoC is a gradual process that needs lots of patience and contribution and the slow but steady accrual of experience. So if you are actually reading this blog in an attempt to know how to get selected, you are kind of late in the process. Better late than never, though.

The only way you get selected in GSoC (at least in most organisations) is via Open Source Contributions. So now you might think, okay, open source contribution is the curriculum.

Yes and no.

If you consider open source as yet another chapter in your coursework, then getting started with open source contribution might be difficult.

There's a lot involved; from clean, readable coding to best practices to communication. It is an entirely new way of working (way of life?) that is going to last you forever and help you in the long run.

For many, like us (like me personally), it's more than even that. It's a belief, a principle, a movement. I am going to talk more about that in the points below.

So even though many blogs will tell you exactly how to pick an issue, and show many contributions and help you pick an easy organisation, I’d encourage you to enjoy the process and get involved wholeheartedly in it. Contribute to the project and organisation you feel excited about. Become a part of the organisation, get to know folks, learn as much as you can, expand your pool of knowledge. If after all that, the worst happens and you are not selected for GSoC, you can still keep contributing to a major open source project which is awesome!

2. Do it for Open Source, Not for Money

I know money is a really important part of life (and I am not denying that) and GSoC money is definitely tempting. So I am not complaining about the money being an intrinsic motivation. What I am trying to say is if you do GSoC only for the money, and stop contributing to Open Source after these 3 months, then the purpose of GSoC is lost.

GSoC, I believe, is meant to be a platform that helps you get started in your Open Source journey. It is that small little push that you need to start contributing to open source projects. Finding and contributing to an organisation all on your own might be a little difficult, GSoC provides you with a platform that helps you find them more easily and have a higher chance of starting your open source contribution in major organisation than you would normally do.

So use GSoC as a vehicle to begin your journey in the open source world. Once you start seeing it from that perspective, you will, hopefully appreciate the principles of open source and keep contributing to the open source world.

Open source projects appreciate great developers like you, so come be a part of it.

3. Take PR Reviews Positively

Now, if you have already grasped the previous points, you know GSoC is only the beginning. Apart from making all projects by you open source, a really important part of the journey is contributing to various wonderful open source projects, which is actually going to be most of your GSoC. And with contribution, comes pull requests (or patches in some cases).

Most times, you will receive plenty of comments and reviews on your pull requests. I have seen many folks get irritated. Many in face, feel that if you can't get a pull request merged without too many reviews, then that organisation is hard to contribute to, in GSoC. This causes many to try for organisations where pull requests get easily merged.

Don't be discouraged by the reviews.

Instead, use them as a learning opportunity. Most reviews are very constructive criticism that are will serve you well throughout your life. It will help you write code that is more readable, more efficient and code that works best in production both in terms of performance and maintainability. In GSoC you get to learn all this directly from upstream projects with super awesome developers and coders … and PR review is where you learn the most.

4. Collaborate, Don't Compete

Over the years, as both student and mentor in GSoC, I’ve seen participants duking it out for issues or work in organisations and projects. This is mainly because everyone has this feeling that if they solve more issues and bugs, they have a higher chance of getting selected or passing the evaluation. But at the same time, this often causes frustration if a PR is getting too many comments. Also, participants tend to start working on something different, leaving their previous work incomplete.

All these will actually just create a bad impression to the mentors and others in the organisation. It will deprive you of lots of peer learning opportunities because you will always be competing with everyone. So, try your best to collaborate with other participants and even the mentors and other contributors.

Collaboration is a central principle in the Open Source community at large. Collaborating with each other not only helps you learn a lot from your peers but also leads to a better, much cleaner project. Collaborate not only on code, but on shaping the best practices of a project, on blogs, on writing documentation and setting guidelines. You will also have a better overview of the entire project rather than just the small piece you work on.

Remember, GSoC is not a competition where you need to be the top scorer to win. Everyone is a winner if they contribute to the projects and help in growing the project. Believe me, most organisations will pass you even when you don't complete the entire proposal you made, if you made other quality contributions to the project and they feel that your work has helped in furthering development of the overall project.

5. Be Part of the Community

While being a part of GSoC, don't just code. Go, be a part of various open source communities. When you are selected for a particular organisation, be always active in their communication channels, be it IRC or slack or gitter or what have you. Help newcomers get started with the project, attend team meetings, make friends, and communicate with everyone. If possible, try to attend different meetups and conferences near your area.

These will help you network and make friends with a lot of people from different parts of the open source communities and you will get to learn even more. The best part about open source is that it allows you to grow beyond any boundaries and being part of different communities is one of the best ways to do this. Not only will you get to learn a lot code wise, but also about different aspects of life and technology and incidents that might help you shape your future.

And, most importantly, continue being involved in these communities even after GSoC ends. I have told plenty of folks, plenty of times before and I can't emphasise enough that GSoC is just the beginning of your journey. Your journey with open source coding and the communities, doesn't end after the 3 months of GSoC. It starts expanding. Yes, due to various circumstances, you might not always be able to actively contribute code to an open source project, but try to carve out time to help others in communities get started. Try to apply the lessons you learn in the communities, in your office work or university projects and when you do get time, contribute to the open source!

by SaptakS (noreply@blogger.com) at March 11, 2019 08:10 AM

February 23, 2019

Farhaan Bukhsh

The Late End Year Review – 2018

I know I am really, really late, but better late than never.
This past year has been really formative for me.

In this short personal retrospective post, I am just going to divide my experience into 3 categories, the good, the bad and the ugly best.

The Bad

  1. My father got really sick and I got really scared by the thought of losing him.
  2. I moved on from the first company I joined, because I was getting a bit stifled and yearned to learn and grow more.
  3. My brother got transferred, so I had to live without family for the first time in my life. I had never lived alone before this.
  4. I was not able to take the 3 month sabbatical, I thought I could.
  5. I couldn’t find a stable home and was on the run from one place to another constantly.

The Good

  1. I learnt how to live alone. I learnt how to find peace while being alone. Because of this, I could also explore more books and more importantly I could spend more time by myself figuring out what kind of person I want to become.
  2. I got a job with Clootrack, where people are amazing to work with and there is so much to learn.
  3. I found the chutzpah to quit my job, even thought I didn’t have a back up. In roundabout way, it gave me the strength to take risks in life and the courage to handle its consequences.
  4. Bad times help you discover good friends. I am not trying to boast about it, (but you are 😝– ed) but I am thankful to God that I have an overwhelming number of good friends.
  5. I got asked out in a coffee shop! This has never happened to me before. (BUT YES! THIS HAPPENED!).
  6. I wrote few poems this year, all of them heartfelt.
  7. I gained a measure of financial independence and the experience of how to handle things when everything is going south.
  8. I finally wrote a project and released it. I was fortunate enough to get few contributors.
  9. I am more aware now, and have stopped taking people and time for granted.
  10. Started Dosa Culture.
  11. Applied to more conferences to give a talk.

The Best

  1. I read more this year and got to learn about a lot of things from Feynman to Krebs. I explored fiction, non fiction, self help, and humour.
  2. I went to Varanasi (home) more than I ever did in the last five years of my life. I spent lots of time with my parents. I am planning to do it more.
  3. Went on a holiday to Pondicherry. I went for a holiday for the first time, with the money I saved up for the trip. I saw the sunrise of 1st January sitting on Rock beach.
  4. Got rejected at all the conferences I applied for. No matter. It motivates me even more, to try harder, to dance on the edge, to learn more, do more. It helps me strive for greatness, while also being a good reality check.
  5. Spent more time on working on hobby projects and contributing to open source.
  6. Got a chance to be a visiting faculty, and teach programming in college.
  7. Lived more! Learnt More! Loved More!

I feel I might be missing quite a few things in the lists, but these are the few, that helped me grow as a person. They impacted me deeply and changed my way of looking at life.

I hope the coming year brings better experiences and more learning!

Until then,
Live Long and Prosper! (so cheesy – ed)

by fardroid23 at February 23, 2019 03:13 PM

February 13, 2019

Kushal Das

Tracking my phone's silent connections

My phone has more friends than me. It talks to more peers (computers) than the number of human beings I talk on an average. In this age of smartphones and mobile apps for A-Z things, we are dependent on these technologies. However, at the same time, we don’t know much of what is going on in the computers equipped with powerful cameras, GPS device, microphone we are carrying all the time. All these apps are talking to their respective servers (or can we call them masters?), but, there is no easy way to track them.

These questions bothered me for a long time: I wanted to see the servers my phone is connecting to, and I want to block those connections as I wish. However, I never managed to work on this. A few weeks ago, I finally sat down to start working to build up a system by reusing already available open source projects and tools to create the system, which will allow me to track what my phone is doing. Maybe not in full details, but, at least shed some light on the network traffic from the phone.

Initial trial

I tried to create a wifi hotspot at home using a Raspberry Pi and then started capturing all the packets from the device using standard tools (dumpcap) and later reading through the logs using Wireshark. This procedure meant that I could only capture when I am connected to the network at home. What about when I am not at home?

Next round

This time I took a bit different approach. I chose algo to create a VPN server. Using WireGuard, it became straightforward to connect my iPhone to the VPN. This process also allows capturing all the traffic from the phone very easily on the VPN server. A few days in the experiment, Kashmir started posting her experiment named Life Without the Tech Giants, where she started blocking all the services from 5 big technology companies. With her help, I contacted Dhruv Mehrotra, who is a technologist behind the story. After talking to him, I felt that I am going in the right direction. He already posted details on how they did the blocking, and you can try that at home :)

Looking at the data after 1 week

After capturing the data for the first week, I moved the captured pcap files into my computer. Wrote some Python code to put the data into a SQLite database, enabling me to query the data much faster.

Domain Name System (DNS) data

The Domain Name System (DNS) is a decentralized system which helps to translate the human memory safe domain names (like kushaldas.in) into Internet Protocol (IP) addresses (like 192.168.1.1 ). Computers talk to each other using these IP addresses, we, don’t have to worry to remember so many names. When the developers develop their applications for the phone, they generally use those domain names to specify where the app should connect.

If I plot all the different domains (including any subdomain) which got queried at least 10 times in a week, we see the following graph.

The first thing to notice is how the phone is trying to find servers from Apple, which makes sense as this is an iPhone. I use the mobile Twitter app a lot, so we also see many queries related to Twitter. Lookout is a special mention there, it was suggested to me by my friends who understand these technologies and security better than me. The 3rd position is taken by Google, though sometimes I watch Youtube videos, but, the phone queried for many other Google domains.

There are also many queries to Akamai CDN service, and I could not find any easy way to identify those hosts, the same with Amazon AWS related hosts. If you know any better way, please drop me a note.

You can see a lot of data analytics related companies were also queried. dev.appboy.com is a major one, and thankfully algo already blocked that domain in the DNS level. I don’t know which app is trying to connect to which all servers, I found about a few of the apps in my phone by searching about the client list of the above-mentioned analytics companies. Next, in coming months, I will start blocking those hosts/domains one by one and see which all apps stop working.

Looking at data flow

The number of DNS queries is an easy start, but, next I wanted to learn more about the actual servers my phone is talking to. The paranoid part inside of me was pushing for discovering these servers.

If we put all of the major companies the phone is talking to, we get the following graph.

Apple is leading the chart by taking 44% of all the connections, and the number is 495225 times. Twitter is in the second place, and Edgecastcdn is in the third. My phone talked to Google servers 67344 number of times, which is like 7 times less than the number of times Apple itself.

In the next graph, I removed the big players (including Google and Amazon). Then, I can see that analytics companies like nflxso.net and mparticle.com have 31% of the connections, which is a lot. Most probably I will start with blocking these two first. The 3 other CDN companies, Akamai, Cloudfront, and Cloudflare has 8%, 7%, and 6% respectively. Do I know what all things are these companies tracking? Nope, and that is scary enough that one of my friend commented “It makes me think about throwing my phone in the garbage.”

What about encrypted vs unencrypted traffic? What all protocols are being used? I tried to find the answer for the first question, and the answer looks like the following graph. Maybe the number will come down if I try to refine the query and add other parameters, that is a future task.

What next?

As I said earlier, I am working on creating a set of tools, which then can be deployed on the VPN server, that will provide a user-friendly way to monitor, and block/unblock traffic from their phone. The major part of the work is to make sure that the whole thing is easy to deploy, and can be used by someone with less technical knowledge.

How can you help?

The biggest thing we need is the knowledge of “How to analyze the data we are capturing?”. It is one thing to make reports for personal user, but, trying to help others is an entirely different game altogether. We will, of course, need all sorts of contributions to the project. Before anything else, we will have to join the random code we have, into a proper project structure. Keep following this blog for more updates and details about the project.

Note to self

Do not try to read data after midnight, or else I will again think a local address as some random dynamic address in Bangkok and freak out (thank you reverse-dns).

by Kushal Das at February 13, 2019 02:47 AM

February 10, 2019

Kushal Das

When I was sleepy

Back in 2005 I joined my first job, in a software company in Bangalore. It was a backend of a big foreign bank. We trained heavily on different parts of software development during the first few months. At the same time, I had an altercation with the senior manager (about some Java code) who was in charge of the new joinees and their placement within the company. The result? Everyone else got a team but me, and I had to roam around within the office to find an empty seat and wait there till the actual seat owner came back. I managed to spend a lot of days in the cafeteria on the rooftop. But, then they made new rules that one can not sit there either, other than at lunch time.

So, I went asking around, talking to all the different people in the office (there were 500+ folks iirc) if they know any team who would take on a fresher. I tried to throw in words like Linux, open source to better my chances. And then one day, I heard that the research and development team was looking for someone with Linux and PHP skills. I went in to have a chat with the team, and they told me the problem (it was actually on DSpace, a Java based documentation/content repository system), and after looking at my resume decided to give me a desktop for couple of weeks. I managed to solve the problem in next few days, and after a week or so, I was told that I will join the team. There were couple of super senior managers and I was the only kid on that block. Being part of this team allowed me to explore different technologies and programming languages.

I will later write down my experiences in more detail, but for today, I want to focus on one particular incident. The kind of incident, which all system administrators experience at least once in their life (I guess). I got root access to the production server of the DSpace installation within a few weeks. I had a Windows desktop, and used putty to ssh in to the server. As this company was backend of the big bank, except for a few senior managers, no one else had access to Internet on their systems. There were 2 desktops in the kiosk in the ground floor, and one had to stand in a long queue to get a chance to access Internet.

One day I came back from the lunch (a good one), and was feeling a bit sleepy. I had taken down the tomcat server, pushed the changes to the application, and then wanted to start the server up again. Typed the whole path to startup.sh (I don’t remember the actual name, I’m just guessing it was startup.sh) and hit Enter. I was waiting for the long screens of messages this startup script spewed as it started up, but instead, I got back the prompt quickly. I was wondering what went wrong. Then, looking at the monitor very closely, I suddenly realised that I was planning to delete some other file and I had written rm at the beginning of the command prompt, forgotten it, and then typed the path of the startup.sh. Suddenly I felt the place get very hot and stuffy; I started sweating and all blood drained from my face in the next few moments. I was at panic level 9. I was wondering what to do. I thought about the next steps to follow. I still had a small window of time to fix the service. Suddenly I realized that I can get a copy of the script from the Internet (yay, Open Source!). So, I picked up a pad and a pen, ran down to the ground floor, and stood in the queue to get access to a computer with Internet. After getting the seat, I started writing down the whole startup.sh on the pad and double checked it. Ran right back up to my cubicle, feverishly typed in the script, (somehow miraculously without any typo in one go.) As I executed the script, I saw the familiar output, messages scrolling up, screen after joyful screen. And finally as it started up, I sighed a huge sigh of relief. And after the adrenalin levels came down, I wrote an incident report to my management, and later talked about it during a meeting.

From that day on, before doing any kind of destructive operation, I double check the command prompt for any typo. I make sure, that I don’t remove anything randomly and also make sure that I have my backups is place.

by Kushal Das at February 10, 2019 02:31 AM

February 05, 2019

Anwesha Das

Have a safer internet

Today, 5th February is the safer internet day. The primary aim of this day is to advance the safe and positive use of digital technology for children and young people. Moreover, it promotes the conversation over this issue. So let us discuss a few ideas.The digital medium is the place where we live our today. It has become our world. However, as compare to the physical world to this world and its rules are unfamiliar to us. Also, adding to that with the advent of social media we are putting our lives, every detail of it in and at the domain of social media. We are then letting governments, industrial lords, political parties, snoops, and the society to judge, to see and monitor us. We, the fragile, vulnerable us, do not have any other option but to watch our freedom, privacy vanishing.

Do we not have anything to save ourselves? Ta Da! Here are some basic ideas are the following which you can try to follow in your everyday life to keep yourself safe in the digital world.

Use unique passphrases

Use passphrases instead of passwords.Passwords are easy to break as well as easy to copy so instead of using “Frank” (a name) or “Achinpur60”(a part of your address), use passphrases like “DiscountDangerDumpster”. It is easy to remember and hard to break. You can assemble 2 more languages (it is easy for us, Indians, right?). I used diceware to generate that password. Moreover, by unique what I mean is that do not use the SAME PASSWORD EVERYWHERE. I can feel how difficult, tedious, impossible it is for you to remember all the lengthy, difficult passphrases (now not passwords remember!) for all your accounts. However, nothing can be done with this. If someone can get your passphrase for one account, he will be able to all of them. Unique passphrases help a lot in this case.

Use password managers

To solve your above-mentioned problem of remembering long passphrases you have a magic thing called password manager. Just move your wand (read mouse) once, and you can find your long passphrases safely protected in their safe vaults. There are many different password managers LastPass, KeePassXC, etc. If you want to know more about this, please read it here.

Do not leave your device (computer, phone, etc) unlocked

My 2 year old once had typed some not so kind words (thanks to autocorrect) to my in-laws and the lovely consequence it brought still shivers me. But thankfully so it was not with someone, having the good technical knowledge and not so good intention, who could cause much greater damage if unlucky then irrecoverable damage than this. So please do not leave your device unlocked.

Do not share your password or your device with anyone

The similar kinds of danger, as aforementioned it poses if you share your password with anyone.

Do block your webcam and phone’s camera

It is now a well-known fact that attackers are spying on us through our web cameras. They are deceiving users by installing webcam spyware. Many of us may think “oh we are safe, our device has indicator lights, so we will be knowing when and if there is any video recording happening.” It is very much possible to disable the activity light by changing the configurations and software hacks. So even if there is no light, your video can very well be taken.

Do not ignore security updates

Most of the time when a security update notification pops up in the morning we very smoothly ignore it for our morning dose of news or checking our social media feed. However, that is the most irresponsible thing you want to do in your day. It may be last chance to secure yourself from the future danger. Mainer times the cyber attackers take advantage of your old, outdated software and attack you through it. It may be your old PDF reader, web browser or your operating system. So, this the most primary thing to your digital security lesson 101 is to keep your software up to date.

Acquire some basic knowledge about your machine

I know (trust me I have passed the phase) please acquire some basic knowledge about your machine, eg which version of operating system you are using, the other software on your machine and their version number. If and when they require any updates or not.

Do not download random websites from the internet.

Do not download random websites from the internet they might contain malware, virus. It might not only affect your machine but all the devices in the network. So, please check the website you are downloading from.

The same caution as above goes for this also. Do not click on the random URLs you receive over email or social media sites.

Use two-factor authentication

Two-factor authentication merely is two steps of validation. It adds an extra layer of security in and for your device. In 2FA the user needs to put two passwords instead of one. It is advisable that you have your 2FA installed on your mobile phone, or even better, use a hardware token like Yubikey. So that if someone wants to hack your account, then they have to get hold of both password and the phone.

Use Tor network

Tor Project is the most trusted and proposed project to remain private, to retain your anonymity. Tor is defined as “free software and an open network that helps you defend against traffic analysis, a form of network surveillance that threatens personal freedom and privacy, confidential business activities, and relationships, and state security.” in their website. Have a look at this to know more.

Take proper legal action

If something terrible happens to you online, please visit the local cyber crime department and lodge a formal complaint there. The local police stations do not deal with the matter related to cyber crimes. So you might directly want to go to the appropriate cyber security cell. If you do not have any idea that where is it, what to do there etc. You go to your Local Police Station take their advice, the information you need and then go the cyber security cell.

Learn GPG encryption

It is always suggested to know and learn GPG, Gnu Privacy Guard if you are up to that level of learning technical things. It is a bit difficult, but, surely a very useful tool to keep your privacy secured.

The steps I mentioned above may sound "too much" to maintain. But let us pretend that your house is your device and password is key to enter there. You normally follow all possible way to keep your house keys safe so the same rules apply here also. The rules are nothing but an habit,like getting up in the morning, it seems difficult for first few times but after that it is organic and normal as it can be. So, build the habbit of keeping safe, only using these tools will not offer you the desired results you need.

Hope you have a happy, safe life in the digital world.

by Anwesha Das at February 05, 2019 05:40 PM

January 12, 2019

Jaysinh Shukla

Python 3.7 feature walkthrough

In this post, I will explain improvements done in Core Python version 3.7. Below is the outline of features covered in this post.

  • Breakpoints

  • Subprocess

  • Dataclass

  • Namedtuples

  • Hash-based Python object file

breakpoint()

Breakpoint is an extremely important tool for debugging. Since I started learning Python, I am using the same API for putting breakpoints. With this release, breakpoint() is introduced as a built-in function. Because it is in a built-in scope, you don’t have to import it from any module. You can call this function to put breakpoints in your code. This approach is handier than importing pdb.set_trace().

Breakpoint function in Python 3.7

Code used in above example

for i in range(100):
    if i == 10:
        breakpoint()
    else:
        print(i)

PYTHONBREAKPOINT

There wasn’t any handy option to disable or enable existing breakpoints with a single flag. But with this release, you can certainly reduce your pain by using PYTHONBREAKPOINT environment variable. You can disable all breakpoints in your code by setting the environment variable PYTHONBREAKPOINT to 0.

Breakpoint environment variable in Python 3.7

I advise putting “PYTHONBREAKPOINT=0” in your production environment to avoid unwanted pausing at forgotten breakpoints

Subprocess.run(capture_output=True)

You can pipe the output of Standard Output Stream (stdout) and Standard Error Stream (stderr) by enabling capture_output parameter of subprocess.run() function.

subprocess.run got capture_output parameter

You should note that it is an improvement over piping the stream manually. For example, subprocess.run(["ls", "-l", "/var"], stdout=subprocess.PIPE, stderr=subprocess.PIPE) was the previous approach to capture the output of stdout and stderr.

Dataclasses

The new class level decorator @dataclass introduced with the dataclasses module. Python is well-known for achieving more by writing less. It seems that this module will receive more updates in future which can be applied to reduce significant line of code. Basic understanding of Typehints is expected to understand this feature.

When you wrap your class with the @dataclass decorator, the decorator will put obvious constructor code for you. Additionally, it defines a behaviour for dander methods __repr__(), __eq__() and __hash__().

Dataclasses.dataclass

Below is the code before introducing a dataclasses.dataclass decorator.

class Point:

    def __init__(self, x, y):
        self.x = x
        self.y = y

After wrapping with @dataclass decorator it reduces to below code

from dataclasses import dataclass


@dataclass
class Point:
    x: float
    y: float

Namedtuples

The namedtuples are a very helpful data structure, yet I found it is less known amongst developers. With this release, you can set default values to argument variables.

Namedtuples with default arguments

Note: Default arguments will be assigned from left to right. In the above example, default value 2 will be assigned to variable y

Below is the code used in the example

from collections import namedtuple


Point = namedtuple("Point", ["x", "y"], defaults=[2,])
p = Point(1)
print(p)

.pyc

.pyc are object files generated everytime you change your code file (.py). It is a collection of meta-data created by an interpreter for an executed code. The interpreter will use this data when you re-execute this code next time. Present approach to identify an outdated object file is done by comparing meta fields of source code file like last edited date. With this release, that identification process is improved by comparing files using a hash-based approach. The hash-based approach is quick and consistent across various platforms than comparing last edited dates. This improvement is considered unstable. Core python will continue with the metadata approach and slowly migrate to the hash-based approach.

Summary

  • Calling breakpoint() will put a breakpoint in your code.

  • Disable all breakpoints in your code by setting an environment variable PYTHONBREAKPOINT=0.

  • subprocess.run([...], capture_output=True) will capture the output of stdout and stderr.

  • Class level decorator @dataclass will define default logic for constructor function. It will implement default logic for dunder methods __repr__(), ___eq__() and __hash__().

  • Namedtuple data structure supports default values to its arguments using defaults.

  • Outdated Python object files (.pyc) are compared using the hash-based approach.

I hope you were able to learn something new by reading this post. If you want to read an in-depth discussion on each feature introduced in Python 3.7, then please read this official post. Happy hacking!

Proofreaders: Jason Braganza, Ninpo, basen_ from #python at Freenode, Ultron from #python-offtopic at Freenode, up|ime from ##English at Freenode

by Jaysinh Shukla at January 12, 2019 07:53 PM