Planet dgplug

August 17, 2017

Anwesha Das

DreamHost fighting to protect the fundamental rights of its users

Habeas data, my data my right, is the ethos of the right to be a free and fulfilling individual. It offers the individual to be him/herself without being monitored.

In The United States, there are several salvos to protect and further the concept.

The First Amendment

The First Amendment (Amendment I) to the United States Constitution establishes the

  • Freedom of speech,
  • Freedom of the press,
  • Freedom exercise of religion,
  • Freedom to assemble peaceably
The Fourth Amendment

The Fourth Amendment(Amendment IV) to the United States Constitution

  • prohibits unreasonable searches and seizures;
  • establishes The right to privacy is protected by the US Constitution, though the exact word privacy has not been used. But if we have a close look to the Amendment to the Bill of Rights, it bars Government to issue a general warrant.
The Privacy Protection Act, 1980

The Act protects press, journalists, media house, newsroom from the search conducted by the government office bearers. It mandates that it shall be unlawful for a government employee to search for or seize “work product” or “documentary materials” that are possessed by a person “in connection with a purpose to disseminate to the public a newspaper, book, broadcast, or other similar form of public communication”, in connection with the investigation or prosecution of a criminal offense, [42 U.S.C. §§ 2000aa (a), (b) (1996)]. An order, a subpoena is necessary for accessing the information, documents.

But the Government for the time again have violated, disregarded these mandates and stepped outside their periphery in the name of security of the state.

The present situation with DreamHost

DreamHost is A Los Angeles based company(private). It provides the following services, Web hosting service, Cloud computing service, Cloud storage service, Domain name registrar. The company since past few months is fighting a legal battle to protect their and one of their customer’s, disruptj20.org fundamental right.

What is disruptj20.org?

The company hosts disruptj20.org in the web. It is a website which organized, encouraged willing individuals to participate against the present US Government. Wikipedia says - “DisruptJ20 (also Disrupt J20), a Washington, D.C.-based political organization founded in July 2016 and publicly launched on November 11 of the same year, stated its initial aim as protesting and disrupting events of the presidential inauguration of the 45th U.S.”

The Search Warrant

There was a Search Warrant issued against DreamHost. It requires them to
disclose, give away “the information associated with www.disruptj20.org that is stored at the premises owned, maintained, controlled, or operated by DreamHost,” [ATTACHMENT A].

The particular list of information to be disclosed and information to be seized by the government can be seen at ATTACHMENT B.

How it affects third parties (other than www.disruptj20.org)?

It demands to reveal to the government of “all files” related to the website, which includes the HTTP logs for the visitors, - means

  • the time and date of the visit,
  • the IP address for the visitor,
  • the website pages viewed by the visitor (through their IP address),
  • the detailed description of the software running in the visitor’s computer.
  • details of emails by the third party to the www.disruptj20.org.

Responding to it the company challenged the Department of Justice (on the warrant). They made an attempt to quash the demand of seizure and disclosure of the information by due legal process and reason.

Motion to show cause

In a usual course of action, the DOJ would respond to the inquiries of DreamHost. But here instead of answering to their inquiries, DOJ chose to file a motion to show cause in the Washington, D.C. Superior Court. DOJ asked for an order to compel them to produce the records,

The Opposition

The Opposition for the denial of the above mentioned motion filed by DreamHost filed an Opposition for the denial of the above mentioned motion. The “Argument” part shows/claims/demonstrates

  • How the Search Warrant Violates the Fourth Amendment and the Privacy Protection Act.
  • The Search Warrant requires the details of the visitors to the website.This severely endangers their freedom of speech (The First Amendment). As described by DreamHost on their blog.

“This motion is our latest salvo in what has become a months-long battle to protect the identities of thousands of unwitting internet users.”

Electronic Frontier Foundation has led their support, help to DreamHost, though they are not representing them in the court. The matter will be heard on August 18 in Washington, D.C.

There are different kinds of securities. Security for state power is a kind that is constantly protected. In contrast to the security for the population which is constantly denied, negated, curbed and restrained. By looking at the series of events, the documentary records of this particular incident raises a doubt -

  • If the Government action is motivated by security or not?

The only security in this case is probably is being considered is security to stay in power, noticing the nature, subject of the website. Now it is the high time that people should stand to save individual’s, commoner’s right to have private space, opinion and protest. Kudous to DreamHost to protect the primary fundamental right of and individual - Privacy.

by Anwesha Das at August 17, 2017 07:13 PM

August 15, 2017

Mario Jason Braganza

Book Review – i want 2 do project. tell me wat 2 do

Click me to buy!


TL;DR? It’s awesome. Buy it right now.

I was looking to dip my toes into some sort of structured help with the summer training and open source in general, because while I knew what I wanted, I just didn’t know how to go about it.

And then I realised that one of our mentors had actually gone and written a whole book on the how to.
So, I bought the paperback.
The binding is really good, the paper really nice (unlike other tech books I’ve read) and the words large enough to read.
I expect to get a lot of use, out of the book.

And lot of use is right.
While it’s a slim volume and a pretty quick read, the book is pretty dense when it comes to the wisdom it imparts.

The book has a simple (yet substantial to execute) premise.
You’ve just tipped your toe into programming, or you’ve learnt a new language, or you’ve probably written a few programs or maybe you’re just brand new.
You want to explore the vast thrilling world that is Open Source Software.
What now?

“i want 2 do project. tell me wat 2 do.” answers the “what now” in painstaking detail.

From communication (Mailing List Guidelines) to the importance of focus (Attention to Detail) to working with mentors (the Project chapters) to the tools (Methodology & tools) to the importance of sharpening the saw (Reading …) and finally the importance of your environment (Sustenance), the book covers the entire gamut that a student or a novice programmer with open source would go through.

Shakthi writes like he speaks; pithily, concisely with the weight of his experience behind his words.

The book is chockfull of quotes (from the Lady Lovelace to Menaechmus to Taleb) that lend heft to the chapters.
The references at the end of each chapter will probably keep me busy for the next few months.

The book’ll save you enormous amounts of time and heartache, in your journey, were you to heed its advice.
It’s that good.

by Mario Jason Braganza at August 15, 2017 01:08 PM

Saptak Sengupta

Uploading Images via APIs in the Open Event Server

APIs help us to send and receive data in some particular data format that can then be used individually or integrated with a frontend UI. In our case, the entire API server is used to manage all the requests from the frontend and send back the necessary response. Usually, the application is to send simple form data which is then stored into the backend database and a valid jsonapi response is shown. However other than normal text, url, datetime data one very important data is media files, in our case event images, user images, slides, etc. In this blog, we will particularly deal with how we can upload images in the server using API.

Sending Data


Firstly, we need to decide how do we send the data in the post request of the API. So we are sending a base64 encoded string representing the image along with the image extension appended to it, for example, data:image/png;base64,iVBORw0KGgoAAAANS. This is a widely used format for showing images over the web. So when we send a POST request we send a json encoded body like:

{
"data": "data:image/png;base64,iVBORw0KGgoAAAANS"
}

Converting Base64 Data to Image


There are 2 parts of the data in the string that we receive. The first part basically tells us the format of the image(.gif in this case) and string encoding(base64 in this case), the second part gives us the encoded string. So, from the first part, we extract the file extension for the image file to be created. We use uuid.uuid4() for a random filename.
filename = '{}.{}'.format(str(uuid.uuid4()).data.split(";")[0].split("/")[1])

Now to write the base64 encoded string as an image file, we first need to get only the encoded string part from the data and then decode it. We use string decode function of python for the decoding purpose. Then we write the data to the image file and save it.
file.write(data.split(",")[1].decode("base64")

API Response


Finally using whatever logic you are using for serving your static files, you generate the static file path for the image saved. And then create another json encoded response which returns you the url for the saved image in the server.
{
"url": "https://xyz.storage.com/asd/fgh/hjk/1233456.png"
}

And there you have your image uploaded and served.

by SaptakS (noreply@blogger.com) at August 15, 2017 12:22 PM

August 11, 2017

Shakthi Kannan

StumpWM on Parabola GNU/Linux-libre

I have installed and configured StumpWM (v1.0.0) on Parabola GNU/Linux-libre. StumpWM is a keyboard driven, tiling Window Manager written in Common Lisp (CL). It is thus very hackable, and you can customize its configuration by writing Lisp code and reload it dynamically. The installation and configuration steps are provided below for reference.

Firstly, the Steel Bank Common Lisp (SBCL) compiler needs to be installed on the system using Pacman.

$ sudo pacman -S sbcl 
$ sudo pacman -S acpi xtrlock emacs mate-terminal iceweasel

The acpi utility is installed to show the battery status in the modeline. The xtrlock utility is used to lock the screen. When the screen is locked, you will need to enter your user password to unlock the same. GNU Emacs, Iceweasel and Mate-terminal software packages are also installed. Quicklisp is a package manager for Common Lisp and is used to install the StumpWM dependencies with SBCL.

An installation directory is created at ~/downloads/quicklisp, and the quicklisplisp file is fetched as follows:

$ mkdir ~/downloads/quicklisp
$ cd ~/downloads/quicklisp
$ curl -O https://beta.quicklisp.org/quicklisp.lisp

The quicklisp.lisp file is loaded using the SBCL interpreter. The installation steps and output are shown below:

~/downloads/quicklisp $ sbcl --load quicklisp.lisp 

    This is SBCL 1.3.17, an implementation of ANSI Common Lisp.
    More information about SBCL is available at <http://www.sbcl.org/>.

    SBCL is free software, provided as is, with absolutely no warranty.
    It is mostly in the public domain; some portions are provided under
    BSD-style licenses.  See the CREDITS and COPYING files in the
    distribution for more information.

      ==== quicklisp quickstart 2015-01-28 loaded ====

        To continue with installation, evaluate: (quicklisp-quickstart:install)

        For installation options, evaluate: (quicklisp-quickstart:help)

* (quicklisp-quickstart:install)

    ; Fetching #<URL "http://beta.quicklisp.org/client/quicklisp.sexp">
    ; 0.82KB
    ==================================================
    838 bytes in 0.00 seconds (0.00KB/sec)
    ; Fetching #<URL "http://beta.quicklisp.org/client/2017-03-06/quicklisp.tar">
    ; 250.00KB
    ==================================================
    256,000 bytes in 0.16 seconds (1602.56KB/sec)
    ; Fetching #<URL "http://beta.quicklisp.org/client/2015-09-24/setup.lisp">
    ; 4.94KB
    ==================================================
    5,054 bytes in 0.00 seconds (0.00KB/sec)
    ; Fetching #<URL "http://beta.quicklisp.org/asdf/2.26/asdf.lisp">
    ; 194.07KB
    ==================================================
    198,729 bytes in 0.09 seconds (2086.79KB/sec)
    ; Fetching #<URL "http://beta.quicklisp.org/dist/quicklisp.txt">
    ; 0.40KB
    ==================================================
    408 bytes in 0.00 seconds (0.00KB/sec)
    Installing dist "quicklisp" version "2017-07-25".
    ; Fetching #<URL "http://beta.quicklisp.org/dist/quicklisp/2017-07-25/releases.txt">
    ; 372.80KB
    ==================================================
    381,744 bytes in 0.15 seconds (2485.31KB/sec)
    ; Fetching #<URL "http://beta.quicklisp.org/dist/quicklisp/2017-07-25/systems.txt">
    ; 241.23KB
    ==================================================
    247,022 bytes in 0.10 seconds (2297.45KB/sec)

      ==== quicklisp installed ====

        To load a system, use: (ql:quickload "system-name")

        To find systems, use: (ql:system-apropos "term")

        To load Quicklisp every time you start Lisp, use: (ql:add-to-init-file)

        For more information, see http://www.quicklisp.org/beta/

    NIL

* (ql:add-to-init-file)

    I will append the following lines to #P"/home/guest/.sbclrc":

      ;;; The following lines added by ql:add-to-init-file:
      #-quicklisp
      (let ((quicklisp-init (merge-pathnames "quicklisp/setup.lisp"
                                             (user-homedir-pathname))))
        (when (probe-file quicklisp-init)
          (load quicklisp-init)))

    Press Enter to continue.

    #P"/home/guest/.sbclrc"

* (ql:quickload "clx")

    To load "clx":
      Install 1 Quicklisp release:
        clx
    ; Fetching #<URL "http://beta.quicklisp.org/archive/clx/2017-06-30/clx-20170630-git.tgz">
    ; 452.92KB
    ==================================================
    463,786 bytes in 0.12 seconds (3806.02KB/sec)
    ; Loading "clx"
    [package xlib]....................................
    ..................................................
    ..................................................
    ..................................................
    ..................................................
    ..................................................
    ..................................................
    ..................................................
    [package xlib/glx]................................
    [package xlib/gl].................................
    [package xlib/dpms]...............................
    [package xlib/xtest]..............................
    [package xlib/xinerama]...........................
    [package xlib-demo/clclock].......................
    [package xlib-demo/clipboard].....................
    [package xlib-demo/demos].........................
    [package xlib-demo/gl-test].......................
    [package xlib-demo/mandel].............
    ("clx")

* (ql:quickload "cl-ppcre")

    To load "cl-ppcre":
      Install 1 Quicklisp release:
        cl-ppcre
    ; Fetching #<URL "http://beta.quicklisp.org/archive/cl-ppcre/2015-09-23/cl-ppcre-2.0.11.tgz">
    ; 156.08KB
    ==================================================
    159,829 bytes in 0.08 seconds (1903.45KB/sec)
    ; Loading "cl-ppcre"
    [package cl-ppcre]................................
    ................
    ("cl-ppcre")

* (ql:quickload "alexandria")

    To load "alexandria":
      Install 1 Quicklisp release:
        alexandria
    ; Fetching #<URL "http://beta.quicklisp.org/archive/alexandria/2017-06-30/alexandria-20170630-git.tgz">
    ; 49.97KB
    ==================================================
    51,168 bytes in 0.04 seconds (1135.65KB/sec)
    ; Loading "alexandria"
    [package alexandria.0.dev].......................
    ("alexandria")

* (exit)

~/downloads/quicklisp $ 

The StumpWM v1.0.0 sources are fetched to ~/downloads/stumpwm directory, extracted and compiled using the following steps:

$ mkdir ~/downloads/stumpwm
$ cd ~/downloads/stumpwm
$ wget https://github.com/stumpwm/stumpwm/archive/1.0.0.tar.gz
$ tar xzvf 1.0.0.tar.gz
$ cd stumpwm-1.0.0/
$ ./autogen.sh 
$ ./configure 
$ make

You can then install the built stumpwm binary, which will get copied to /usr/local/bin/stumpwm as shown below:

$ sudo make install

$ which stumpwm
/usr/local/bin/stumpwm

I use the Lightweight X11 Display Manager (lxdm), and thus created a /usr/share/xsessions/stumpwm.desktop file to login to StumpWM from the login manager:

[Desktop Entry]
Encoding=UTF-8
Name=StumpWM
Comment=StumpWM
Exec=/usr/local/bin/stumpwm
Type=Application

A sample ~/.stumpwmrc configuration file is given below (should be self-explanatory):

;; -*-lisp-*-

(in-package :stumpwm)

;; Startup message: display the machine's name
(setf *startup-message* (machine-instance))

;; Turn on the modeline
(if (not (head-mode-line (current-head)))
    (toggle-mode-line (current-screen) (current-head)))

;; Lock screen
(define-key *root-map* (kbd "L") "exec xtrlock")

;; I like messages to be centered on the screen.
(setf *message-window-gravity* :center)
(setf *input-window-gravity* :center)

;; I thought that this mode-line was fabulous!
(defvar *battery-status-command*
  "acpi -b | awk -F '[ ,]' '{printf \"%s%s\", $3, $5}' | sed s/Discharging/\-/ | sed s/Unknown// | sed s/Full// | sed s/Charging/+/")

(defvar *vol-status-command*
  "amixer get Master | grep \"[[:digit:]]\\+%\" -o | tr -d \"\\n\"")

(setf *screen-mode-line-format*
      (list "%w [^B%n^b] ^>"
      '(:eval (run-shell-command *battery-status-command* t))
      " | Vol. "
      '(:eval (run-shell-command *vol-status-command* t))
      " | %d"))

;; urxvt
(define-key *root-map* (kbd "c") 
  "exec urxvt +sb -fn \"xft:Ubuntu Mono:pixelsize=15,style=regular\"")

;; Window information format
(setf *window-info-format* "%wx%h %n (%t - %c)")

;; Window format
(setf *window-format* "%m%n%s%10t")

;; Emacs
(defvar *emacs-command* nil
  "Start an emacs client frame. Starts an emacs daemon if necessary.")
(setf *emacs-command* "bash -c -i 'emacsclient -c -a \"\"'")

(define-key *root-map* (kbd "e") "run-emacs")

(defcommand run-emacs () ()
    (run-shell-command (concat "exec " *emacs-command*)))

;; iceweasel
(defcommand iceweasel-browser () ()
  "run iceweasel"
  (run-or-raise "iceweasel" '(:instance "iceweasel")))
(define-key *root-map* (kbd "b") "iceweasel-browser")

;; mate-terminal
(defcommand mate-terminal () ()
  (run-or-raise "mate-terminal --hide-menubar" '(:class "mate-terminal")))

(define-key *root-map* (kbd "C") "mate-terminal")

;; Clear rules
(clear-window-placement-rules)

(define-frame-preference "Default"
  ;; frame raise lock (lock AND raise == jumpto)
  (0 t   t :instance "emacs")
  (1 t   t :instance "iceweasel-browser"))

;; dvorak and தமிழ்
(stumpwm:run-shell-command "sh -c 'setxkbmap us,in dvorak,tam_unicode grp:ctrls_toggle'")

;; Start default applications
(run-emacs)
(iceweasel-browser)

On logging into StumpWM, GNU Emacs and Iceweasel browser are opened automatically. I also switch between English and Tamil keyboard layouts when required, and the two Control keys are used to toggle between them. A StumpWM screenshot is shown below:

StumpWM screenshot

You are encouraged to read the StumpWM manual to know more about its usage and configuration.

August 11, 2017 09:00 PM

Anwesha Das

Blocking of archive.org in India

My morning starts with the quick look at twitter.

Yesterday some tweets drew my attention - The Internet Archive over at http://web.archive.org has been blocked in India. There was a long thread going on, and people were asking, updating thoughts and opinions on the matter.
Ok, this is something important.

What is archive.org?

I searched. I did not have any idea about this previously.
I found that one of the most important projects on the site is “The WayBack machine”; for the site serves/saves/documents the history of the world wide web. Therefore it gives you a salient history of different sites on a single search.

But for last few days many people, in India, could not access archive.org and The Wayback Machine.
Instead, this is what people saw

Specifically, the http://web.archive.org has been blocked but when you add the secure “s”; https://web.archive.org can be accessed.

It was important to have information about which ISPs were actually blocking the site. Tweets poured in from people, across the country. You can find an elaborate list of these ISPs in this blog post by Medianama.

Officials from archive.org were trying to contact the authorities, in India, at the Ministry of Electronics and Information Technology (MEITY) and the The Telecom Department (DoT). Unfortunately they never responded.

Recent update:

According to a tweet, the site was banned according to a judgment made by The Madras High Court, in Prakash Jha Productions v/s Bharat Sanchar Nigam Limited and Others (being, O.A.Nos. 705 to 707 of 2017 in C.S. No. 576 of 2017). It could not be confirmed due to the unavailability of the original judgment document.
According to the tweet by Anivar Arvind, it was a John Doe order. John Doe order means an order passed by the competent Court of Justice against unidentified defendants.

The confusion:

The silence and lack of/no communication on the part of the Government authorities created a lot of confusion.

There were questions such as -
1. Why is the site blocked?
2. Who blocked the site?
3. Is there is any provision to remove the block? And measures (if any) to be taken.

Unfortunately the request is not been heard by the concerned authorities till now.

The sole requirement in this context is, transparency and clarity.
We, the common people have the right to know what is happening in our digital space.
Denying that is a repudiation of a basic and primary right of ours.

The situation has posed a serious question in front of us,
Is this a step taken by the authorities to curb, and restrain our digital (fundamental) rights? (though the Government is trying its level best to prove that we do not have one.)

by Anwesha Das at August 11, 2017 07:49 AM

July 21, 2017

Jaysinh Shukla

PyDelhi Conf 2017: A beautiful conference happened in New Delhi, India

PyDelhi Conf 2017

TL;DR

PyDelhi conf 2017 was a two-day conference which featured workshops, dev sprints, both full-length and lightning talks. There were workshop sessions without any extra charges. Delhiites should not miss the chance to attend this conference in future. I conducted a workshop titled “Tango with Django” helping beginners to understand the Django web framework.

Detailed Review

About the PyDelhi community

PyDelhi Community

PyDelhi conf 2017 volunteers

The PyDelhi community was known as NCR Python Users Group before few years. This community is performing a role of an umbrella organization for other FLOSS communities across New Delhi, India. They are actively arranging monthly meetups on interesting topics. Last PyCon India which is a national level conference of Python programming language was impressively organized by this community. This year too they took the responsibility of managing it. I am very thankful to this community for their immense contribution to this society. If you are around New Delhi, India then you should not miss the chance to attend their meetups. This community has great people who are always happy to mentor.

PyDelhi conf 2017

Conference T-shirt

Conference T-shirt

PyDelhi conf is a regional level conference of Python programming language organized by PyDelhi community. It is their second year organizing this conference. Last year it was located at JNU University. This year it happened at IIM, Lucknow campus based in Noida, New Delhi, India. I enjoyed various talks which I will mention later here, a workshops section because I was conducting one and some panel discussions because people involved were having a good level of experience. 80% of the time slot was divided equally between 30 minutes talk and 2-hour workshop section. 10% were given to panel discussions and 10% was reserved for lightning talks. The dev sprints were happening in parallel with the conference. The early slot was given to workshops for both the days. One large conference hall was located on a 2nd floor of the building and two halls at the ground floor. Food and beverages were served on the base floor.

Panel discussion

Panel Discussion

Desk

Registration desk

Lunch

Tea break

Keynote speakers

Mr. Richardo Rocha

  • Mr. Ricardo Rocha: Mr. Rocha is a software engineer at CERN. I got some time to talk with him post-conference. We discussed his responsibilities at CERN. I was impressed when he explained how he is managing infrastructure with his team. On inquiring opportunities available at CERN he mentioned that the organization is always looking for the talented developers. New grads can keep an eye on various Summer Internship Programs which are very similar to Google Summer of Code program.

Mr. Chris Stucchio

  • Mr. Chris Stucchio: Mr. Stucchio is director of Data Science at Wingify/ VWO. I found him physically fit compared to other software developers (mostly of India). I didn’t get much time to have a word with him.

Interesting Talks

Because I took the wrong metro train, I was late for the inaugural ceremony. I also missed a keynote given by Mr. Rocha. Below talks were impressively presented at the conference.

I love discussing with people rather than sit in on sessions. With that ace-reason, I always lose some important talks presented at the conference. I do not forget to watch them once they are publicly available. This year I missed following talks.

Volunteer Party

I got a warm invitation by the organizers to join the volunteer party, but I was little tensed about my session happening on the next day. So, I decided to go home and improve the slides. I heard from friends that the party was awesome!

My workshop session

Tango with Django

Me conducting workshop

I conducted a workshop on Django web framework. “Tango with Django” was chosen as a title with a thought of attracting beginners. I believe this title is already a name of famous book solving the same purpose.

Dev sprints

Dev sprints

Me hacking at dev sprints section

The dev sprints were happening parallel with the conference. Mr. Pillai was representing Junction. I decided to test few issues of CPython but didn’t do much. There were a bunch of people hacking but didn’t find anything interesting. The quality of chairs was so an impressive that I have decided to buy the same for my home office.

Why attend this conference?

  • Free Workshops: The conference has great slot of talks and workshops. Workshops are being conducted by field experts without expecting any other fees. This can be one of the great advantages you leverage from this conference.

  • Student discounts: If you are a student then you will receive a discount on the conference ticket.

  • Beginner friendly platform: If you are novice speaker than you will get mentorship from this community. You can conduct a session for beginners.

  • Networking: You will find senior employees of tech giants, owner of innovative start-ups and professors from well-known universities participating in this conference. It can be a good opportunity for you to network with them.

What was missing?

  • Lecture hall arrangement: It was difficult to frequently travel to the second floor and come back to the ground floor. I found most people were spending their time on the ground floor rather than attending talks going on upstairs.

  • No corporate stalls: Despite having corporate sponsors like Microsoft I didn’t find any stall of any company.

  • The venue for dev sprints: The rooms were designed for teleconference containing circularly arranged wooden tables. This was not creating a collaborative environment. Involved projects were not frequently promoted during the conference.

Thank you PyDelhi community!

I would like to thank all the known, unknown volunteers who performed their best in arranging this conference. I am encouraging PyDelhi community for keep organizing such an affable conference.

Proofreaders: Mr. Daniel Foerster, Mr. Dhavan Vaidya, Mr. Sayan Chowdhury, Mr. Trent Buck

by Jaysinh Shukla at July 21, 2017 12:44 PM

Mario Jason Braganza

On Margin

This post is for my younger colleagues on the student planet.
Yes kids, this is a long post. Suck it up and read it over the weekend.
This is important stuff.
And please go, read this on the blog.



Introduction

In The Intelligent Investor, published in 1949, Benjamin Graham[^n] wrote:

In the old legend the wise men finally boiled down the history of mortal affairs into the single phrase, “This too will pass.”

Confronted with a like challenge to distill the secret of sound investment into three words, we venture the motto;

MARGIN OF SAFETY.

… the function of the margin of safety is, in essence, that of rendering unnecessary an accurate estimate of the future.

(emphasis mine)

What does that mean to me?

That in order to do anything with my life, in any area, I should have enough to face the worst that I think, could happen to me.

I’ll simplify it further.

Margin is having enough room to breathe;
and knowing what “enough” means in every sphere of life.

It has implications throughout life and across various societal disciplines even.
Ben Graham’s quote above comes from the field of finance.
What about engineering?

Consider a highly-engineered jet engine part. If the part were to fail, the engine would also fail, perhaps at the worst possible moment—while in flight with passengers on board … We’ve designed the part for 10,000 hours of average flying time.
That brings us to a central question: After how many hours of service do we replace this critical part? 9,999 hours? Why replace it any sooner than we have to? Why waste money?

The first problem is, we know nothing of the composition of the 10,000 hours any individual part has gone through. Were they 10,000 particularly tough hours, filled with turbulent skies? Was it all relatively smooth sailing?
And, how confident are we, that the part will really last the full 10,000 hours? What if it had a slight flaw during manufacturing? What if we made an assumption about its reliability that was not conservative enough? What if the material degraded in bad weather to a degree we didn’t foresee?

The challenge is clear, and the implication obvious: we do not wait until the part has been in service for 9,999 hours. Perhaps at 7,000 hours, we seriously consider replacing the part, and we put a hard stop at 7,500 hours.

The difference between waiting until the last minute and replacing it comfortably early gives us a margin of safety. We leave ourselves a cushion. (Ever notice how your gas tank indicator goes on long before you’re really on empty? It’s the same idea.)

Go read the whole post. It’s awesome.


So, Why Do We Need Margin?

To paraphrase Dr. Richard Swenson, who’s literally written the book on Margin (the book has a moral, Christian bent. Don’t let that bother you);

  • Progress gives us more and more, which leads to
  • stress, change, complexity, speed, intensity, and overload.
  • which is bad, because all humans have physical, mental, emotional, and financial limits that are relatively fixed
  • Thus progress is on a collision course with human limits and once we can’t take progress any more, we flame and burn out. On the unsaturated side of our limits, humans can be open and expansive. Cross them however, and the rules of life totally change.

What would you think if this page had no margins?
What would you think of me if I crammed the print top to bottom and side to side so that every blank space was filled up?
The result would be aesthetically displeasing and chaotic.

Like some of our lives!

Marginless, according to Swenson, is being thirty minutes late to the doctors office because you were twenty minutes late getting out of the bank because you were ten minutes late dropping the kids off at school because the car ran out of fuel two klicks from the petrol pump and you forgot your wallet.
Margin, on the other hand, is having breath left at the top of the staircase, money left at the end of the month, and sanity left at the end of adolescence.

No Margin, causes hyperstress and overload. (terms that to folks like us, need no definition)

Here’s Dr. Swenson again,

To be healthy, we require margin in at least four areas:

  • emotional energy,
  • physical energy,
  • time,
  • and finances.

Yet, right now for the most part we are emotional wrecks, stressed, alone, and so exhausted in spirit.
Physically, we are overfed, underactive, and sleep-deprived.
We have no time, at all.
And when it comes to finances, our bank balances look as barren as the surface of Mars.


What Can We Do about It?

Thank Pareto, grab the low hanging fruit and do the twenty percent of the work that’ll yield the most margin in your life.

Remember, we live lives, full of intention now :)

Let’s intentionally tackle each domain and see what we can do.

Emotional:

  • Bond with your family
  • Make time for friends
  • Switch off the TV! … and your other screens
  • Get a pet or go play with doggies in the park
  • Pray, meditate.

Physical:

  • Eat less, move more
  • Walk, Run!
  • Cut out sugar
  • Drink water.
  • Have the discipline (or build it up) to keep at it.
    (I didn’t have any and I’m paying the price!)
  • Most importantly, get enough sleep.

Time:

  • You’ll never “find time”. Be intentional. Make time!
  • Learn to plan your day and your week.
  • Plan your day with plenty of spaces in the middle
  • Did you cut out FB? Twitter? TV? :P
  • Learn to say No. No is a complete sentence.

Financial:

  • Stop eating out! Learn to make do.
  • The only debt you have should be your home. (and no more than 40% of income at that)
  • Have 6 months living expenses in the bank.
  • Spend way less than you make. (The bigger the buffer, the more chaos you can handle.)
  • Be comfortable with charity and giving now, when it’s hard to do. Like Dave Ramsey says, “If you will live like no one else, later you can live and give like no one else.”

And finally here’s James Clear, echoing Taleb
Leave Room for the Unexpected!

If your life is designed only to handle the expected challenges, then it will fall apart as soon as something unexpected happens to you.
Always be stronger than you need to be.
Always leave room for the unexpected.

Thank you for reading. Here’s a puppy.


This is the bonus section

Want to hear from an actual practitioner?

So what is the lesson?
Save your current income, even as you build complementary skills like frugality, adaptability, and the ability to earn money in more than one way.
Then you’ll be ready to retire much earlier, because the sum of your safety margins will be enough to make you feel comfortable taking the leap.
And most importantly, stop worrying!

That’s Mr. Money Mustache (real name, Pete Adeney), software developer who retired at age 30! (watch the video)


Resources:
(in the grand tradition of @mbuf feeding us well with knowledge)

P.S. This post is part of the Life Outside Code series.


by Mario Jason Braganza at July 21, 2017 12:07 PM

July 17, 2017

Kushal Das

Encrypting drives with LUKS

Encrypting hard drives should be a common step in our regular computer usage. If nothing else, this will help you sleep well, in case you lose your computer (theft) or that small USB disk you were carrying in your pocket. In this guide, I’ll explain how to encrypt your USB disks so that you have peace of mind, in case you lose them.

But, before we dig into the technical details, always remember the following from XKCD.

What is LUKS?

LUKS or Linux Unified Key Setup is a disk encryption specification, first introduced in 2004 by Clemens Fruhwirth. Notice the word specification; instead of trying to implement something of its own, LUKS is a standard way of doing drive encryption across tools and distributions. You can even use drives from Windows using the LibreCrypt application.

For the following example, I am going to use a standard 16 GB USB stick as my external drive.

Formatting the drive

Note: check the drive name/path twice before you press enter for any of the commands below. A mistake, might destroy your primary drive, and there is no way to recover the data. So, execute with caution.

In my case, the drive is detected as /dev/sdb. It is always a good idea to format the drive before you start using it. You can use wipefs tool to clean any signature from the device,

$ sudo wipefs -a /dev/sdb1

Then you can use fdisk tool to delete the old partitions , and create a new primary partition.

Next step is to create the LUKS partition.

$ sudo cryptsetup luksFormat /dev/sdb1

WARNING!
========
This will overwrite data on /dev/sdb1 irrevocably.

Are you sure? (Type uppercase yes): YES
Enter passphrase: 
Verify passphrase:

Opening up the encrypted drive and creating a filesystem

Next, we will open up the drive using the passphrase we just gave, and create a filesystem on the device.

$ sudo cryptsetup luksOpen /dev/sdb1 reddrive
Enter passphrase for /dev/sdb1
$ ls -l /dev/mapper/reddrive
lrwxrwxrwx. 1 root root 7 Jul 17 10:18 /dev/mapper/reddrive -> ../dm-5

I am going to create an EXT4 filesystem on here.
Feel free to create which ever filesystem you want.

$ sudo mkfs.ext4 /dev/mapper/reddrive -L reddrive
mke2fs 1.43.4 (31-Jan-2017)
Creating filesystem with 3815424 4k blocks and 954720 inodes
Filesystem UUID: b00be39d-4656-4022-92ea-6a518b08f1e1
Superblock backups stored on blocks: 
        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208

Allocating group tables: done                            
Writing inode tables: done                            
Creating journal (16384 blocks): done
Writing superblocks and filesystem accounting information: done   

Mounting, using, and unmounting the drive

The device is now ready to use. You can manually mount it with the mount command. Any of the modern desktops will ask you to unlock using the passphrase if you connect the device (or try to double click on the file browser).

I will show the command line option. I will create a file hello.txt as an example.

$ sudo mount /dev/mapper/reddrive /mnt/red
$ su -c "echo hello > /mnt/red/hello.txt"
Password:
$ ls -l /mnt/red
total 20
-rw-rw-r--. 1 root root     6 Jul 17 10:26 hello.txt
drwx------. 2 root root 16384 Jul 17 10:21 lost+found
$ sudo umount /mnt/red
$ sudo cryptsetup luksClose reddrive

When I attach the drive to my system, the file browser asks me to unlock it using the following dialog. Remember to choose forget immediately so that the file browser forgets the password.

On passphrases

The FAQ entry on cryptsetup page, give us hints and suggestions about passphrase creation.

If paranoid, add at least 20 bit. That is roughly four additional characters for random passphrases and roughly 32 characters for a random English sentence.

Key slots aka different passphrases

In LUKS, we get 8 different key slots (for passphrases) for each device(partition). You can see them using luksDump sub-command.

$ sudo cryptsetup luksDump /dev/sdb1 | grep Slot
Key Slot 0: ENABLED
Key Slot 1: DISABLED
Key Slot 2: DISABLED
Key Slot 3: DISABLED
Key Slot 4: DISABLED
Key Slot 5: DISABLED
Key Slot 6: DISABLED
Key Slot 7: DISABLED

Adding a new key

The following command adds a new key to the drive.

$ sudo cryptsetup luksAddKey /dev/sdb1 -S 5
Enter any existing passphrase: 
Enter new passphrase for key slot: 
Verify passphrase:

You will have to use any of the existing passphrases to add a new key.

$  sudo cryptsetup luksDump /dev/sdb1 | grep Slot
Key Slot 0: ENABLED
Key Slot 1: DISABLED
Key Slot 2: DISABLED
Key Slot 3: DISABLED
Key Slot 4: DISABLED
Key Slot 5: ENABLED
Key Slot 6: DISABLED
Key Slot 7: DISABLED

Removing a passphrase

Remember that removing a passphrase is based on the passphrase itself, not by the key slot number.

$ sudo cryptsetup luksRemoveKey /dev/sdb1
Enter passphrase to be deleted: 
$ sudo cryptsetup luksDump /dev/sdb1 | grep Slot
Key Slot 0: ENABLED
Key Slot 1: DISABLED
Key Slot 2: DISABLED
Key Slot 3: DISABLED
Key Slot 4: DISABLED
Key Slot 5: DISABLED
Key Slot 6: DISABLED
Key Slot 7: DISABLED

Now in case you don’t know the passphrase, then you can use luksKillSlot.

$ sudo cryptsetup luksKillSlot /dev/sdb1 3
Enter any remaining passphrase:

Overview of the disk layout

The disk layout looks like the following. The header or phdr contains various details like magic value, version, cipher name, following the 8 keyblocks (marked as kb1, kb2.. in the drawing), and then the encrypted bulk data block. We can see all of those details in the C structure.

struct luks_phdr {
        char            magic[LUKS_MAGIC_L];
        uint16_t        version;
        char            cipherName[LUKS_CIPHERNAME_L];
        char            cipherMode[LUKS_CIPHERMODE_L];
        char            hashSpec[LUKS_HASHSPEC_L];
        uint32_t        payloadOffset;
        uint32_t        keyBytes;
        char            mkDigest[LUKS_DIGESTSIZE];
        char            mkDigestSalt[LUKS_SALTSIZE];
        uint32_t        mkDigestIterations;
        char            uuid[UUID_STRING_L];

        struct {
                uint32_t active;

                /* parameters used for password processing */
                uint32_t passwordIterations;
                char     passwordSalt[LUKS_SALTSIZE];

                /* parameters used for AF store/load */
                uint32_t keyMaterialOffset;
                uint32_t stripes;
        } keyblock[LUKS_NUMKEYS];

        /* Align it to 512 sector size */
        char                _padding[432];
};

Each (active) keyblock contains an encrypted copy of the master key. When we enter the passphrase, it unlocks the master key, that in turn unlocks the encrypted data.

But, remember, all of this is of no use if you have a very simple passphrase. We have another XKCD to explain this.

I hope this post encourages you to use encrypted drives more. All of my computers have their drives encrypted; (I do that while installing the Operating System.) This means, without decrypting the drive you can not boot the system properly. On a related note, remember to turn off your computer completely, (not hibernation or suspend mode) when you’re traveling.

by Kushal Das at July 17, 2017 09:10 AM

July 12, 2017

Kushal Das

Article on Hacker Ethic and Free Software movement

As I have mentioned in the dgplug summer training page, focusing on the Free Software movement is a big part of this year’s training program. A few weeks back there was a tweet from @gnome about travel ban, and many could not figure out why Gnome was writing about this topic. Amongst the many proper replies, Miguel de Icaza’s reply was to the point. This incident made Anwesha and me stop and think; and then made us rethink, about how we wanted to conduct the sessions on the Free Software movement and Software Licensing.

I was born in the beginning of the 80s and Anwesha even later. Our introduction to the movement was from the stories we heard (from many people); from Levy’s famous book, Hackers: Heroes of the Computer Revolution and the seminal Free as in Freedom.

My introduction to the FSF came through ilug-calcutta, and from Sayamindu. Later, at foss.in 2005, I made another friend (for life), Praveen A (he is from the same batch). And even later through out various conferences, I was introduced to other members of FSF India. In 2007, I was part of the 4th GPLv3 meet organizing team in Bangalore. That was my introduction to RMS, and his personality (I will write a blog post later about various incidents from that conference). That had a big impact on me.

Coming back to the story of the tweet, we also saw similar ignorance from newcomers, as they never got a chance to learn about the past, nor did get to meet the various people involved (distance and time). So, Anwesha and I, tried to write a brief history, including the hacker ethic, and beginning of the Free Software movement. A lot of stories mentioned in the article are from the books mentioned above. At the very end, I have written about how the different software we use everyday came about initially. I took the help of various FSF bulletins for the same.

This Monday I took a session on the same topic in the #dgplug IRC channel. When I made mention of the GNU C Library and the time Ronald McGrath started it, Siddhesh called attention to an announcement he (Ronald) made a few days ago (about stepping down from maintainership of that same GNU Library). I also pointed that Siddhesh is now one of the maintainer of Glibc. That gave the students a sense of impact and immediacy; a feeling of involvement and ownership.

Today evening from 13:30 UTC, Anwesha took a session on Software Licenses 101 in the #dgplug channel on Freenode. There will be more follow up sessions in the coming days.

Link to the article once again

by Kushal Das at July 12, 2017 04:28 PM

July 05, 2017

Shakthi Kannan

"i want 2 do project. tell me wat 2 do" workshop, reserved-bit, Pune

I had organized a one-day workshop based on my book i want 2 do project. tell me wat 2 do at reserved-bit, Pune on Saturday, June 3, 2017. Thanks to Nisha Poyarekar and Siddhesh Poyarekar for providing their makerspace as venue for the workshop.

Workshop in progress

The objective of the workshop is to share the methodology and best practices on working with Free and Open Source Software projects, and also to guide the participants on career options. Although there is plenty of free documentation (including my own) available on the subject, some people prefer a formal, systematic approach to learning in a private coaching environment. Hence, this workshop is tailored for the same and is made a paid workshop, similar to personal tutoring.

The book has been in circulation for many years. So, I had to give two options - pay only for the workshop, or pay for both the workshop and the book (if they have not already bought the book). I have also kept additional copies of my book at the reserved-bit makerspace if people are interested. I had covered the following topics:

  1. Careers with F/OSS
  2. “i want 2 do project. tell me wat 2 do” best practices
  3. Introduction to Git
  4. Hands-on problem solving exercises
  5. Real-world project example
  6. Goal-driven development

The feedback has been positive and I am sharing the same below:

Feedback 1
Feedback 2
Feedback 3
Feedback 4
Feedback 5

If you are interested in attending such a workshop, do write to me: author at shakthimaan dot com.

July 05, 2017 01:45 PM

June 26, 2017

Sanyam Khurana

Why you should contribute to Open Source

My first step with FOSS was during the summer of 2014, where I got introduced to DGPLUG (Durgapur Linux Users Group). Every year, they conduct a training over IRC covering a wide range of topics. Since then, I've been contributing to different projects, primarily to Mozilla's Gecko Engine. Recently I also started contributing to CPython.

Over this journey as CuriousLearner over the past few years, I've been fortunate enough to collaborate with some of the best engineers, learnt a lot of skills & gained some nice perspective on different technologies by just contributing to Open Source. I owe my learning to them.

I've been always asked this question by my friends, colleagues, and strangers whom I often run into in local meet-ups and conferences.

  • Why should I contribute to Open Source
  • What's in it for me

So, I thought I would cover this topic from the perspective of a student as well as the perspective of a working professional.

Learn new skills

When I say skills, I'm not referring to just coding skills or learning yet another framework. There are tons of skills to be learnt. These include coding, debugging, testing, writing docs, collaboration, participating in meetings etc. In FOSS, you won't just learn technical skills, but soft skills as well. That means you'll learn how to communicate effectively over different mediums and making sense to other people who might be from different backgrounds.

I've seen many engineers are hesitant when it comes to communication. They can think and write really nice code, but when it comes to explaining themselves, they lack the communication skills, which is vital to be a better professional.

Improve existing skills

It's not always about learning a new tech that you heard about. It's important to improve on things you think you might already know. Contributing to a FOSS project would make you learn intricate details that your mentors have gained working as a software craftsmen for several years.

Meet like-minded people

Many organizations do stand-up / all-hands meetings where everyone comes together and explains what are they're working on, what they plan to do next and if they face any blocker. So, students, there is a high possibility that when you graduate, you'll do this in your organization :) You'll meet a lot of people having similar interests and you'll learn a lot of things just by hanging out with them (most of the time on IRC).

Learn people skills

Learn team management, resolving internal conflicts, brainstorming on a new feature, public-speaking, time-management, conducting meet-ups & learn to prioritize your work. You will just expand your horizons.

Learn to READ

One of the best things I've been ever taught is to "READ". (And those who've met me personally knows how many times I've been repeating this thing :)).

When I say read, I mean you'll read the documentation, development guide and most importantly the source code.

Always remember that code is meant to be read more times than it is written. When you contribute to any FOSS project, one of the important skill you add to your skill-set is to make your brain make sense out of the code that may look gibberish to many people.

This is a very important skill. Once you're in a company, you'll be given the responsibility of maintaining huge code base and constantly adding features to it. There won't be someone who would spoon-feed you by explaining what every line of code means.

I've seen many people struggling hard in their jobs to figure out what a particular piece of code does. But if you've already worked with humongous code bases, this shall be a cake-walk for you. You would already know how to scrutinize bugs, debug errors and most importantly, would have already taught your brain to think and READ.

Find mentors

FOSS is all about mentors & mentees. There are people just like me and you who volunteer their time and skill to do good for the community. You'll find a lot of mentors who would be willing to teach you different skills. Remember, that you've got this opportunity to learn from domain experts :)

And no matter what, always respect everyone's time.

Teach others

While it is important to get your code reviewed, it is much more important to review other's code. This would help you in learning different coding style and also about how someone approached a particular problem. This would help in expanding your logical skills & as always you'll be learning a lot yourself along the way with having healthy discussions in code reviews.

It feels great to make (small) changes

You don't necessarily need to work on big features in a project or become a life-long contributor. Contribute to a project that you use on daily basis (like Mozilla Firefox) and I promise you that it would be gratifying when you see the whole world rely on the small code changes you did.

A small change in the documentation means you might save someone countless hours of debugging and scratching their head on how to use a particular piece of code. Always remember, documentation bugs are not something inferior; you'll learn a lot in writing documentation, right from the tools such as Sphinx and rST to expanding your domain knowledge about the project.

Building your reputation (& career) along the way

All the code you'll write in Open Source projects would be public and can be taken as a demonstration of how you write code & what you can do. It depicts how you work with others and most importantly give you an enormous amount of experience in working with people having diversified skill-set.

Once you get acquainted with the code base, you'll be invited to develop stand-alone features and who knows you might get that commit bit turned on when you've developed enough reputation in the community ;)

Apart from all the things mentioned above, I think Mozilla's mission says a lot about why I contribute to FOSS

Doing good is part of our code

Alright, I didn't realize, I'll be able to write this much. Hopefully, this would help & motivate people to get involved in contributing to Open Source. As always, if you have any queries, or want help in contributing to any project, feel free to mail me on Sanyam [at] SanyamKhurana [dot] com.

by Sanyam Khurana at June 26, 2017 11:41 AM

June 22, 2017

Saptak Sengupta

Avoid Race Condition in CSV exports





One of the biggest problems that appear while writing any file in the server side is a race condition. A race condition when two or more users make request to write to the same file in the server side and since neither of the operations are completed, so there is a certain problem in the integrity of the data written and may also cause the operation to be completely blocked. So how to stop this and still maintain that a large number of files or memory is not used? Let’s see.


What is a Race Condition?


A race condition is a situation when two or more processes perform a write operation on the same file at the same time. This causes an undesirable situation since the process should be in a sequential manner. So the situation causes something like a ‘race’ between the two server requests to change or modify the same file. So in such a scenario, there is no guarantee that the data written to the file is correct or will be meaningful at the end. Maximum probability is it won’t be.


Avoiding Race Condition


The general way of avoiding a race condition is adding a file lock. We lock a file while it is being written by some user or process and prevent other users from being able to write to the file. They have to wait for the ongoing writing process to complete to start another write process. But making that might increase the wait time quite considerably for users when we are looking to export CSV data about an event.







So to keep the process fast and use the advantage of distributed system, we write the CSV for different request in different files. We create the filenames by appending a random hexadecimal string to the original filename.


Avoid Wastage of Memory


However as we can understand by the proposed method above, there will be a huge wastage of memory since a large amount of files will be created. So to avoid wastage of memory, we run a cron job. The cron job deletes all the csv files after a certain amount of time. For this, we use apscheduler.







Thus, we avoid the race condition while also maintaining that a lot of memory isn’t wasted.

by SaptakS (noreply@blogger.com) at June 22, 2017 09:45 PM

June 20, 2017

Sanyam Khurana

Contributing enhancement patch to CPython

I started contributing to CPython this year from PyCon Pune sprints. That was one of the best PyCon I've ever attended. Since then, I've spent some time in going through documentation, core dev-guide and now I'm still reading about internals of the Python programming language.

I recently contributed an enhancement patch to Python 3.7 which might be back-ported to Python 3.6. Let's understand the scenario and the enhancement:

New contributor to Python 3 might use old Python 2 syntax treating print as a statement rather than a function and then see an error like:

>>> print "hello"
  File "<stdin>", line 1
    print "hello"
                ^
SyntaxError: Missing parentheses in call to 'print'

The task was to improve the error message based on different kinds of user input. This shall ensure that we don't only show error to the user but also suggest the correct syntax to be used as a hint. So after the patch being applied, the error message shall say this:

>>> print "hello"
  File "<stdin>", line 1
    print "hello"
                ^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print("hello")?

Sounds easy? Right? No, we aren't done yet. This was a simple print statement dealing with just a string. Potentially someone can try to print anything. What if someone wants to print a tuple?

>>> print 4,
  File "<stdin>", line 1
    print 4,
          ^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print(4, end=" ")?

So, the correct syntax is suggested to the user according to Python3.

Along with this, there are cases when someone adds soft-spaces, or just use excessive whitespace like:

>>> print    "hello"
  File "<stdin>", line 1
    print    "hello"
                   ^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print("hello")?

This was the first patch where I had to deal with C code. It was more of reading and understanding what the current code does. I had to do a complete re-write of this patch two times in a span of just 1.5 days (Thanks to Nick and Serhiy for the quick reviews).

Here's why:

The first time I wrote the patch, I used raw C APIs, which means I used malloc and free for memory allocation and deallocation for manipulating the string input that was received from the user.

While this might work, but there is already a lot of work done by Python core-devs & they've created an entire C API layer. So, in the first review, I was pointed to use this API layer, so I had to re-write the patch. One of the learning I had is that virtually any API can fail. So, it is necessary to always check the results before using them.

Most of the methods from the C-API layer that were used in this patch were dealing with Unicode objects.

Another important learning was to deallocate the previous memory at each check point (if some of the API call fails and there is no memory allocated on heap then deallocate all the previous allocated memory).

Apart from handling the striping of characters that were input on the shell and detecting the tuple syntax, I also wrote test cases for the new feature. In CPython, most of the test cases are written in Python (except a few). Typically in this use case, we only need to check the functionality of the Python layer, so test cases were written in Python.

Let's see how the added feature does what it is intended to do:

// Static helper for setting legacy print error message
static int
_set_legacy_print_statement_msg(PySyntaxErrorObject *self, Py_ssize_t start)
{
    PyObject *strip_sep_obj = PyUnicode_FromString(" \t\r\n");
    if (strip_sep_obj == NULL)
        return -1;

    // PRINT_OFFSET is to remove `print ` word from the data.
    const int PRINT_OFFSET = 6;
    Py_ssize_t text_len = PyUnicode_GET_LENGTH(self->text);
    PyObject *data = PyUnicode_Substring(self->text, PRINT_OFFSET, text_len);

    if (data == NULL) {
        Py_DECREF(strip_sep_obj);
        return -1;
    }
    PyObject *new_data = _PyUnicode_XStrip(data, 2, strip_sep_obj);
    Py_DECREF(data);
    Py_DECREF(strip_sep_obj);

    if (new_data == NULL) {
        return -1;
    }
    // gets the modified text_len after stripping `print `
    text_len = PyUnicode_GET_LENGTH(new_data);
    const char *maybe_end_arg = "";
    if (text_len > 0 && PyUnicode_READ_CHAR(new_data, text_len-1) == ',') {
        maybe_end_arg = " end=\" \"";
    }
    PyObject *error_msg = PyUnicode_FromFormat(
        "Missing parentheses in call to 'print'. Did you mean print(%U%s)?",
        new_data, maybe_end_arg
    );
    Py_DECREF(new_data);
    if (error_msg == NULL)
        return -1;

    Py_XSETREF(self->msg, error_msg);
    return 1;
}

This function returns the following:

  • 1 if the error message is successfully set.
  • -1 in case any of the C-API fails.

So, first we use strip_sep_obj to define all the symbols that we need to strip from the input received from the user. Next, when we get the string, it would be of the form print "Hello".

PyObject *strip_sep_obj = PyUnicode_FromString(" \t\r\n");
if (strip_sep_obj == NULL)
    return -1;

// PRINT_OFFSET is to remove `print ` word from the data.
const int PRINT_OFFSET = 6;

What we're interested in is about the text after the print and space. That accounts for the PRINT_OFFSET that is set to constant 6.

Next, _PyUnicode_XStrip uses to strip strip_sep_obj from the data we assigned earlier. The 2 in this function is a constant value that implies stripping the input from both sides of the data.

PyObject *new_data = _PyUnicode_XStrip(data, 2, strip_sep_obj);

Next, we get the length of the text and check if the input meant to print a tuple by checking the last character from the Unicode buffer to be equal to string literal ,.

const char *maybe_end_arg = "";
if (text_len > 0 && PyUnicode_READ_CHAR(new_data, text_len-1) == ',') {
    maybe_end_arg = " end=\" \"";
}

After setting up the maybe_end_arg we format the new message and use Py_XSETREF to set the new error_msg to the PySyntaxErrorObject instance.

Py_XSETREF(self->msg, error_msg);

Here is the full patch if you want to see. The meaning of each of the methods used can be easily found from the official documentation.

I hope to write about my learning in more detail and also explain about different parts of the programming language in a future post.

by Sanyam Khurana at June 20, 2017 03:04 PM

June 17, 2017

Dhriti Shikhar

June ’17 Golang Bangalore Meetup

The June Golang Bangalore Meetup was conducted on Saturday, June 17th, 2017. There were around 35-40 people who attended the meetup.

DCf3kQJVwAA8VSC.jpg:large

The meetup started at 10:15 with the first talk by Nurali Virani who works at SAP Labs, Bengaluru. He talked about “Understanding Type System In Go”.

DCgErtoU0AAVDds.jpg:large

Saifi gave an awesome talk about “Working with C code and plugins in Go”. The slides related to his talk can be found here

DCgNU5GU0AAFIhH.jpg:large

Umasankar Mukkara is the Co-Founder and CEO at CloudByte Inc. He talked about OpenEBS and their experience with Golang. The slides of his talk could be found here. OpenEBS is an open source project written in Golang. You can find the source code of OpenEBS here.

Satyam Zode who is also a fellow Golang programmer at OpenEBS presented a talk about “Package oriented design in Go”. The slides related to his talk could be found here

IMG_20170617_115317IMG_20170617_115325

I thank Uma, OpenEBS and Nexus Ventures Partners for helping us to organize the June Golang Bangalore Meetup by providing venue and food. There were also a few goodies for the participants by OpenEBS. Fortuanately, this event was recorded and we will be sharing the recorded videos soon when they will be ready.

 

 


by Dhriti Shikhar at June 17, 2017 01:01 PM

June 14, 2017

Sayan Chowdhury

PyLadies Pune Meetup - June 2017

Back after a long time. It’s been a long time since I blogged. This was mostly because I was busy with different conferences. Yes,I attended three conferences PyCon Pune, FOSSASIA, and PyCon US since the last time I blogged. I would be back writing reports on all the three events and I have a pending third and the final blog post to be done on fedimg.

Now, let me rollback to the topic for which this blog post is meant to be.

11th June 2017, PyLadies Pune organized the meetup for the month of June. The meetup started a bit late with Shilpee Chamoli taking a 101 class on Data Science. The workshop focused on installing dependencies (Jupyter, Pandas, etc), and working on a dataset on ‘survival on the Titanic’. Since few people were having trouble with installing the dependencies, Kushal and a few other people used Azure Notebooks.

Shilpee Chamoli (Picture by: Kushal Das)

Anwesha and I had recorded videos during PyCon US which had messages from Lynn Root, Ewa Jodlowska, Carol Willing, and Jackie Kazil for the PyLadies Pune group members encouraging them on progressively working ahead and helping the community grow.

Jackie Kazil

Carol Willing

Ewa Jodlowska

Lynn Root

After lunch, a mini-dev-sprint was planned where the participants would come up with a project idea. Chandan, Praveen, Kushal and me were doing the mentoring. I was mentoring a group who are planning to build a recommendation system for restaurants (as of now). Based on my experience, I shared with them on which are the important things to focus first and follow the principle “release early and release often”. On the sidelines, I was also working on my project “invento”

The meetup ended at around 5 PM in the evening, though few people stayed up late working on the project.

June 14, 2017 05:00 PM