Planet dgplug

May 28, 2016

Kushal Das

Day -1 of PyCon US 2016

I reached Portland two days back, was happy to see that Remy helped us to find a hotel just opposite to the conference center. As I am typing this, I can see the empty roads, and the main doors of the venue. Yesterday also I woke up by 5AM, the virtue of being in a place 12:30 hours apart from your standard timezone :) After writing the article about Microbit support in Fedora (it is still under review) I moved to the conference venue. It was very easy to find the staff room. As usual Doug,Lvh,Eric were already there, later I met Ewa, and then met Betsy for the first time. While discussing security practices when I asked, Lvh pointed out that getting golang vendored sources in the source code repository and then not updating them later, is more of a software engineering problem than a security problem as he sees.

Later in the day few of us walked towards downtown to get some lunch, and then we decided to visit Powell’s again. Mike Pirnat gave some valuable insights about children books, and card games :) After coming back to the venue, we saw more people are coming in. Rami was the first person to meet, PyCon is really a great place to meet old friends. Later I met Glyph, Hynek, Guido, Harry, Mike Muller, Graham Dumpleton, and Yarko (I don’t think I will be able to write down people’s names like this from tomorrow, the list will be too long :D ). I also found PyDanny, and Audrey coming in for the registration. It was good see her walking around after the big surgery.

Guido pointed out the xkcd comics about the Python patch. The discussion then moved to regex, legal side of the software field. I also had a chat about how CentOS is helping CPython with CI effort. It turns out that Guido knew about CentOS, but never looked deep into what is that :) He was seems be happy about help CentOS is doing by providing the beefy boxes to do build+test super fast. I am actually going to talk about that today in the Language Summit.

Later at night a large group of us moved to a nearby Ethiopian place for dinner. The food was great, we had more people coming in for the next few hours. Ntoll also came in and joined our table. Came back to the hotel room around 22:30, and tried to sleep early so that I can wake up in time. Today I will be in the registration desk from 7AM just like other years, and then I will move into the Language Summit. See you all in PyCon :)

by Kushal Das at May 28, 2016 12:46 PM

May 23, 2016

Kushal Das

dgplug summer training student Tosin Damilare James Animashaun

Your name (blog/twitter) and what do you do

My name is Tosin Damilare James Animashaun (IRC: acetakwas; Twitter: @acetakwas; Github: github.com/takwas, Blog: takwas.github.io).

I am currently a part-time Software Engineering student at NIIT, Lagos. I am also a co-founder at a startup called Krohx. We are in the final stages of deploying a job/hiring web platform that targets seekers of short-term jobs while also easing the process of getting service delivery for the hirers.

With my involvements with the burgeoning Python community in Nigeria, I recently got elected as secretary for the organization.

How did you learn about the training? (++ My Experience) I fail to recall

exactly how I learnt about the community now, but I remember joining the mailing list and going on from there. This was in fact my first time of joining a mailing list. Upon learning about the #dgplug IRC channel, I promptly enlisted the channel to my IRC client’s favourites.

I was so enthusiastic about this, as I had no prior experience of an online tutoring session. The feeling was truly exhilarating.

Having come from a Java background, I was really interested in learning the Python programming language as soon as I caught on waves with. At the time, I had volunteered to write some scripts to help automate some of the manual processing of NIIT’s annual scholarship exam records/documents at my school for that year. I accomplished that using Java, but in hind-sight, I feel I could have saved a lot of time if I had utilised Python in the situation.

This was a blog post I wrote as part of tasks given to us. I have failed to maintain that blog though. I now write here

How this training changed (if at all) your life? Python is surely the more

beginner-friendly language and it’s making its rounds these days. Every programmer should have it in their arsenal. One of the things I love most about the language is it versatility in application.

No doubt, I have continued to improve with Python everyday, but more so in my programming journey. I have become more and more familiar with various mainstream concepts and tools. The idea of contributing to open-source can be overwhelming at first, but following the path laid out in this course, one would be surprised to learn how easy it could be.

I have volunteered to mentor attendees at a Django Girls workshop, which held in Lagos, Nigeria. I picked-up Django in a week prior to the event, because frankly, there wasn’t much of a learning curve to it since I already used Flask, a different Python-based web development framework.

Flask was a framework I first got introduced to at the summer training, but I did not revisit it until months later. As a backend web-developer, I now do most of my work in Flask.

In my experience, there is no better or faster way to grow, than to collaborate; and that is what open source preaches. Note that growth here is not only about the project, but the growth applies to each contributing individual. Heck, even if you do not contribute, you will grow by reading good open-source code – this I can attest to. And joining the dgplug community has surely made open-source a more approachable endeavour. # #

Have you contributed to any upstream project(s)? If yes, then details.

While (and a bit disappointingly) I have yet to contribute to an upstream project, I do have quite a number of them lined up. Most probably, the first would be to the [official Slack Client]() written in Python. Having recently developed a bot, which was used in conducting the election of committee members of the Python Users Nigeria Group, I do have a number of changes I plan to push to the aforementioned API library.

With this forked project, I have also tried my hands at revamping the architecture of the official dgplug bot ekanora, which is used in running the summer training sessions.

Some other projects I’d like to contribute to include:

Any tips for the next batch of participants. (++Final words)

I am not sure I am in the best position to answer this. However, I would advise that you endeavour to remain focused through this; simply keep at it. Even if your experience at the summer camp doesn’t go as you expect, you should definitely not count it loss. Instead, make it a reason to put in more effort and grow as you go.

Depending on your personal activity schedule, you might miss one/two classes. But that shouldn’t deter you from continuing the program. The logs are available for covering such lapses. Also, you could consult the summer training docs to get a hang of the agenda, so you can anticipate classes beforehand and be better prepared.

While I participated in the training, there were times when there was no power supply, and I had to use my phone. Thankfully, there are IRC clients for mobile phones (I recommend AndChat).

The class sessions have a few rules, and the community would love it if you played by them. Not to worry, they are no stringent rules; just there to keep everyone behaving. For instance, the use of SMS-speak is discouraged.

You should keep notes beside you when attending the sessions. Writing is a good way to memorize things in my opinion. Also, although most clients would have this by default, you should ensure your IRC client is configured to log conversations. Going back over what you’ve learnt would do you great good as you mightn’t be able to keep up with the speed of the class at times.

If you have some time before program begins, two things I’d advise you become familiar with are: - Your OS (Linux, preferably Fedora or Ubuntu) - An IRC client (I suggest HexChat)

I learned quite a lot from the summer training. I was really obsessed with attending whenever I could. Although coming from a different country, the timing wasn’t favourable, often meeting me in the middle of the day’s activities – school especially. I made efforts to invite a few friends to join on – one or two did, but the determination to keep on was obviously lacking, which reminds me of a statement I heard a while back that reads something like this:

"If 40 people begin a course like this together, only about 5 get to
finish."

That in my experience is very often the case. Be aware that a lot of people are on this bandwagon. The question to ask yourself, is “Do I want to be among the surviving commutants at the end of this journey?” Unless you plan to experiment with this experience, if your answer is yes, then I congratulate you as you begin the journey that could potentially kick-start your software engineering career/journey.

Know this, free and open source software is for the good of all. The open source community is wide and continues to grow. It welcomes all and sundry, but it poses a few hurdles to sieve the wheat from the chaff. In the end, the gains are worth it.

by Kushal Das at May 23, 2016 05:45 PM

May 22, 2016

Farhaan Bukhsh

Step 2: Configuring Jenkins

Jenkins is one of the major part in setting up Poor Man’s CI. Lets look into how Jenkins can be configured and how can we make it automate task.

Jenkins can be downloaded for the OS you are using from their website. After downloading the mistake that I did was using Jenkins with Global Credentials, as pointed by lsedlar on the channel, because of this I was not able to get the “Trigger by URL” option in the project.

Initial configuration is pointed by lsedlar in his blog. I will be covering extra configuration to have it working for local development. First and foremost being the authentication , this can be done by  Manage Jenkins –> Configure Global Security. Selection_013

 

Give Read, View and Discover to anonymous and add another user and give all the permission to that user. You need to restart Jenkins service.

sudo systemctl restart jenkins.service

On web ui jenkins will ask you to sign in ,  create a user with the username you gave all the permission and log in with the user. Now add New Item  and create a Freestyle Project. Now configure the project , click on  “This build is parameterized” and configure it according to Poor Man CI’s. Once that is done, select the option as shown below:

Selection_014

Once that is done you can use this token to trigger build using a POST request. The trick is you need to pass parameters too with the URL. Next thing is you need to tell Jenkins what to do and where to do. Since we are dealing with Jenkins and git we need a local repo or some URL to the git repo. For every operation carrying out in the repository the directory should have the group and user set to jenkins else you cat just put the repo in /var/lib/jenkins.

Download and install Git Plugin for Jenkins. Once that is done you need to point the git plugin to the repository you are going to test.

Selection_015

Once jenkins know where to perform action you need to tell what to perform this is done in the build section of the configuration and select Execute Shell.

if [ -n "$REPO" -a -n "$BRANCH" ]; then
git remote rm proposed || true
git remote add proposed "$REPO"
git fetch proposed
git checkout origin/master
git config --global user.email "you@example.com"
git config --global user.name "Your Name"
git merge --no-ff "proposed/$BRANCH" -m "Merge PR"
fi

We are almost done, the last thing is we need an auth token for the user. Go to Manage Jenkins –> Manager User. Now get the api token for the user. Make sure that branch you are passing as parameter exists in the repository. Lets trigger the build using cuRL.

USER:fhackdroid

API Token: 728507950f65eec1d77bdc9c2b09e14b

Token: BEEFCAFE

BRANCH:checking

curl -X POST http://fhackdroid:728507950f65eec1d77bdc9c2b09e14b@localhost:8080/job/pagureExp/buildWithParameters\?token\=BEEFCAFE\&REPO\=file:///$\{JENKINS_HOME\}/new_one_three\&BRANCH\=checking\&cause\=200


by fardroid23 at May 22, 2016 07:20 AM

May 17, 2016

Farhaan Bukhsh

Part 1: Setting Up Fedmsg

I was trying to make Poor man’s CI work for local development. Since it has been written in a way that it has been put to productions directly. So I started to talk about it in fedora-admin channel, I got a lot of input from pingou and lsedlar.

Finally, I tried to divide the task into various parts the first one was making fedmsg work on my system locally. Fedmsg is way in which application talk to each other in Fedora infrastructure. A message contains a lot of information about the changes in the application.

Well setting up was a big problem for me because the version of fedmsg being installed was throwing a segment fault whenever it tried running fedmsg-tail. I pinged Sayan regarding this issue, we worked a lot over it finally the solution we found was using virtualenv and installing a specific stable version of fedmsg which was :

        pip install fedmsg==0.16.4

Now, since fedmsg now got properly set up , fedmsg has to take messages from my local Pagure, fedmsg-relay is a service which binds two ports one where the message is being emitted and the other where it has to be listen to.

If a project doesn’t have a fedmsg.d/<somefile>.py  then fedmsg will take inputs from /etc/fedmsg.d/endpoint.py , in my case fedmsg-relay was using the latter location so I modified that file a bit to get message from my local instance.

In a different terminal we can run fedmsg-tail  –really-pretty to see the messages. Any change being done to any repo in local Pagure instance is now being logged.

Selection_010fedmsg

With these changes I was able to configure in such a way that I can see my local instance emitting messages on fedmsg.

Tip: Using virtualenv wrapper its an amazing tool to manage various virtualenv. And most of the projects comes with a convention that setup.py has install and develop option.

 


by fardroid23 at May 17, 2016 04:31 PM

May 09, 2016

Shakthi Kannan

Dired Mode

[Published in Open Source For You (OSFY) magazine, August 2015 edition.]

Introduction

Dired is a directory editor in GNU Emacs. It opens a buffer containing a list of directories and files to operate on. It is a ‘read-only’ mode and hence you cannot input any text. This article explores some of the basic commands that can be used in Dired mode.

Let us first create a sample directory, sub-directories and files that we can use for our demonstration. Open a terminal and execute the following commands:

$ mkdir /tmp/test/network -p
$ mkdir /tmp/test/kernel -p

$ date > /tmp/test/date.txt
$ locale > /tmp/test/locale.txt

$ cat /etc/resolv.conf > /tmp/test/network/resolv.conf
$ ifconfig > /tmp/test/network/ifconfig.output

$ dmesg > /tmp/test/kernel/dmesg.txt

In general, filename extensions don’t have any meaning on *nix systems. Some applications do check the filename extension before using them. The extension is only for the benefit of the user, and hence you can have a filename without any extension on *nix.

Invocation

After opening GNU Emacs, you can enter Dired mode using M-x dired. It will prompt you for the directory to be opened in the buffer where you can input ‘/tmp/test’. You will see a buffer with the following contents:

/tmp/test:
total used in directory 24 available 146721468
drwxrwxr-x  4 shakthi shakthi 4096 Jul  3 11:36 .
drwxrwxrwt 12 root    root    4096 Jul  3 11:38 ..
-rw-rw-r--  1 shakthi shakthi   29 Jul  3 11:36 date.txt
drwxrwxr-x  2 shakthi shakthi 4096 Jul  3 11:36 kernel
-rw-rw-r--  1 shakthi shakthi  270 Jul  3 11:36 locale.txt
drwxrwxr-x  2 shakthi shakthi 4096 Jul  3 11:36 network

The contents of the buffer are similar to the ls -al output as observed in the terminal. The files were created by user ‘shakthi’, and this is indicated in the owner and group fields. The ’.’ entry represents the current ‘/tmp/test’ directory. The ’..’ listing represents the parent directory, which is ‘/tmp’ and owned by the ‘root’ user.

Exit

You can exit from Dired mode by simply pressing q (quit-window) in the Dired buffer.

Navigation

You can move to a previous line or the next line in the Dired buffer using the p and n keys, respectively. If you wish to move the cursor to the previous and next directories, you can use the ‘<’ and ‘>’ keys. If the cursor is at ‘kernel’ directory as shown below …

/tmp/test:
total used in directory 24 available 146721468
drwxrwxr-x  4 shakthi shakthi 4096 Jul  3 11:36 .
drwxrwxrwt 12 root    root    4096 Jul  3 11:38 ..
-rw-rw-r--  1 shakthi shakthi   29 Jul  3 11:36 date.txt
drwxrwxr-x  2 shakthi shakthi 4096 Jul  3 11:36 kernel      <-- CURSOR
-rw-rw-r--  1 shakthi shakthi  270 Jul  3 11:36 locale.txt
drwxrwxr-x  2 shakthi shakthi 4096 Jul  3 11:36 network

… then, when you press ‘>’, the cursor will move to ‘network’, which is the next directory in the buffer.

/tmp/test:
total used in directory 24 available 146721468
drwxrwxr-x  4 shakthi shakthi 4096 Jul  3 11:36 .
drwxrwxrwt 12 root    root    4096 Jul  3 11:38 ..
-rw-rw-r--  1 shakthi shakthi   29 Jul  3 11:36 date.txt
drwxrwxr-x  2 shakthi shakthi 4096 Jul  3 11:36 kernel      
-rw-rw-r--  1 shakthi shakthi  270 Jul  3 11:36 locale.txt
drwxrwxr-x  2 shakthi shakthi 4096 Jul  3 11:36 network     <-- CURSOR

You can move to the parent directory using the ‘^’ key. To enter into a directory, move to its listing in the Dired buffer, and simply hit the return key.

Viewing files

To view a file, you can place the cursor on its entry and use the f or v key, or simply hit the return key. This will open the file in a new buffer (view). To return to the Dired buffer, you can type C-x b and the minibuffer will prompt you with the message “Switch to buffer (…)”. If you then press TAB, it will open a new Completions buffer that will list the available buffers. You can also type the entire ‘test’ buffer name, or type it partially and hit TAB for auto-completion, and hit the return key to get to the ‘test’ buffer. If you wish to close a buffer, you can use C-k to kill it. This will only close the buffer, but, the file will still exist! You can use the ‘+’ key to create a new sub-directory.

Marking and unmarking files

Dired mode allows you to operate on multiple files and directories. In order to run commands on them, you need to first select the files. The m key can be used to mark a file or directory for subsequent operations. For example, pressing ’m’ on the date.txt entry will mark it, and this is indicated by an asterisk in front of the line, as shown below:

  /tmp/test:
  total used in directory 24 available 146933380
  drwxrwxr-x  4 shakthi shakthi 4096 Jul  5 09:53 .
  drwxrwxrwt 10 root    root    4096 Jul  5 09:46 ..
* -rw-rw-r--  1 shakthi shakthi   29 Jul  5 09:46 date.txt
  drwxrwxr-x  2 shakthi shakthi 4096 Jul  5 09:46 kernel
  -rw-rw-r--  1 shakthi shakthi  270 Jul  5 09:46 locale.txt
  drwxrwxr-x  2 shakthi shakthi 4096 Jul  5 09:46 network

You can unmark the above selection using the u key. This is applicable for both files and directories. To undo a selection, you can also use M-del (M and the Delete key). Suppose, you wish to mark all the directories in the Dired buffer, you can use ‘* /’ key combination. The resultant Dired buffer is shown below:

  /tmp/test:
  total used in directory 24 available 146933432
  drwxrwxr-x  4 shakthi shakthi 4096 Jul  5 09:53 .
  drwxrwxrwt 10 root    root    4096 Jul  5 09:46 ..
  -rw-rw-r--  1 shakthi shakthi   29 Jul  5 09:46 date.txt
* drwxrwxr-x  2 shakthi shakthi 4096 Jul  5 09:46 kernel
  -rw-rw-r--  1 shakthi shakthi  270 Jul  5 09:46 locale.txt
* drwxrwxr-x  2 shakthi shakthi 4096 Jul  5 09:46 network

If you want to invert the selection, use t. The buffer will look like the following:

  /tmp/test:
  total used in directory 24 available 146933432
  drwxrwxr-x  4 shakthi shakthi 4096 Jul  5 09:53 .
  drwxrwxrwt 10 root    root    4096 Jul  5 09:46 ..
* -rw-rw-r--  1 shakthi shakthi   29 Jul  5 09:46 date.txt
  drwxrwxr-x  2 shakthi shakthi 4096 Jul  5 09:46 kernel
* -rw-rw-r--  1 shakthi shakthi  270 Jul  5 09:46 locale.txt
  drwxrwxr-x  2 shakthi shakthi 4096 Jul  5 09:46 network

To mark all the files, you can use ‘* s’ key combination.

Modifying the buffer

After you have selected files, you can remove them from the listing by pressing the k key. This does not delete the files! You can always press g to refresh the Dired buffer contents. You can change the ‘ls’ command line options used in the Dired listing using the C-u s key combination.

Actions

You can perform a number of operations on the marked files and directories. To copy a file, you can press C on a file, and it will prompt you in the minibuffer regarding where you would like to create the new copy. You can use R to rename a file. If you would like to delete a file, you can press D. You can change the mode of a file with the M command. You can compress or uncompress a file with the Z command.

A regular expression search can be done on selected files with the A command. You can create a symbolic link file with the S key. You can run a shell command on marked files using the ’!’ command. For example, if you want to perform a word count (wc) on the date.txt file, you can press ’!’ on the date.txt entry listing, and it will prompt you in the minibuffer with “! on date.txt:” message. If you input ‘wc’, the results will again be shown in the minibuffer. On my system, it returned ‘1 6 29 date.txt’. You can also run asynchronous shell commands on files using the ‘&’ command, and a new buffer will be opened that will contain the results.

Deletion

You can mark files for deletion using the d command. After selecting the files or directories, if you press x, the respective actions will be performed on the files. To demonstrate an example, let us use ‘d’ on the ‘kernel’ directory to mark it for deletion. The letter ‘D’ will be added to its entry as shown below:

  /tmp/test:
  total used in directory 24 available 146933432
  drwxrwxr-x  4 shakthi shakthi 4096 Jul  5 09:53 .
  drwxrwxrwt 10 root    root    4096 Jul  5 09:46 ..
  -rw-rw-r--  1 shakthi shakthi   29 Jul  5 09:46 date.txt
D drwxrwxr-x  2 shakthi shakthi 4096 Jul  5 09:46 kernel
  -rw-rw-r--  1 shakthi shakthi  270 Jul  5 09:46 locale.txt
  drwxrwxr-x  2 shakthi shakthi 4096 Jul  5 09:46 network

When you press ‘x’, you will be prompted to confirm deletion with the message “Delete kernel (yes or no)”. When you input “yes” and hit the return key, the directory contents will be removed. GNU Emacs will perform auto-save on files, and also create backup files ending with ‘~’. You can mark such files using the ‘~’ command.

Regex

You can use ‘% d’ to select files based on a regular expression (regex) to mark for deletion. To simply mark files based on regex, you can use ‘% m’ shortcut. To rename the marked files, you can use ‘% R’.

Find

You can search for files matching a pattern using the M-x find-name-dired command. Suppose you wish to find all the .txt files in our original /tmp/test directory, you can use ‘M-x find-name-dired’ in the Dired buffer. It will prompt you in the minibuffer with the following message “Find-name (directory): /tmp/test”. After you press the return key, it will ask for the regex with the message “Find-name (filename wildcard):”. If you input ‘*.txt’, it will return the following contents:

/tmp/test/:
find . \( -name \*.txt \) -ls
6826601    4 -rw-rw-r--   1 shakthi  shakthi        29 Jul  5 09:46 date.txt
6826616   64 -rw-rw-r--   1 shakthi  shakthi     63050 Jul  5 09:46 kernel/dmesg.txt
6826602    4 -rw-rw-r--   1 shakthi  shakthi       270 Jul  5 09:46 locale.txt

find finished at Sun Jul  5 10:27:02

You can also find files based on regex patterns that exist in the file contents using the M-x find-grep-dired. Suppose, you wish to find the files that have the text ‘eth’ in them, you can use ‘M-x find-grep-dired’, which will prompt you in the minibuffer with the message “Find-grep (directory): /tmp/test”. After you press the return key, it will prompt you for the regex with the message “Find-grep (grep regexp):”, where you can input ‘eth’ and this will return the following results:

/tmp/test/:
find . \( -type f -exec grep -q -e eth \{\} \; \) -ls
6826605    4 -rw-rw-r--   1 shakthi  shakthi      1620 Jul  5 09:46 network/ifconfig.output
6826616   64 -rw-rw-r--   1 shakthi  shakthi     63050 Jul  5 09:46 kernel/dmesg.txt

find finished at Sun Jul  5 10:31:27

You can also execute a find operation on marked files using M-x find-dired.

Help

To open the help menu for Dired, you can use the h shortcut key. To see the Dired log messages, you can press the ? key.

You may refer to a quick reference card at https://www.gnu.org/software/emacs/refcards/pdf/dired-ref.pdf, and try out more Dired commands.

May 09, 2016 12:45 PM

May 02, 2016

Dhriti Shikhar

Machine Learning

Learning

To gain knowledge or understanding or skill through:

  • study
  • instruction
  • experience

 

Machine Learning

The field of study that gives computers the ability to learn without the need of explicitly programming.

The goal is to device programs that learn and improve performance with experience without human intervention.

 

Training Data

  1. Set of examples (input -> output) for learning
  2. Used to build model

 

Test data

  1. Used to test:
    • how good your model can predict
    • estimate model properties
  2. It is always outside training data set but follows some probability distribution as training data

 

Feature

  1. also called predictor
  2. It is a meaningful attribute
  3. Internal representation of data
  4. quantity describing an instance
  5. property of an instance

 

Tuple

  1. A Record in data base
Screenshot from 2016-05-02 23-34-01Features are columns and Tuples are rows

 

If we increase the number of records, attributes in a data set, then Machine Learning problem also becomes a Big Data problem.

 

Supervised learning

  1. It uses training data set consisting of input -> correct output to train the model
  2. Example:
    • Page Ranking Algorithm
    • Next word recommendation in Instant Messaging Application/ Whatsapp/ SMS

 

Unsupervised learning

  1. No training data set exists
  2. Most difficult algorithms are unsupervised learning because there is no “fixed” objective.
  3. used in Explaratory Data Analysis (EDA)
  4. Example:
    • used in recommendation systems to determine users who are similar to me from existing database

 

tiff infomationMachine Learning types

 

 Classification  Clustering
We have a set of pre-defined classes and we want to know which class a new object belongs to. Group a set of objects and find whether there is some relationship between objects.
It is predictive modelling. We give          pre-defined groups and predict group of new data. It is descriptive modelling. We try to find groups which occur naturally in data .

 

Classification

There are 6 items categorised in 2 classes:

tiff infomationExample of Classification

 

Each category has a label e.g. Eatables and Non-Eatables. If we have to predict the class of a new item “strawberry”, then it will be assigned a label “Eatable”

 

Clustering

There are 6 items categorised in 2 groups:

tiff infomationExample of Clustering

 

Each category is unnamed i.e. there is no label attached to the group. If we have to predict the group of a new item “strawberry” then it will be in the first group.

 

Accuracy

  1. How often is the prediction correct?
  2. Accuracy is not reliable metric for real performance of model because it will yield misleading results if training data set is unbalanced (i.e.  number of samples in different classes vary greatly).
  3. Example:
    1. Let number of cats be 95 and number of dogs be 5
    2. Classifier can easily bias into classifying all samples as cats
    3. Overall acuuracy = 95%
    4. BUT 100% recognition rate for cats and 0% recognition rate for dogs
  4. One of the ways to improve accuracy is to provide more balanced data.

 

This is one of the interesting things explained by Satish Patil in Pune Python Meetup that:

There is no right or wrong model. There is no best or worst model. There is ONLY useful and non-useful model. 

Nobody knows how much percentage of accuracy is good. How much accuracy is needed depends on Business Context.

Consider a company which wants to launch a new product and they want the probability of success of the product using Machine Learning. So, it is the company which DECIDES that if they get probability below 60%, then they will not launch the product. So, this is not something that the developer decides. This totally depends on the business context.

 

Market Basket Analysis

  1. Also called affinity analysis
  2. Association Rule:
    • discovering interesting relation/connection/association between specific objects
  3. Sometimes, certain products are typically purchased together like:
    • beer and chips
    • beer and diapers
    • bread and eggs
    • shampoo and conditioner
  4. So, market basket analysis tells a retailer that promotion involving just one of the items from the set would likely drive sales of the other
  5. This technique is used by retailers to:
    • improve product placement
    • marketing
    • new product development
    • making discount plans

 

Titanic Data Set

The titanic data set was used in the machine learning talk in Pune Python Meetup. It can be downloaded here.

There are some features in the data set which can be ignored as they are not important like:

  • Passenger ID
  • Name
  • Ticket Number
  • Cabin

and there are some important features which help in classifying like:

  • Survived
  • Gender

 

Impurity Measure

  1. Measures how well are the classes separated
  2. Should be 0 when all data belong to one class

 

Entropy

  1. Entropy can be a measure of quality of model
  2. It is a measure of how distributed are the probabilities.
  3. The more equal is the share for the probability values in all the classes, the higher is the entropy.  The more skewed is the share among the classes, lesser is the entropy.
  4. The goal in machine learning is to get a very low entropy in order to make the most accurate decisions and classifications

 

Decision Tree

  1. A way of graphically representing an sequential decision process
  2. Non-leaf nodes are labelled with attribute/ question
  3. Leaf nodes are labelled with class
tiff infomationdecision tree based on titanic data set

 

Pruning

  1. Data can contain noise:
    • instance can contain error
    • wrong classification
    • wrong attribute value
  2. If a particular feature is not used by a tuple or if the feature is not influencing, then it is removed.

 

Data Preprocessing

  1. Converting data into interval form
  2. Machine learning algorithms learn from data so its important to feed it the right data
  3. Data preprocessing basically involves:
    • correcting mistakes
    • handle missing values
    • handle outliers
    • normalize values
    • nominal values

 

Missing Value

  1. The value of an attribute which is not known or does not exist
  2. Example:
    • value was not measured
    • instrument malfunction
    • attribute does not apply
  3. If a column contains “Not Available”, then it is NOT considered as a missing value.

 

Outliers

  1. samples which are far away from other samples
  2. They can be mistake/ noise or represent a special behaviour
  3. Outliers are generally removed

 

Questions that were asked in meetup

  1. Can data be extended to multiple dimension?
  2. Can distance be other than Euclidian?
    • Yes, Manhattan distance
  3. Are there online courses that teach ML intro?
    • Yes
  4. What is “k” in k-means?
    • k is no. of clusters
  5. Can we use ML for trading?
    • Yes
  6. Any daily life clustering example
  7. Any software product based on unsupervised learning?
    • Google Maps
    • Matrimony/ Dating websites
    • Red Coupon (real estate)
    • Amazon recommendation
    • Netflix
  8. Order in which features is given, is that important?
    • No
  9. Why do we say that one model is better than the other?
  10. What if accuracy is not the concern?
    • Accuracy is one way of looking at prediction
  11. Do you think that if model changes, something in feature has changed?
  12. We have tools like WEKA, so why would anyone prefer Python or R?
    • depends on the language available or language the company uses
  13. How do we know that a particular feature is important or not?
  14. What if some features are more influential than others? How will the decision tree be affected?
  15. How to handle outliers in a decision tree?
  16. Will the algorithm figure out the relationship between input and output?
    • This is possible through Regression

 


by Dhriti Shikhar at May 02, 2016 06:57 PM

April 30, 2016

Dhriti Shikhar

Event Report: April Pune Python Meetup

April Pune Python Meetup (@PythonPune) was conducted on April 30, 2016 at Redhat, Pune. Around 70 people registered for the meetup but the turnout was around 72-73. A few people registered on the spot.

Python Pune Meetups are organised by Chandan Kumar (@ciypro) who is a fellow RedHat employee, a python programmer and FOSS enthusiast who has contributed to many upstream projects.

The meetup started around 10:45 with the introduction where everybody introduced themselves. Almost everybody knew python. There were 1-2 people who did not know python. There were a few people who were experience in machine learning and some who were completely new to Machine Learning. I had a course on machine learning in my college where i learnt the theory and did some practical assignments in R language. The crowd was diverse consisting of students, data scientists, professors and people of various age groups 18 – 70.

This speakers of this meetup were Satish Patil (@DataGeekSatish) and Sudarshan Gadhave (@sudarshan1989) who took a session on Introduction to Machine Learning. 

4Satish Patil in Pune Python Meetup

 

5Sudarshan Gadhave in Pune Python Meetup

Satish Patil is the Founder and Chief Data Scientist of Lemoxo Technologies, Pune where he advises companies large and small on their data strategy. He has 10+ years of research experience in the field of drug discovery and development. He told a few real life machine learning examples from his field in the meetup!

Satish is passionate about applying technology, artificial intelligence, design thinking and cognitive science to better understand, predict and improve business functions. He has a great interest in Machine Learning, Artificial Intelligence, Data Visualisation, Big Data.

Satish covered the following topics:

  • What is Machine Learning
  • The Black Box of Machine Learning
  • features
  • training and test data set
  • classification
  • clustering
  • pure and impure states
  • entropy
  • decision tree
  • supervised and unsupervised learning
  • market basket analysis
  • data pre-processing
  • Titanic data set
  • K means algorithm

Although Machine Learning is a vast concept and it definetly requires more sessions to grasp, but Satish made a remarkable effort in making us understand all the above topics in layman terms.

There are a lot of books, courses, material available online for Machine Learning, so why this meetup? Well, the best part about this meetup was the way Satish explained the BUSINESS CONTEXT of MACHINE LEARNINGThis was something new for me to learn. Getting to know the real life examples from the entrepreneur-cum-data scientist was really interesting.

1.jpgThe Machine Learning Workshop in Pune Python Meetup

The details of his talk will be in my next blog.

Chandan Kumar talked about Fedora Labs. The Fedora science spin comes pre-installed with essential tools for scientific and numerical work like IDE, tools and libraries for programming in Python, C, C++, Java and R. It basically eliminates the need to download a bunch a scientific packages which you need.

If you need any help regarding the spin, you can get help from #fedora-science channel on Freenode on IRC.

As Chandan Kumar ALWAYS encourages us to contribute to open source, he introduced us to WHAT CAN I DO FOR FEDORA?. Pune Python meetups and Devsprint are a great platform to seek for help if you want to contribute to opensource.

3Chandan Kumar in Pune Python Meetup

 

Thanks to Satish Patil and Sudarshan Gadhave for conducting an awesome workshop! We hope to see more such workshops by you in the meetups.

Thanks to RedHat for the food, beverages and venue.

Thanks to Chandan Kumar, Pravin Kumar (@kumar_pravin), Amol Kahat, Sudhir Verma for organising such interesting meetups where we always learn something new:)

 

 


by Dhriti Shikhar at April 30, 2016 07:44 PM

April 27, 2016

Trishna Guha

Publish Docker image with Automated build

docker1

Docker is an open source tool that helps us to develop, ship and run applications.  Each and every application needs a Dockerfile. Instructions are written over Dockerfile in a layered/step-by-step structure to run the application. The Dockerfile is then built to create Docker image. We can then push the image either to Docker Hub or keep it in our Workstation. Finally to run the application we have to run container from the image which can be treated as a concept like an instance of the image since we can have more than one container of a single image. If the Docker image is not in our local Workstation we first have to pull the image from Docker hub and then run container from it. But when we need to make changes in Dockerfile(as needed to make changes in application) we have to build a new Docker image again from it. Or we can make change in the container we are running and commit it but that doesn’t reflect the Dockerfile which contains all the instructions to run an application say it for development or production purpose. Also building Docker image locally from Dockerfile makes us doing things like build, push and all manually. Automating build of Docker image from Dockerfile that is either on Github or Bitbucket is the only solution for this:) .

In Short:  We first create Dockerfile and then push it to Github or Bitbucket. After authenticating Github account with Dockerhub we choose repository on Dockerhub from Github that contains the Dockerfile. After that the Dockerfile triggers build which in result creates Docker image getting ready to pull.

I will share an example of making a CentOS image having Apache httpd server pre-installed.

First we need to create a Dockerfile which can be viewed here.

FROM centos:centos6
MAINTAINER trishnag <trishnaguha17@gmail.com>
RUN yum -y update; yum clean all
RUN yum -y install httpd
RUN echo "This is our new Apache Test Site" >> /var/www/html/index.html
EXPOSE 80
RUN echo "/sbin/service httpd start" >> /root/.bashrc 

Then push the Dockerfile to Github. I have created a repository named CentOS6-Apache and pushed the Dockerfile to it. The repository can be found here.

After doing so

  • Go to DockerHub and Settings —> Linked Account —> Link your Github account.
  • Create —> Create Automated Build —> Create Auto-build Github.
  • Select the repository that contains Dockerfile of your application.
  • Create followed by the Automating build process.

After the image is built successfully you will be able to see Success in Build Details which indicates that the image is successfully built. The Image is now live on docker hub https://hub.docker.com/r/trishnag/centos6-apache.

Now we have to test the image to make sure whether Apache httpd server is actually pre-installed or not.

docker pull trishnag/centos6-apache  #Pull the image from Dockerhub
docker run -t -i -d trishnag/centos-apache /bin/bash #Run the container as daemon
docker ps #Get the name of the container

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
8bcd1199bb8f trishnag/centos6-apache "/bin/bash" 2 minutes ago Up 2 minutes 80/tcp jovial_cray

docker logs jovial_cray #To see What is happening on bash of container.Gives httpd status
docker inspect jovial_cray | grep IPAddress #Shows the IPAddress
curl 172.17.0.1 #curl the IPAddress we got
This is our new Apache Test Site

We just have got the text that we echoed out to index.html. Hence we can finally come to conclusion that Apache httpd server has already been pre-installed in the image.

Now even if we commit changes to the Dockerfile on Github we really don’t have to worry about the image. The build process of Docker image starts automatically once any changes are committed to Dockerfile. When we pull the newly built image and run container we will be able to find those changes.

Automating build process really makes our life easier.

Resources:

 


by Trishna Guha at April 27, 2016 06:53 PM

April 16, 2016

Suraj Deshmukh

All in one Kubernetes cluster on Fedora

This tutorial will walk through the installation steps of Kubernetes, which will run Kubernetes Master and Kubernetes Node on the same machine. Setup a Fedora VM and follow along

Install required packages:

sudo -i
dnf -y install --enablerepo=updates-testing kubernetes etcd iptables openssl

Create a entry for localhost, fed-master and fed-minion. These are all 127.0.0.1 because we are doing all-in-one setup. We are also adding localhost entry because sometime not having this has caused failure.

echo "127.0.0.1 localhost
127.0.0.1 fed-master
127.0.0.1 fed-node
" >> /etc/hosts

Disable and Stop iptables and firewalld

systemctl disable iptables-services firewalld
systemctl stop iptables-services firewalld

File /etc/kubernetes/apiserver should look like as follows. Make sure service-cluster-ip-range does not overlap with any of network attached to your machine

# The address on the local server to listen to.
KUBE_API_ADDRESS="--insecure-bind-address=0.0.0.0"

# Comma separated list of nodes in the etcd cluster
KUBE_ETCD_SERVERS="--etcd-servers=http://127.0.0.1:2379"

# Address range to use for services
KUBE_SERVICE_ADDRESSES="--service-cluster-ip-range=10.254.0.0/16"

# Add your own!
KUBE_API_ARGS=""

Make /var/run/kubernetes

mkdir /var/run/kubernetes
chown kube:kube /var/run/kubernetes
chmod 750 /var/run/kubernetes

Generate certificates to configure service account

mkdir -p /etc/pki/kube-apiserver
openssl genrsa -out /etc/pki/kube-apiserver/serviceaccount.key 2048
sed -i.back '/KUBE_API_ARGS=*/c\KUBE_API_ARGS="--service_account_key_file=/etc/pki/kube-apiserver/serviceaccount.key"' /etc/kubernetes/apiserver
sed -i.back '/KUBE_CONTROLLER_MANAGER_ARGS=*/c\KUBE_CONTROLLER_MANAGER_ARGS="--service_account_private_key_file=/etc/pki/kube-apiserver/serviceaccount.key"' /etc/kubernetes/controller-manager

Start all the services required for Kubernetes Master and Kubernetes Node both

for SERVICES in etcd kube-apiserver kube-controller-manager kube-scheduler kube-proxy kubelet docker; do
	systemctl restart $SERVICES
	systemctl enable $SERVICES
	systemctl status $SERVICES
done

If all the services started correctly that means that Kubernetes all-in-one cluster is set and you can use it. Here are some optional steps to verify if this cluster is setup.

Create a pod definition like this below

# cat pod.json 
{
    "apiVersion": "v1",
    "kind": "Pod",
    "metadata": {
        "labels": {
            "app": "apache-centos7"
        },
        "name": "apache-centos7"
    },
    "spec": {
        "containers": [
            {
                "image": "centos/httpd",
                "name": "apache-centos7",
                "ports": [
                    {
                        "containerPort": 80,
                        "hostPort": 80,
                        "protocol": "TCP"
                    }
                ]
            }
        ]
    }
}

Create a pod using that pod definition

# kubectl create -f ./pod.json 
pod "apache-centos7" created

See if that pod is created

# kubectl get pod
NAME             READY     STATUS    RESTARTS   AGE
apache-centos7   0/1       Pending   0          48s

Wait until the pod’s status changes from Pending to Running

# kubectl get pod
NAME             READY     STATUS    RESTARTS   AGE
apache-centos7   1/1       Running   0          5m

Get the IP address of the Pod

# kubectl describe pod apache-centos7 | grep 'IP:'
IP:		172.17.0.1

See for yourself the output

# curl 172.17.0.1

Ref:


by surajssd009005 at April 16, 2016 12:58 PM

Tosin Damilare James Animashaun

After DjangoGirls, What Next?

These are my thoughts to the "girls" who stormed Lagos for Django. But I'd recommend same to any attendee of a DjangoGirls workshop anywhere.


My recent foray into the social circle of techies has proven to me that growth is often driven by community. Hence, the first advice I'll offer is that you take DjangoGirls back to your city.

The organizer of DjangoGirls Lagos, Aisha Bello herself only attended this event for the first time about a year ago. Isn't it wonderful that she's been able to put up two of such events within this period? Even though not everyone of you might be able to replicate what she has done, I encourage the few ones among you who can, to endeavour to.

Also endeavour to join communities; create them if they don't exist. For starters, the Nigeria Python Users Group has a mailing list where you can get updates on the Python community in Nigeria as well as a Slack community where you can interact with other programmers.


Did you get the contacts of your mentors? If you didn't, head over to the DjangoGirls Lagos page and get their Twitter handles. Bug them! Well, don't annoy them, but do not hesitate to ask them questions when you're stuck. They'd be glad to help, I'm sure.

And really, the learning goes both ways. Post-workshop, one of the participants of the workshop showed me an error she had encountered, and I was glad to learn that by tinkering with it, she found her way out of the maze. But that's not all. From seeing her approach, I picked up new knowledge from it too.


This post is meant to give you a quasi-roadmap of some sort, which you don't have to follow religiously but keep close enough to consult when you're lost.

So you learnt Python alongside one of its most popular frameworks: Django; and now you can code the next Facebook (Oh, were you told you could only build blogs with that new super power you picked up?) Now, I think the world has more blogging apps than we can keep track of, don't you think so?

So get creative; I assure you you've gotten past the first and most basic hurdle to creating just about any kind of web app: a social network or an e-commerce platform or something else that's interesting. Think of how important this is in an age when everything is moving to the web. You've definitely gotten a head start!

But you'd need to be prepared to learn new stuff as you go. Don't let that put you off as those things would be equally exciting. And you don't have to know it all before you begin; you can learn on-the-job.


Speaking of what you learnt at the workshop, I'm sure you know you were merely introduced to the technologies, not taught comprehensively. In my opinion, you may need to go over the whole tutorial again; then try to build something different (like a ToDo application) from the example shown. Afterwards, for some more comprehensive Python tutorial that is hands-on, try codecademy. It'll be fun, trust me.

There are several tutorials available on web platforms Youtube, Edx, Coursera, and Udacity among others. Who knows, you could build the next e-school for Africa. Pick any of these courses, and try to follow along. You don't have to be in a hurry, although you may find yourself skipping stuff you have already learnt.

I'll advise you pick up some Javascript as you go. I assure you that a mix of skills in these two languages will empower you so well.

I also recommend you put up blog posts as you learn new stuff as if to teach same to others (teaching is a proven way to better learn anything). Medium would be a good place to start, but there are several alternatives. You should also read blogs often. Let me mention that Django Girls has a blog you might be interested in checking out and so does PyLadies. See more interesting Python blogs here


During the workshop, I told my mentees that Python is a suitable language for such a workshop for three reasons:

  1. It's easy to learn.
  2. It's platform-independent.
  3. It has a lot of application domains.

Take caution not to misinterpret reason-1 to mean "easy to master" (See answer to "Is Python easy to master?"), especially because of reason-3: "Python has a lot of application domains". Be it scientific, systems administration, machine learning/artificial intelligence, statistical data analysis, game development or scripting, web scraping, and web development. For those with IoT in mind, especially with Raspberry Pi and or Arduino, Python is readily applicable to those platforms.

So you see there's quite a long way to go, yet with what little knowledge you've gained, you're able to achieve something worthwhile.


Moving on, you can share your work with others, and work collaboratively. You were introduced to version-controlling using Git and Github, which are well worth some learning time. You can learn Git through this course. Don't worry if you don't get it all at once, I struggled through this too at first. This video by Git evangelist, Scott Chacon also deals a good deal on the subject.


How much time it'd take to become quite comfortable with all of this would depend on three things:

  1. Your interest
  2. Your consistency
  3. Your learning speed

However, I'd say 6 months should be fine. Within this time you would have learnt enough to kick-start an exciting project. In fact, some people might have gotten halfway with some project in this time.


Good luck in your programming journey!

by Tosin Damilare James Animashaun at April 16, 2016 12:55 PM

April 12, 2016

Chandan Kumar

Lessons learned while packaging OpenStack's test sub-packages

On Mar 3, 2016, I started working on python-<openstack-service>-tests subpackage for all OpenStack Projects present in RDO.

It took me about a month to complete the whole task. But in this time interval, I have made lots of mistakes and learned a lot.

Below are my findings

RDO test sub-packages naming conventions

  • For openstack-<service> package, the test sub-package name is python-<service>-tests. python-<service> will be the required package.
  • If tests are maintained in a separate upstream project like <service>-tests, openstack-<service>-tests is the package name.

Placing the python test binaries at proper place

From %files macro of python-<service> subpackage, exclude tests and <service>_tempest_tests directories and include these directories under %files macro of python-<service>-tests subpackage.

The advantage of using %exclude is to avoid shipping twice the same files and creating conflicts.

%files -n python-<service>
%exclude %{python2_sitelib}/<service>/tests
%exclude %{python2_sitelib}/<service>_tempest_tests

%files -n python-<service>-tests
%{python2_sitelib}/<service>/tests
%{python2_sitelib}/<service>_tempest_tests

Use %license macro to place LICENSE

It is our common problem that we generally include LICENSE file under %doc macro which is still correct in any sub-package but %license has advantages like handling license specifics (a lot of duplication) and allowing some optimizations.

For more information check here : http://www.rpm.org/ticket/116

Do not do rm -rf <service>.egg-info

Last night, we faced a lot of problems due to this silly mistake. If we do rm -rf <service>.egg-info in %prep section, there are chances that test-fixtures (might required by tests) present under tests and <service>_tempest_tests directories does not get copied at %{python2_sitelib}/<service>/tests and %{python2_sitelib}/<service>_tempest_tests in python-<service>-tests subpackage and also causes subfolders not to be installed by setuptools.

So, think twice before you do this step in the spec file. For more information, check the Python Eggs from Fedora Python Packaging guidelines: <https://fedoraproject.org/wiki/Packaging:Python_Eggs>

Use rpmls command to verify the list of files with in a package

Use rpmls command from rpmdevtools package to check the files and their permissions included in the packages, so that you can fix later on based on your needs.

$ rpmls *.rpm

Use correct path in <service>_tempest_plugin

We had completed the packaging of python-<service>-tests sub-packages. When these packages are consumed in puppet-openstack-integration CI, some of the OpenStack Components is unable to discover tempest tests due to wrong path mentioned in the codebase. For Example: In mistral

# https://review.openstack.org/#/c/303792/1/mistral_tempest_tests/services/base.py
main_package = 'mistral'
# it should be
main_package = 'mistral_tempest_tests'

However, we have fixed the same in Zaqar and Mistral

That's it! Now all RDO OpenStack packages contain tests sub-packages.

Thanks to Alan, Javier, Haikel and Emilien for helping in getting this huge task done on time and always inspiring me to learn more.

by Chandan Kumar at April 12, 2016 04:12 PM

April 11, 2016

Sayan Chowdhury

Receiving Weechat Notifications on mobile via Pushover

I recently switched to use Weechat as my primary IRC client from X-Chat. I did a setup of Weechat on my Digital Ocean box.

Weechat setup turned out to be super simple and out of the box. I am running Weechat within a tmux session. But the problem I was facing was due to poor internet connection (thanks to Airtel for this and this was my primary reason for switching to weechat from Xchat) I got disconnected from the DO box quite often.

So, I looked over for a solution to get IRC notfications to my mobile device and Weechat seems to already have a script for that called pushover.pl.

So, The script pushes the notifications from weechat to your device via Pushover.

Installing the script

1 /script install pushover.pl

Create an account in Pushover, verify your email and create an application. Once the application is created you find your application token, and your user key.

Configuring

1 /set plugins.var.perl.pushover.token <your application token>
2 /set plugins.var.perl.pushover.user <your user key>

Now, your weechat is set to send messages through Pushover, install the application on your mobile device and you will start receiving the irc messages whenever you are mentioned or a private message.

The script also offers other customizations like

 1 'enabled' => ['on', "Turn script on or off"]
 2 'service' => ['pushover', 'Notification service to use.']
 3 'token' => ['', 'pushover API token/key']
 4 'user' => ['', "pushover user key"]
 5 'nma_apikey' => ['', "nma API key"]
 6 'pb_apikey' => ['', "Pushbullet API key"]
 7 'pb_device_iden' => ['', "Device Iden of pushbullet device"]
 8 'sound' => ['', "Sound (empty for default)"]
 9 'priority' => ['', "priority (empty for default)"]
10 'show_highlights' => ['on', 'Notify on highlights']
11 'show_priv_msg' => ['on', 'Notify on private messages']
12 'redact_priv_msg' => ['off', 'Hide the private message text']
13 'only_if_away' => ['off', 'Notify only if away status is active']
14 'only_if_inactive' => ['off', 'Notify only if buffer is not active']
15 'blacklist' => ['', 'Comma separated blacklist for notifications']
16 'verbose' => ['1', 'Verbosity level']
17 'rate_limit' => ['0', 'Rate limit in seconds']
18 'short_name' => ['off', 'Use short buffer name in notification']

April 11, 2016 06:14 AM

April 09, 2016

Tosin Damilare James Animashaun

After DjangoGirls, What Next?

These are my thoughts to the "girls" who stormed Lagos for Django. But I'd recommend same to any attendee of a DjangoGirls workshop anywhere.


My recent foray into the social circle of techies has proven to me that growth is often driven by community. Hence, the first advice I'll offer is that you take DjangoGirls back to your city.

The organizer of DjangoGirls Lagos, Aisha Bello herself only attended this event for the first time about a year ago. Isn't it wonderful that she's been able to put up two of such events within this period? Even though not everyone of you might be able to replicate what she has done, I encourage the few ones among you who can, to endeavour to.

Also endeavour to join communities; create them if they don't exist. For starters, the Nigeria Python Users Group has a mailing list where you can get updates on the Python community in Nigeria as well as a Slack community where you can interact with other programmers.


Did you get the contacts of your mentors? If you didn't, head over to the DjangoGirls Lagos page and get their Twitter handles. Bug them! Well, don't annoy them, but do not hesitate to ask them questions when you're stuck. They'd be glad to help, I'm sure.

And really, the learning goes both ways. Post-workshop, one of the participants of the workshop showed me an error she had encountered, and I was glad to learn that by tinkering with it, she found her way out of the maze. But that's not all. From seeing her approach, I picked up new knowledge from it too.


This post is meant to give you a quasi-roadmap, which you don't have to follow religiously but keep close enough to consult when you're lost.

So you learnt Python alongside its arguably most popular web-development framework: Django; and now you can code the next Facebook (Oh, were you told you could only build blogs with that new super power you picked up?) Now, I think the world has more blogging apps than we can keep track of, don't you think so?

So get creative; I assure you you've gotten past the first and most basic hurdle to creating just about any kind of web app: a social network or an e-commerce platform or something else that's interesting. Think of how important this is in an age when everything is moving to the web. You've definitely gotten a head start!

But you'd need to be prepared to learn new stuff as you go. Don't let that put you off as those things would be equally exciting. And you don't have to know it all before you begin; you can learn on-the-job.


Speaking of what you learnt at the workshop, I'm sure you know you were merely introduced to the technologies, not taught comprehensively. In my opinion, you may need to go over the whole tutorial again; then try to build something different -- like a ToDo application -- from the example shown. Afterwards, for some more comprehensive Python tutorial that is hands-on, try codecademy. It'll be fun, trust me.

There are several tutorials available as MOOCs or video tutorials on web platforms like Youtube, Edx, Udemy, Coursera, and Udacity among others. Who knows, you could build the next e-school for Africa. Pick any of these courses, and try to follow along. You don't have to be in a hurry, although you may find yourself skipping stuff you have already learnt.

I'll advise you pick up some Javascript as you go. I assure you that a mix of skills in these two languages will empower you so well.

I also recommend you put up blog posts as you learn new stuff as if to teach same to others (teaching is a proven way to better learn anything). Medium would be a good place to start, but there are several alternatives. You should also read blogs often. Let me mention that Django Girls has a blog you might be interested in checking out and so does PyLadies. See more interesting Python blogs here


During the workshop, I told my mentees that Python is a suitable language for such a workshop for three reasons:

  1. It's easy to learn.
  2. It's platform-independent.
  3. It has a lot of application domains.

Take caution not to misinterpret reason-1 to mean "easy to master" (See answer to "Is Python easy to master?"), especially because of reason-3: "Python has a lot of application domains". Be it scientific, systems administration, machine learning/artificial intelligence, statistical data analysis, game development or scripting, web scraping, and web development. For those with IoT in mind, especially with Raspberry Pi and or Arduino, Python is readily applicable to those platforms.

So you see there's quite a long way to go, yet with what little knowledge you've gained, you're able to achieve something worthwhile.


Moving on, you can share your work with others, and work collaboratively. You were introduced to version-controlling using Git and Github, which are well worth some learning time. You can learn Git through this course. Don't worry if you don't get it all at once, I struggled through this too at first. This video by Git evangelist, Scott Chacon also deals a good deal on the subject.


How much time it'd take to become quite comfortable with all of this would depend on three things:

  1. Your interest
  2. Your consistency
  3. Your learning aptitude

However, I'd say 6 months should be fine. Within this time you would have learnt enough to kick-start an exciting project. In fact, some people might have gotten halfway with some project in this time.


Good luck in your programming journey!

by Tosin Damilare James Animashaun at April 09, 2016 02:10 PM

April 08, 2016

Suraj Deshmukh

Setting up ElasticKube on CentOS

ElasticKube is ‘Enterprise container management for Kubernetes’ an option to various other open-source software projects like Nulecule, LocalKube, Ansible, etc. but only supports Kubernetes.

AAEAAQAAAAAAAAUNAAAAJDE0ZjJiNTkxLTFjNzQtNDZjYy1hY2JhLTZiZTJkMjRkYmE4Ng

You need a working Kubernetes cluster to setup ElasticKube on it. Easiest way to create a single node Kubernetes cluster, follow the steps.

$ git clone https://github.com/projectatomic/adb-atomic-developer-bundle.git
$ cd adb-atomic-developer-bundle/components/centos/centos-k8s-singlenode-setup/
$ vagrant up
$ vagrant ssh

Now you have a Vagrant Box with running Kubernetes cluster on it.                                             m7S2-kubernetes

To setup ElasticKube follow following instructions. Disable SELinux first, because without this ElasticKube won’t run, this is a security risk but for testing purposes you can disable it.

$ sudo setenforce 0

Create namespace named “kube-system”

$ cat namespace.yaml
kind: "Namespace"
apiVersion: "v1"
metadata:
  name: "kube-system"
  labels:
    name: "kube-system"

Install ElasticKube

$ curl -s https://elastickube.com | bash

When successfully installed, note the IP address at the end of the above command, which says “Please complete the installation here: http://10.254.177.213“.

Find the IP address of this machine

$ ip a sh eth0

Open new terminal and create a SSH port-forwarding this is done so that you can access the ElasticKube running inside the Vagrant Box.

$ ssh -L 'some-port':'IP-address-from-output':80 vagrant@'IP-address-of-VM'
password: vagrant

e.g.

$ ssh -L 9898:10.254.177.213:80 vagrant@192.168.121.24

Now open browser and goto localhost:9898 and you are good to go.

 

Ref:


by surajssd009005 at April 08, 2016 06:40 AM

April 04, 2016

Shakthi Kannan

Introduction to GNU Emacs

[Published in Open Source For You (OSFY) magazine, July 2015 edition.]

Introduction

GNU Emacs is a very popular text editor written in C and Emacs Lisp. It can be run on many platforms, and can be easily customised and extended for user needs. It was created by Richard Stallman, the founder of the GNU project. This article is the first in a series on how to use it. There are a number of tutorials available for GNU Emacs; this series of articles provide one approach to understanding and learning the software. You are encouraged to refer to the official GNU Emacs reference manual for more information, and it supersedes everything. Here are a few interesting quotes on Emacs.

“I use Emacs, which might be thought of as a thermonuclear word processor” – Neal Stephenson (In the Beginning… Was the Command Line)

“Emacs is undoubtedly the most powerful programmer’s editor in existence. It’s a big, feature-laden program with a great deal of flexibility and customizability. … Emacs has an entire programming language inside it that can be used to write arbitrarily powerful editor functions.” – Eric S. Raymond (from ‘The Art of UNIX Programming’)

“Personally, I feel inspired whenever I open Emacs. Like a craftsman entering his workshop, I feel a realm of possibility open before me. I feel the comfort of an environment that has evolved over time to fit me perfectly - an assortment of packages and keybindings which help me bring ideas to life day after day.” – Daniel Higginbotham (in ‘Clojure for the Brave and True’)

“EMACS could not have been reached by a process of careful design, because such processes arrive only at goals which are visible at the outset, and whose desirability is established on the bottom line at the outset. Neither I nor anyone else visualized an extensible editor until I had made one, nor appreciated its value until he had experienced it. EMACS exists because I felt free to make individually useful small improvements on a path whose end was not in sight.” – Richard Stallman

Installation

You can use your favourite GNU/Linux distribution package manager to install GNU Emacs. On Debian/Ubuntu, you can install with the following command:

$ sudo apt-get install emacs

On Fedora, you can use the Yum package manager as shown below:

$ sudo yum install emacs

The emerge tool can install GNU Emacs on Gentoo, as follows:

# emerge --ask app-editors/emacs

On Arch, the Pacman software can help you in installing GNU Emacs:

$ sudo pacman -S emacs

Use the Zypper package manager in SUSE as shown below:

$ sudo zypper install emacs

Startup

If you use the Unity interface, you can search for emacs in the Dash, and it will show the Emacs icon that you can click to open the editor. This is illustrated in the following figure.

Unity Emacs search image

On the Metacity interface or any other desktop environment with a panel, you can open GNU Emacs from “Applications” -> “Accessories” -> “Emacs” as shown below:

Metacity Emacs

You can also open the editor from the terminal by simply typing ‘emacs’ and hitting the return key.

$ emacs

The version that I have used for this article is GNU Emacs 24.3.1.

Exit

To exit from GNU Emacs, you need to use C-c C-q, where ‘C’ stands for the Control key. You can also use your mouse and close the editor by clicking on the ‘x’, but, GNU Emacs was designed to be completely usable with a keyboard, and I am going to encourage you to only use the keyboard shortcuts.

Concepts

While you can work using GUI editors, I am going to teach you to work entirely on the keyboard to experience the power of shortcuts in GNU Emacs. You can disconnect the mouse and your touchpad when working on GNU Emacs. Just as an exercise, I’d encourage you to remove your mouse completely and see how you can work with a computer for one day. You will realise that a lot of user interfaces are heavily dependent and designed for mouse interactions! By only using the keyboard, you can be blazingly fast and productive.

The keyboard shortcuts in GNU Emacs may seem to involve many keys. But, please bear with me on this, because the way the shortcuts are designed, you will be able to remember them easily. As you practice, you will gain insight into how consistently they have been defined.

GNU Emacs is a ‘stateless’ editor for the most part. By ‘stateless’, I mean that there are no specific state transitions that need to happen before you can use the commands. There does exist the concept of modes. When you open GNU Emacs, you will see menus, buffer and a mode line as illustrated in the following figure:

Emacs default screen

As mentioned earlier, we will not be clicking on the menus or icons with a mouse, but only use keyboard shortcuts. Everything is a buffer in GNU Emacs. Each buffer can have one major mode and one or more minor modes. The mode determines the keyboard shortcuts that are applicable primarily on the buffer. Examples of major modes are given below:

Mode Description
Text mode Writing text
HTML mode Writing HTML
cc mode Writing C, C++ and C-like programs
Dired mode Handling files and directories
Shell mode Working with shell
LaTeX mode Formatting TeX and LaTeX files
Picture mode Creating ASCII art
Outline mode Writing outlines
SQL mode Interacting with SQL databases
Lisp mode Writing Lisp programs

The mode line exists below the buffer and it gives you a lot of information on the status of the buffer, such as what modes are active and other useful information. Below the mode line is the mini-buffer, where any commands you issue are indicated and prompts for user input are shown. This is the overall view of the default GNU Emacs interface.

In today’s user interface applications, an application is treated as a window on a desktop. But, when you open GNU Emacs, you are actually opening a frame. A frame can be split it into many windows. Everything is a buffer in GNU Emacs. So, you can have many frames of GNU Emacs, and inside each you can have one or more windows containing buffers (or files).

Features

Although GNU Emacs was designed primarily for text editing, it can do a lot more. ‘A lot’ is probably a highly simplified term. It has got support for syntax highlighting for a large number of programming languages. You can also generate code snippets from templates using the yasnippet package. You can also enter markup or markdown text. There is support for indentation of text and programs depending on the programming languages and modes. Internationalization support is available, and you can use it to even enter text in Indian languages.

A number of configurations are available for setting it up for your development work, including automating tasks for compiling, executing, testing and deployment. When you become familiar with Emacs Lisp, you can implement your own modules. Since, it is Lisp, it is also easily extensible. You can write your own macros to perform repeated tasks. You can also query an inbuilt help system for information, shortcuts, tutorials and other documentation. You are not at all dependent on the Internet for information, and thus you can work offline too. Version control support is available for many centralized (cvs, svn) and decentralized systems (Git, Hg).

org-mode is a very popular mode for managing your notes. You can use it for planning your day-to-day activities. GNU Emacs can be used as publishing software. You can create wikis, blogs and publish books using it. This article is written using org-mode. Spell-checking modules are also available for your documentation needs. It is also possible to export plain text into a number of formats (PDF, HTML etc.).

A number of Emacs lisp packages are available for networking. You can use Gnus for checking your e-mails, and reading and writing to newsgroups. Emacs Relay Chat (ERC) can be used for connecting to Internet Relay Chat (IRC) channels. There are modules that support the Jabber protocol for communicating with chat servers. There is also support for viewing Web pages inside Emacs. There are a large number of Emacs modules available through package repositories such as MELPA (melpa.org), and Marmalada (marmalade-repo.org).

History

The first version of Emacs (macros) was written by Richard Stallman and Guy L. Steele, Jr. in 1976 for the TECO editor in MIT. It was written in a low-level language for the PDP-10 assembler. People were able to freely hack on the code, make improvements and share their changes. This was the original hacker culture that existed in MIT. Unfortunately, business entities started to make software proprietary and this hacker culture ceased to exist, especially in the MIT AI labs. Richard Stallman wanted to revive the hacker culture and started the GNU project. He wrote the second implementation of the editor entirely in C in 1984, and released it as the first program of the GNU project. Today, it is a very popular editor that is widely used, and has more than 570 contributors. The official web site for GNU Emacs is at http://www.gnu.org/software/emacs/.

April 04, 2016 03:15 PM