Planet dgplug

July 19, 2019

Kushal Das

Setting up WKD

We fetch any GPG public key from the keyservers using the GPG fingerprint (or parts of it). This step is still a problematic one for most of us. As the servers may not be responding, or the key is missing (not pushed) to the server. Also, if we only have the email address, there is no easy way to download the corresponding GPG key.

Web Key Directory to rescue

The Web Key Directory comes to the picture. We use WKD to enable others to get our GPG keys for email addresses very easily. In simple terms:

The Web Key Directory is the HTTPS directory from which keys can be fetched.

Let us first see this in action:

gpg --auto-key-locate clear,wkd --locate-key mail@kushaldas.in

The above will fetch you the key for the email address, and you can also assume the person who owns the key also has access to the https://kushaldas.in server.

There are many available email clients, which will do this for you. For example Thunderbird/Enigmail 2.0 or Kmail version 5.6 onwards.

Setting up WKD for your domain

I was going through the steps mentioned in the GNUPG wiki, while weasel pointed to me to a Makefile to keep things even more straightforward.

all: update install

update:
        rm -rfv openpgpkey
        mkdir -v openpgpkey
        echo 'A85FF376759C994A8A1168D8D8219C8C43F6C5E1 mail@kushaldas.in' | /usr/lib/gnupg/gpg-wks-client -v --install-key
        chmod -v 0711 openpgpkey/kushaldas.in
        chmod -v 0711 openpgpkey/kushaldas.in/hu
        chmod -v 0644 openpgpkey/kushaldas.in/hu/*
        touch openpgpkey/kushaldas.in/policy

        ln -s kushaldas.in/hu openpgpkey/
        ln -s kushaldas.in/policy openpgpkey/

install: update
        rsync -Pravz --delete ./openpgpkey root@kushaldas.in:/usr/local/www/kushaldas.in/.well-known/

.PHONY: all update install

The above Makefile is using gpg-wks-client executable and also pushing the changes to the right directory on the server.

Email providers like protonmail already allow users to publish similar information. I hope this small Makefile will help you to set up your domain.

by Kushal Das at July 19, 2019 05:35 AM

July 15, 2019

Jason Braganza (Work)

How Do You Keep Keep Going?

Or how do you actually go do anything else you committed to do for yourself?
I always got confused on what to do when the going got tough and life happened and my goals then got waylaid.
Other than feeling lost and giving up on projects and promising to do better tomorrow, or next time?
(which took a looooooong time to come)
What could I do?

James Clear offers a lovely heuristic, that I have been applying to my writing since the year began.
(along with Seth’s advice to queue things up)

3. Reduce the scope, but stick to the schedule.

I've written previously about the importance of holding yourself to a schedule and not a deadline.
There might be occasions when deadlines make sense, but I'm convinced that when it comes to doing important work over the long–term, following a schedule is much more effective.

When it comes to the day-to-day grind, however, following a schedule is easier said than done.
Ask anyone who plans to workout every Monday, Wednesday, and Friday, and they can tell you how hard it is to actually stick to their schedule every time without fail.

To counteract the unplanned distractions that occur and overcome the tendency to be pulled off track, I've made a small shift in how I approach my schedule.

My goal is to put the schedule first and not the scope, which is the opposite of how we usually approach our goals.

For example, let's say you woke up today with the intention of running 3 miles this afternoon.
During the day, your schedule got crazy and time started to get away from you.
Now you only have 20 minutes to workout.

At this point, you have two options.

The first is to say, “I don't have enough time to workout today,” and spend the little time you have left working on something else.
This is what I would usually have done in the past.

The second option is to reduce the scope, but stick to the schedule. Instead of running 3 miles, you run 1 mile or do five sprints or 30 jumping jacks.
But you stick to the schedule and get a workout in no matter what.
I have found far more long-term success using the this approach than the first.

On a daily basis, the impact of doing five sprints isn't that significant, especially when you had planned to run 3 miles.
But the cumulative impact of always staying on schedule is huge. No matter what the circumstance and no matter how small the workout, you know you're going to finish today's task.
That's how little goals become lifetime habits.

Finish something today, even if the scope is smaller than you anticipated.

If you like this tip the whole post is even more awesome.
Go find out Time Management Tips That Actually Work on his blog.

P.S. You should subscribe to the mailing list, you know. :)
P.P.S I haven’t missed a single week since I started doing this!


by Mario Jason Braganza at July 15, 2019 02:30 AM

July 09, 2019

Kushal Das

Highest used Python code in the Pentesting/Security world

python -c 'import pty;pty.spawn("/bin/bash")'

I think this is the highest used Python program in the land of Pentesting/Security, Almost every blog post or tutorial I read, they talk about the above-mentioned line to get a proper terminal after getting access to a minimal shell on a remote Linux server.

What does this code do?

We are calling the Python executable with -c and python statements inside of the double quote. -c executes the Python statements, and as we are running it as non-interactive mode, it parses the entire input before executing it.

The code we pass as the argument of the -c has two statements.

import pty
pty.spawn("/bin/bash")

pty is a Python module which defines operations related to the pseudo-terminal concept, it can create another process, and from the controlling terminal, it can read/write to the new process.

The pty.spawn function spawns a new process (/bin/bash in this case) and then connects IO of the new process to the parent/controlling process.

demo of getting bash

In most cases, even though you get access to bash using the way mentioned above, TAB completion is still not working. To enable it, press Ctrl+z to move the process to sleep, and then use the following command on your terminal.

stty raw -echo

stty changes terminal line settings and part of the GNU coreutils package. To read about all the options we set by using raw -echo, read the man page of stty.

Many years ago, I watched a documentary about Security firms showcasing offensive attacks, that was the first I saw them using Python scripts to send in the payload and exploit the remote systems. Now, I am using similar scripts in the lab to learn and having fun with Python. It is a new world for me, but, it also shows the diverse world we serve via Python.

by Kushal Das at July 09, 2019 05:34 AM

July 08, 2019

Jason Braganza (Personal)

Books I’ve Read, June Edition

’Twas a good month for reading :)

June

  • The Song of the Bird, Anthony de Mello
    (absolutely read. buy and give people copies.
    this book for me, goes beyond a quake book.
    it has shaped my life, and thoughts, since boyhood, subconsciously then and with intent now.)

  • The Mysterious Affair at Styles (Poirot #1), Agatha Christie
    (re-reading my way through the Poirot canon.
    these books take me to the age I think I belong to, the late 1800s, early 1900s absolutely delightful)

  • Epigrams on Men, Women and Love, Honoré de Balzac
    (beautiful set of quotes)

  • Mother American Night, John Perry Barlow
    (a man who lived life. founder of the EFF and the FSF.
    and more importantly (to me), lyricist for the Grateful Dead)

  • Word by Word, Anne Lamott
    (must listen (it’s an old audiobook.)
    excellent companion to Bird by Bird.
    imagine Anne teaching you how to write using BbB as a text book.
    she’s awesome.
    the book’s awesome.)

  • Indian Love Poems, Peter Pauper Press
    (absolutely loved it)

  • Love Poems and Love Letters for All the Year, Peter Pauper Press

  • Flower Thoughts, Peter Pauper Press

  • Thoughts for a Good Life, Peter Pauper Press

  • Epigrams by Oscar Wilde, Peter Pauper Press

  • Murder on the Links (Poirot #2), Agatha Christie
    (need i say, you ought to read it :))

More lists of books to read? Subscribe!


by Mario Jason Braganza at July 08, 2019 12:15 AM

Robin Schubert

Book review: Shakthi Kannan - I want to do project. Tell me wat to do.

I want to do project.

I remember when I graduated from the University, I knew some Physics and how to do some coding or how to write a thesis. What I didn't know was how to collaborate and work in a team, or how to address my issues in mailing lists and IRC channels appropriately. Actually I didn't even know that I didn't know.

I've written about my experience with the #dgplug summertraining earlier. Very slowly I started to discover that I'm lacking the very basics of communication, coding style and organization. I knew some programming but I did not have the means to let anyone else but me benefit from that.

Shakthi Kannan, the author of this book, is one of the mentors in the dgplug summertraining and I owe him much for the valuable lessons learnt.

Rules & Tools to guide you

This book will not teach you coding, but it can tell you how to code. It teaches the rules and tools you need to know, to contribute to Free and Open Source Software and to become part of a world wide community who's efforts power most of today's internet and devices.

It is full of habits and styleguides you should adopt, manners and means to endure, that you need to know to start off with valuable contributions and to avoid frustration early on.

The pure basics

While the content of this book is something that most graduated students are not aware of, these are the pure basics which you need to know and follow for FOSS contribution. If you ever wrote an email to a mailing list or posted a question to an IRC channel and wondered why you either received no helpful responses or none at all, you probably know what this is about.

It contains many real world examples of miserable communication or disadvantageous code formatting that FOSS developers don't have the time to deal with every day. You will have a much better experience if you know the pitfalls to look out for beforehand.

There are chapters that walk you through the steps of triaging a bug, fixing it and making the fix available to be merged; the standard workflows for FOSS contribution.

If you think you want to contribute but you don't know where to start: Read this book!

by Robin Schubert at July 08, 2019 12:00 AM

July 06, 2019

Jason Braganza (Personal)

Happy Birthday Abbygail

Click the pic for a larger one


Love looks pretty on you.
Makes you soft, tender, proud.
Makes you sit up and take notice.
Gives you a home to set down your things.


What a blessing it is, to have music and dancing and poetry.
What a gift it is, to look at someone and say,
I’m so happy to have found you
at last, at last, at long, long last
you’re here.

Lang Leav

It has also been a blessing to walk by your side, sharing our lives.
It’s hard to think of me, without you anymore.
Like Leav writes, “Every time I see my name, I hear it in your voice.”
Happy Birthday, my heart!
I love you.


by Mario Jason Braganza at July 06, 2019 03:45 AM

June 20, 2019

Bhavin Gandhi

infracloud.io: Introducing : Tracing Cassandra with Jaeger

This is the blog post about a plugin for Cassandra, which I wrote few days back. It covers basic information about three pillars of observability, which are logging, metrics and tracing. Thanks to Sameer, who helped me with my doubts related to Java and Maven. The blog post was published at infracloud.io on 19th June, 2019. Introducing : Tracing Cassandra with Jaeger

by @_bhavin192 (Bhavin Gandhi) at June 20, 2019 04:07 PM

June 14, 2019

Bhavin Gandhi

Triaging bugs of GNU Emacs

I have been using Emacs for more than 6 months. mbuf and me were discussing about some way to package Emacs as container image so that we can run that image to test things, triage bugs for a particular version etc. Before getting started with that, he wanted me to have idea about existing bugs of Emacs and how to triage them, so that I get better idea of whole work flow.

by @_bhavin192 (Bhavin Gandhi) at June 14, 2019 12:23 PM

June 08, 2019

Robin Schubert

qutebrowser

It's been the third plugin for Firefox, that was intended to enable the use of vim keybindings to browse the web or open and close tabs. I've been a long time Firefox user now; it is free and open source and the Mozilla Foundation does a lot for the FOSS community (although I cannot judge about that). Still, I've switched to Pale Moon for quite some time, as I found its privacy policy to be more straight forward, and I often do not understand decisions Firefox makes for removing or implementing several modules (RSS, pulseaudio, etc). However, when I was feeling a bit limited by Pale Moon (e.g. with not supporting rtmp at all), I was switching back to Firefox for compatibility reasons.

However, I have one requirement: I do want to control the browser with the keyboard, rather than with my mouse, preferably using vim keybindings. I was using plug-ins to gain that functionality for a long time, starting with Pentadactyl, then Vimperator and finally Vim-Vixen. Eventually, they all broke after some Firefox updates and I had to look around for the next solution.

A week ago I tried qutebrowser for the first time, I don't think I will need something else any time soon. qutebrowser does exactly what I want it to do, and coming from vim-controlled Firefox, it was so intuitive to use as if I was using it for years already.

I actually stumbled about it when I was looking for successful FOSS PyQt5 projects, so qutebrowser became the first browser for which I started reading the source code - even before I started to use the browser regularly.

As of now, after ~4 weeks of daily use, I did not find any page or web element that could not be rendered properly. The only issue I had was when I ran

pip install --user PyQt5

to install the Python Qt5 bindings for local use. Unfortunately, this put a newer version of PyQt5 to my $PATH which conflicted with qutebrowser. So I uninstalled and installed into a virtual environment instead, which is much cleaner anyway.

This post is meant to be my personal recommendation for a fast and lightweight FOSS browser and a big THANK YOU for all the contributors!

by Robin Schubert at June 08, 2019 12:00 AM

June 02, 2019

Shakthi Kannan

Building Erlang/OTP sources with Ansible

[Published in Open Source For You (OSFY) magazine, September 2017 edition.]

Introduction

Erlang is a programming language designed by Ericsson primarily for soft real-time systems. The Open Telecom Platform (OTP) consists of libraries, applications and tools to be used with Erlang to implement services that require high availability. In this article, we will create a test Virtual Machine (VM) to compile, build, and test Erlang/OTP from its source code. This allows you to create VMs with different Erlang release versions for testing.

The Erlang programming language was developed by Joe Armstrong, Robert Virding and Mike Williams in 1986 and released as free and open source software in 1998. It was initially designed to work with telecom switches, but is widely used today in large scale, distributed systems. Erlang is a concurrent and functional programming language, and is released under the Apache License 2.0.

Setup

A CentOS 6.8 Virtual Machine (VM) running on KVM will be used for the installation. Internet access should be available from the guest machine. The VM should have at least 2 GB of RAM alloted to build the Erlang/OTP documentation. The Ansible version used on the host (Parabola GNU/Linux-libre x86_64) is 2.3.0.0. The ansible/ folder contains the following files:

ansible/inventory/kvm/inventory
ansible/playbooks/configuration/erlang.yml

The IP address of the guest CentOS 6.8 VM is added to the inventory file as shown below:

erlang ansible_host=192.168.122.150 ansible_connection=ssh ansible_user=bravo ansible_password=password

An entry for the erlang host is also added to the /etc/hosts file as indicated below:

192.168.122.150 erlang

A ‘bravo’ user account is created on the test VM, and is added to the ‘wheel’ group. The /etc/sudoers file also has the following line uncommented, so that the ‘bravo’ user will be able to execute sudo commands:

## Allows people in group wheel to run all commands
%wheel	ALL=(ALL)	ALL

We can obtain the Erlang/OTP sources from a stable tarball, or clone the Git repository. The steps involved in both these cases are discussed below:

Building from the source tarball

The Erlang/OTP stable releases are available at http://www.erlang.org/downloads. The build process is divided into many steps, and we shall go through each one of them. The version of Erlang/OTP can be passed as an argument to the playbook. Its default value is the release 19.0, and is defined in the variable section of the playbook as shown below:

vars:
  ERL_VERSION: "otp_src_{{ version | default('19.0') }}"
  ERL_DIR: "{{ ansible_env.HOME }}/installs/erlang"
  ERL_TOP: "{{ ERL_DIR }}/{{ ERL_VERSION }}"
  TEST_SERVER_DIR: "{{ ERL_TOP }}/release/tests/test_server"

The ERL_DIR variable represents the directory where the tarball will be downloaded, and the ERL_TOP variable refers to the top-level directory location containing the source code. The path to the test directory from where the tests will be invoked is given by the TEST_SERVER_DIR variable.

Erlang/OTP has mandatory and optional package dependencies. Let’s first update the software package repository, and then install the required dependencies as indicated below:

tasks:
  - name: Update the software package repository
    become: true
    yum:
      name: '*'
      update_cache: yes

  - name: Install dependencies
    become: true
    package:
      name: "{{ item }}"
      state: latest
    with_items:
      - wget
      - make
      - gcc
      - perl
      - m4
      - ncurses-devel
      - sed
      - libxslt
      - fop

The Erlang/OTP sources are written using the ‘C’ programming language. The GNU C Compiler (GCC) and GNU Make are used to compile the source code. The ‘libxslt’ and ‘fop’ packages are required to generate the documentation. The build directory is then created, the source tarball is downloaded and it is extracted to the directory mentioned in ERL_DIR.

- name: Create destination directory
  file: path="{{ ERL_DIR }}" state=directory

- name: Download and extract Erlang source tarball
  unarchive:
    src: "http://erlang.org/download/{{ ERL_VERSION }}.tar.gz"
    dest: "{{ ERL_DIR }}"
    remote_src: yes

The ‘configure’ script is available in the sources, and it is used to generate the Makefile based on the installed software. The ‘make’ command will build the binaries from the source code.

- name: Build the project
  command: "{{ item }} chdir={{ ERL_TOP }}"
  with_items:
    - ./configure
    - make
  environment:
    ERL_TOP: "{{ ERL_TOP }}"

After the ‘make’ command finishes, the ‘bin’ folder in the top-level sources directory will contain the Erlang ‘erl’ interpreter. The Makefile also has targets to run tests to verify the built binaries. We are remotely invoking the test execution from Ansible and hence -noshell -noinput are passed as arguments to the Erlang interpreter, as show in the .yaml file.

- name: Prepare tests
  command: "{{ item }} chdir={{ ERL_TOP }}"
  with_items:
    - make release_tests
  environment:
    ERL_TOP: "{{ ERL_TOP }}"

- name: Execute tests
  shell: "cd {{ TEST_SERVER_DIR }} && {{ ERL_TOP }}/bin/erl -noshell -noinput -s ts install -s ts smoke_test batch -s init stop"

You need to verify that the tests have passed successfully by checking the $ERL_TOP/release/tests/test_server/index.html page in a browser. A screenshot of the test results is shown in Figure 1:

Erlang test results

The built executables, libraries can then be installed on the system using the make install command. By default, the install directory is /usr/local.

- name: Install
  command: "{{ item }} chdir={{ ERL_TOP }}"
  with_items:
    - make install
  become: true
  environment:
    ERL_TOP: "{{ ERL_TOP }}"

The documentation can also be generated and installed as shown below:

- name: Make docs
  shell: "cd {{ ERL_TOP }} && make docs"
  environment:
    ERL_TOP: "{{ ERL_TOP }}"
    FOP_HOME: "{{ ERL_TOP }}/fop"
    FOP_OPTS: "-Xmx2048m"

- name: Install docs
  become: true
  shell: "cd {{ ERL_TOP }} && make install-docs"
  environment:
    ERL_TOP: "{{ ERL_TOP }}"

The total available RAM (2 GB) is specified in the FOP_OPTS environment variable. The complete playbook to download, compile, execute the tests, and also generate the documentation is given below:

---
- name: Setup Erlang build
  hosts: erlang
  gather_facts: true
  tags: [release]

  vars:
    ERL_VERSION: "otp_src_{{ version | default('19.0') }}"
    ERL_DIR: "{{ ansible_env.HOME }}/installs/erlang"
    ERL_TOP: "{{ ERL_DIR }}/{{ ERL_VERSION }}"
    TEST_SERVER_DIR: "{{ ERL_TOP }}/release/tests/test_server"

  tasks:
    - name: Update the software package repository
      become: true
      yum:
        name: '*'
        update_cache: yes

    - name: Install dependencies
      become: true
      package:
        name: "{{ item }}"
        state: latest
      with_items:
        - wget
        - make
        - gcc
        - perl
        - m4
        - ncurses-devel
        - sed
        - libxslt
        - fop

    - name: Create destination directory
      file: path="{{ ERL_DIR }}" state=directory

    - name: Download and extract Erlang source tarball
      unarchive:
        src: "http://erlang.org/download/{{ ERL_VERSION }}.tar.gz"
        dest: "{{ ERL_DIR }}"
        remote_src: yes

    - name: Build the project
      command: "{{ item }} chdir={{ ERL_TOP }}"
      with_items:
        - ./configure
        - make
      environment:
        ERL_TOP: "{{ ERL_TOP }}"

    - name: Prepare tests
      command: "{{ item }} chdir={{ ERL_TOP }}"
      with_items:
        - make release_tests
      environment:
        ERL_TOP: "{{ ERL_TOP }}"

    - name: Execute tests
      shell: "cd {{ TEST_SERVER_DIR }} && {{ ERL_TOP }}/bin/erl -noshell -noinput -s ts install -s ts smoke_test batch -s init stop"

    - name: Install
      command: "{{ item }} chdir={{ ERL_TOP }}"
      with_items:
        - make install
      become: true
      environment:
        ERL_TOP: "{{ ERL_TOP }}"

    - name: Make docs
      shell: "cd {{ ERL_TOP }} && make docs"
      environment:
        ERL_TOP: "{{ ERL_TOP }}"
        FOP_HOME: "{{ ERL_TOP }}/fop"
        FOP_OPTS: "-Xmx2048m"

    - name: Install docs
      become: true
      shell: "cd {{ ERL_TOP }} && make install-docs"
      environment:
        ERL_TOP: "{{ ERL_TOP }}"

The playbook can be invoked as follows:

$ ansible-playbook -i inventory/kvm/inventory playbooks/configuration/erlang.yml -e "version=19.0" --tags "release" -K

Build from Git repository

We can build the Erlang/OTP sources from the Git repository. The complete playbook is given below for reference:

- name: Setup Erlang Git build
  hosts: erlang
  gather_facts: true
  tags: [git]

  vars:
    GIT_VERSION: "otp"
    ERL_DIR: "{{ ansible_env.HOME }}/installs/erlang"
    ERL_TOP: "{{ ERL_DIR }}/{{ GIT_VERSION }}"
    TEST_SERVER_DIR: "{{ ERL_TOP }}/release/tests/test_server"

  tasks:
    - name: Update the software package repository
      become: true
      yum:
        name: '*'
        update_cache: yes

    - name: Install dependencies
      become: true
      package:
        name: "{{ item }}"
        state: latest
      with_items:
        - wget
        - make
        - gcc
        - perl
        - m4
        - ncurses-devel
        - sed
        - libxslt
        - fop
        - git
        - autoconf

    - name: Create destination directory
      file: path="{{ ERL_DIR }}" state=directory

    - name: Clone the repository
      git:
        repo: "https://github.com/erlang/otp.git"
        dest: "{{ ERL_DIR }}/otp"

    - name: Build the project
      command: "{{ item }} chdir={{ ERL_TOP }}"
      with_items:
        - ./otp_build autoconf
        - ./configure
        - make
      environment:
        ERL_TOP: "{{ ERL_TOP }}"

The ‘git’ and ‘autoconf’ software packages are required for downloading and building the sources from the Git repository. The Ansible Git module is used to clone the remote repository. The source directory provides an otp_build script to create the configure script. You can invoke the above playbook as follows:

$ ansible-playbook -i inventory/kvm/inventory playbooks/configuration/erlang.yml --tags "git" -K

You are encouraged to read the complete installation documentation at: https://github.com/erlang/otp/blob/master/HOWTO/INSTALL.md.

June 02, 2019 07:45 PM

May 28, 2019

Saptak Sengupta

What's a ShadowDOM?

I first came to know about Shadow DOM from my curiosity about how browsers implement tags like <video> with all the controllers or <input> which changes based on the type attribute. We can never see any HTML code for these implementation and yet they are shown. How? That is when I stumbled upon this blog by Dimitri Glazkov which explains beautifully the concept of shadow DOM encapsulation used by browsers to implement such tags.

However, none of the browsers allowed developers to write their own custom shadow DOM (though google chrome had a v0 version implemented). I stumbled upon shadow DOM again while looking at an issue in jQuery to fix. So now, since 2018, most of the browsers have started supporting shadow DOM APIs and hence jQuery needed to implement support for that too.
https://caniuse.com/#feat=shadowdomv1
So, what is this shadow DOM and why do we even use it?

What's a DOM?

W3C specification describes it as "a method of combining multiple DOM trees into one hierarchy and how these trees interact with each other within a document, thus enabling better composition of the DOM".

Now, to understand that, we need to understand what a DOM is. DOM or Document Object Model is a tree-like structure containing the different elements (or tags) and strings of text that are shown by the markup language (like HTML, XML, etc.).

So, let's say we have a HTML code something like this:

<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>Title</title>
</head>
<body>
<div>
<h1>This is header</h1>
<p>This is a
<a href="https://www.saptaks.blog">
link
</a>
</p>
</div>
</body>
</html>

So, visually you can show the DOM structure as something like:




Shadow DOM

Now, shadow DOM allows us to create separate hidden DOM trees that are attached to the elements of a regular DOM tree. This allows you to implement functional encapsulation, i.e. someone parsing the regular DOM tree and applying styling to the regular DOM tree doesn't know or affect the properties and functionalities of the shadow DOM tree. Hence you use the shadow DOM without knowing the intricate details of how the DOM is implemented. This is important, because this follows the basic ideas of Object Oriented Programming.

The shadow DOM tree starts with a shadow root and can then have any regular DOM element attached underneath it.

Let's see an example:

HTML
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>Title</title>
</head>
<body>
<div id="shadowHost">
</div>
</body>
</html>

JavaScript
const shadowHost = document.getElementById('shadowHost');
const shadowRoot = shadowHost.attachShadow({mode: 'open'});
shadowRoot.innerHTML = '<h1>Hello Shadow DOM</h1>';

So, this will create a shadow DOM. Visually you can represent this as:


So, as you can see, there are few different parts in a shadow DOM apart from it being just another DOM.
  • Shadow tree: The DOM tree inside the shadow DOM.
  • Shadow boundary: the place where the shadow DOM ends, and the regular DOM begins.
  • Shadow root: The root node of the shadow tree.
  • Shadow host: The regular DOM node that the shadow DOM is attached to.
  • Shadow child: The tree below the shadow root node.
The shadow DOM cannot be attached to few different elements as mentioned in the spec. Some of the reasons are:
  • The different form tags such as <input>, <textarea>, etc or any other html tag for which the browser implements its own Shadow DOM
  • Elements like <img> or <br> or <hr> which are usually self enclosing tags and don't usually contain a child node.
Also if you see in the code there is a "{mode: open}". The mode determines whether you can edit the style of the shadow DOM from outside the shadow DOM or not.

Why do we need Shadow DOM anyways?

There are few different scenarios where you might want to use shadow DOM. The most important functionality of shadow DOM is the implementation of the concept of encapsulation. So basically, someone using the shadow DOM host doesn't need to care the style and implementation of the DOM inside. So few use-cases would be following:
  • The browser implements Shadow DOM for various different tags such as <video>, <audio>, <input>, <select>, etc.
  • You can make your own custom Shadow DOM when you need to create an element that you want to not be modified by the styling of remaining DOM.
  • You can also use Shadow DOM, when you want to create a separation of a particular DOM element from the main DOM element.
So is Shadow DOM very shadow-ey? Well maybe, since it stays hidden from the main DOM traversal. But at the same time it is often very useful because of its encapsulation properties.

by SaptakS (noreply@blogger.com) at May 28, 2019 01:55 PM

April 08, 2019

Subho

Increasing Postgres column name length

This blog is more like a bookmark for me, the solution was scavenged from internet. Recently I have been working on an analytics project where I had to generate pivot transpose tables from the data. Now this is the first time I faced the limitations set on postgres database. Since its a pivot, one of my column would be transposed and used as column names here, this is where things started breaking. Writing to postgres failed with error stating column names are not unique. After some digging I realized Postgres has a column name limitation of 63 bytes and anything more than that will be truncated hence post truncate multiple keys became the same causing this issue.

Next step was to look at the data in my column, it ranged from 20-300 characters long. I checked with redshift and Bigquery they had similar limitations too, 128 bytes. After looking for sometime found a solution, downloaded the postgres source, changed NAMEDATALEN to 301(remember column name length is always NAMEDATALEN – 1) src/include/pg_config_manual.h, followed the steps from postgres docs to compile the source and install and run postgres. This has been tested on Postgres 9.6 as of now and it works.

Next up I faced issues with maximum number columns, my pivot table had 1968 columns and postgres has a limitation of 1600 total columns. According to this answer I looked into the source comments and that looked quite overwhelming 😛 . Also I do not have a control over how many columns will be there post pivot so no matter whatever value i set , in future i might need more columns, so instead I handled the scenario in my application code to split the data across multiple tables and store them.

References:

  1. https://til.hashrocket.com/posts/8f87c65a0a-postgresqls-max-identifier-length-is-63-bytes
  2. https://stackoverflow.com/questions/6307317/how-to-change-postgres-table-field-name-limit
  3. https://www.postgresql.org/docs/9.6/install-short.html
  4. https://dba.stackexchange.com/questions/40137/in-postgresql-is-it-possible-to-change-the-maximum-number-of-columns-a-table-ca

by subho at April 08, 2019 09:25 AM

April 06, 2019

Tosin Damilare James Animashaun

To be Forearmed is to be Help-ready

I felt compelled to write this after my personal experience trying to get help with my code on IRC.

We all love to make the computer do things exactly the way we want, so some of us choose to take the bold step of learning to communicate with the machine. And it is not uncommon to find many of our burgeoning kind go from location to location on the web space trying to get help along the way. We are prompt to ask questions when we sight help.

When you learn to program, you are often encouraged to learn by doing.

The domain of computer programming or software development is a very practical one. Before now, I had carried this very principle everywhere with me -- in fact, preached it -- but hadn't really put it to use.

The thing about learning languages (or technologies) by reading big manuals is that, often times, beginners will approach the process like they would any other literature book. But that is clearly a wrong approach as empirical evidence has shown. You don't read these things to simply stomach them. Instead, you swallow and then post-process. In essence, you ruminate over stuff.


In truth, the only way you can really process what you read is to try things out and see results for yourself.

Weeks ago, while building an app, I visited IRC frequently to ask questions on just about everything that was unclear to me. While this mode of communication and seeking help is encouraged, abuse of it is strongly discouraged. The good guys over on the IRC channels get pissed off when it appears you're boycotting available resources like documentation, preferring to be spoonfed the whole time. (Remember this is not Quora, where the philosophy is for you to ask more and more questions).

This was the sort of thing that happened to me when I began flooding the channels with my persistent querying. Most of the time the IRC folks kept pointing me to the documentation, as workarounds for most of the issues I had were already documented. A lot of things became much clearer when I decided to finally read-the-docs.

What did I learn from that? "Do your own research!" It's so easy to skip this part, but if you make efforts at finding things out for yourself, you'll be surprised at how much you can dig out without having to bug people. However, this does not guarantee that even the few important questions you'll ask may not be met with hostility, but do not let that discourage you. The people who appear to be unwelcoming are doing so only as a way to discourage you from being over-dependent on the channel. Another advantage of finding things for yourself is that, you learn the why and not just the how.

I think it's fair to quote Armin Ronacher here,

"And it's not just asking questions; questioning other people, like what other people do or what you do yourself.

By far, the worst parts in all of my libraries are where I took the design from somewhere else. And it's not because I know better, it's because pretty much everything everybody does at any point in time has some sort of design decision behind it ... that major decision process.

So someone came up with a solution for a particular problem, and he thought about it and then wrote it down. But if you look at someone else's design, you might no longer understand why the decision was made in the first place. And ... in fact, if the original implementation is ten years old, who knows if the design ideas behind it are still entirely correct."


Personally, I like to answer questions and help put people on track. Nonetheless, if the queries got too overwhelming -- especially coming from the same person -- I would eventually lose interest in answering questions.


Let me remind you of some tidbits or etiquettes of IRC:

  • Construct your questions well (concise, well written and straight-to-the-point questions are more likely to attract help)

  • Don't EVER post code in a channel! Pastebin[1] it and share the link in the channel instead. While at it, don't post your entire code (unless you specifically need to). Post only the relevant portion -- the one you have an issue with. The only exception to this is if the snippet of code is considerably short, say one or two lines.

  • Don't be overly respectful. Yes, dont be too respectful -- cut all the 'Sirs'. Only be moderately polite.

  • Ensure you have and use a registered nick. This gives you an identity.

  • This last one is entirely my opinion but it's also based on what I have observed. Don't just be a leech, try to contribute to the community. Answer questions when you can.


So where do you look to before looking to IRC? There are three sources you may read from before turning to internet-relay-chat for help:

  • Read the documentation. * Documentation is the manual the creator or experts of a software product or tool provide their users with. So you want to know the ins and outs of a technology? That's the right place to look.

  • Read blog posts related to your topic-area. Blog posts are often based on people's experiences, so you're likely to find help from there, especially if the writer has faced the same issue. Remember to bookmark the really helpful ones as you go ;).

  • Last and very important. Read the source code!. This is two-fold: First is actually looking into your own code carefully and seeing what syntax or semantic errors you might have made. Secondly, you look into the original code of libraries/frameworks you are using if they are open source, otherwise revert to documentation. With this, you have it all stripped to its bare bones. Point blank! The source code reveals everything you need to know, once you know how to read it.


So why not arm yourself properly before going to post that question. That way, you would not only make it easier to get help [for yourself], but you would be better informed.



  1. Some Pastebin platforms I use:

Note: Because Hastebin heavily depends on Javascript, some people have complained of text-rendering issues possibly arising from browser-compatibility issues with it. So take caution using it. That said, I love its ease-of-use. It supports the use of keyboard shortcuts such as [Ctrl]+[S] to Save.

by Tosin Damilare James Animashaun at April 06, 2019 09:40 PM

March 28, 2019

Shakthi Kannan

"Opportunities in FLOSS" at Computer Society of India, Madras

I had given a talk on “Opportunities in Free (Libre) and Open Source Software”(FLOSS) on Saturday, March 2, 2019 at the Computer Society of India, Madras Chapter, jointly organized by the IEEE Computer Society, Madras Chapter and ACM India Chennai Professional Chapter. The Computer Society of India, Education Directorate is located opposite to the Institute of Mathematical Sciences in Taramani, close to the Tidel Park. Students, IT professionals and professors from IIT Madras, Anna University and engineering colleges in and around Chennai attended the event.

Session in progress

At around 6:00 p.m. people had assembled for networking, snacks and beverage. I started the “Building Careers with FLOSS” presentation at 6:30 p.m. Prof. Dr. D. Janakiram, CSE, IIT Madras also shared his insights on the benefits of learning from source code available under a FLOSS license. The session was very interactive and the audience asked a lot of good questions. Dr. B. Govindarajulu, author of the famous “IBM PC and Clones: Hardware, Troubleshooting and Maintenance” book then presented his views on creating a Computer History Museum in Chennai. Dinner was served around 8:00 p.m. at the venue.

Group photo

A review of my book has been published in Volume 14: No. 1, January-March 2019 IEEE India Council Newsletter. The excerpts of Chapter 4 of my book on “Project Guidelines” is also available. I had also written an article on “Seasons of Code” which is published in this edition of the IEEE newsletter.

Special thanks to Mr. H. R. Mohan, Editor, IEEE India and Chairman, ACM Professional Chapter, Chennai for organizing the event and for the logistics support.

March 28, 2019 01:30 PM