Planet dgplug

September 17, 2021

Robin Schubert

What it means being human (to me)

Disclaimer

One of these posts; not technical, written with a hot head, controversial topics. Don't read if that already annoys you.

The argument

I've had a hot discussion yesterday, and I'm not quite happy how it went. It's not important how we came there but the core argument was about why I think that less to no money should be spend on military and arms, while the same money should be spent on peace studies and education of diplomats - which I refer to as people who try to understand a different culture and initiate an exchange of values and ethics - not eloquent deceitful but open and direct, fair and square.

My opposition's opinion was different, claiming that we would never live in peace without an army enforcing the peace. That's an opinion that I could argue about all day long, but then he said what really is upsetting me; It's human nature to have wars and to only care for oneself.

I strongly disagree.

As for myself, I don't want that. Does this make me not a human? When we hear, read or see in the news what war crimes are committed, how people are forced to live (if they may live), what people are capable of doing to each other, we call this inhuman. What do we actually mean when we say so? Are the people committing these crimes not human people, or is it the act that is inhuman?

Again, my opponent argues that you should not call someone a liar, but rather say you lied - so to condemn the act of lying but not the person. Well, do we do that in other situations? May we call someone a murderer or a thief when they kill people or steal things? Is it a matter of frequency how often I eat meat while still calling myself a vegetarian?

Being human

The definition of what is or is not human may differ vastly. However, it is schizophenic that if we agree something is inhuman, to not take the consequences; As a human being stop acting inhuman!

Like being a vegetarian or not being a liar and murderer, for me being human is a continuous process that demands continuous work on ourselves. I have to work on myself to act how I want a human to act. When I think that the way meat is "produced" today on earth is inhuman, then I will have to change my diet. I have a vote, I can choose what to buy (or not to buy) or what to write on my blog. I can show people that I disagree and I can sit together with them, discuss differences of opinions and find a rational consent, because that's what I think how a human would act.

On a fun side note, my opponent also argued that animals also fight themselves, so this is just natural. Well, be an animal then.

by Robin Schubert at September 17, 2021 12:00 AM

September 14, 2021

Darshna Das

Docker – just a glimpse!

Earlier I had written a blog post about kubernetes and containers, but as days passed I had to get accustomed to my new job life and hence I thought of giving a month to get habituated to this new routine. Over this month I realised that I should now make my schedule in such a way where I have to keep progressing my about Kubernetes at the same time. Hence, after a month of observation that docker is also same as Kubernetes but where docker runs on a single cluster Kubernetes runs on a cluster. While I started learning Kubernetes because I had no idea how these tools are used I got very bored and eventually the interest died. But I wanted know about containers and how people are using them, so I started from again, smaller steps but longer run.

Dockers are also containerized environment where one can pull an image and their dependencies and libraries. It helps in solving the compatibility issue of different applications with respect to OS, libraries and dependencies. But in Linux kernel one can run only Linux containers that is one can run Linux softwares having different flavors of OS but one cannot run windows container on Linux kernel.

The difference between containers and virtual machines is that a Virtual machine consists of hypervisor and on top of sits different Virtual machines which has their own operating system and hypervisors which makes the system heavy due to high storage capacity. It also takes up minutes to boot up whereas Dockers on the other hand just sits on top of OS and creates different environments a.k.a containers and the application can have it’s own libraries and dependencies plus it takes less storage capacity compared to VM. Containers are less isolated that Virtual Machines as they rely on same OS or kernel. One can pull down any required image from docker hub which is a public registry.

The difference between Image and containers is that an image is a package or a template which is used to create containers and the containers are running image instances that are isolated and have their own environments.

by climoiselle at September 14, 2021 05:54 PM

September 13, 2021

Darshna Das

Last 9 days(100 days of journey)

Day 32 – Watched another detailed video about Regex module and the concept was pretty much clear. No code done but just read about regex.

Day 33 – Solved one code using class and sub-class. It was simple and I understood it.

Day 34 – Started watching videos on OOPS series, and got to know about class instance and did 2 codes using class to calculate simple area and parameters. This challenge helped me atleast to move further and reach to OOPS concept. Earlier I could not maintain the practice and hence these topics seemed so far and untouchable.

Day 35 – An immense hectic day, went to college and got very very tired. No coding done!

Day 36 – Solved a question using class and sub-class, watched a video first and got hold of the concept and also learnt about static method and class method.

Day 37 – Happy Ganesh Chaturthi, no coding done but started contribution to Kubernetes docs.

Day 38 – Learnt about inheritance, watched a video and did a code using inheritance, but an error popped up, did second code raise an exception error read about.

Day 39 – A friend helped me out with the error, seemed like I had a doubt for argument passing, reviewed the code but did not completed the code.

Day 40 – Solved the issue, pushed the solutions into the PR and also got some PDFs on OOPS concept. Did another code on try and except block and watched a video regarding it.

by climoiselle at September 13, 2021 04:10 PM

September 12, 2021

Sanyam Khurana

The Grit To Get Out

This is the first post on my blog in a while. I guess this is coming after almost 2 years 9 months. Yes, never wrote those end-of-year review posts too.

A lot has happened.

  • I left playing guitar at the end of 2018.
  • I didn't work on my book after …

by Sanyam Khurana at September 12, 2021 01:18 PM

Jason Braganza

On Resilience, Philosophy and Creative Success

This post was first sent to my newsletter on September 5th, 2021.
You really ought to subscribe :)


small flower banks of a river in the desert in Ladakh

A small bloom, defying the ravages of the wind and the desert in Ladakh

Read more… (2 min remaining to read)

by Mario Jason Braganza at September 12, 2021 12:15 AM

September 03, 2021

Kushal Das

Default values, documentation and Ansible

While testing my qubes_ansible project on the upcoming Qubes OS 4.1 project, I noticed something really strange. But, before getting into that, this Ansible module and the connection plugin are for Qubes OS only, and based on the excellent Python modules provided by the Qubes team.

The error goes like this during the fact gathering steps (reformatted for the blog):

fatal: [debian-10]: UNREACHABLE! => {
    "changed": false,
    "msg": "Failed to create temporary directory.In some cases, you may have
    been able to authenticate and did not have permissions on the target
    directory. Consider changing the remote tmp path in ansible.cfg to a path
    rooted in \"/tmp\", for more error information use -vvv. Failed command
    was: ( umask 77 && mkdir -p \"` echo ~The *user* is the default user in
    Qubes./.ansible/tmp `\"&& mkdir \"` echo ~The *user* is the default user in
    Qubes./.ansible/tmp/ansible-tmp-1630548982.9355698-7707-90110802425258 `\"
    && echo ansible-tmp-1630548982.9355698-7707-90110802425258=\"` echo ~The
    *user* is the default user in
    Qubes./.ansible/tmp/ansible-tmp-1630548982.9355698-7707-90110802425258 `\"
    ), exited with result 1, stderr output: mkdir: cannot create directory
    ‘~The ~The *user* account as default in Qubes OS. ~The ~The *user* account
    as default in Qubes OS. account as default in Qubes OS. ~The ~The *user*
    account as default in Qubes OS. ~The ~The *user* account as default in
    Qubes OS. account as default in Qubes OS. account as default in Qubes OS.
    is the default user in Qubes.’: File name too long\n",
    "unreachable": true
}

Most important part is the default user's home directory part, echo ~The user is the default user in Qubes./.ansible/tmp. For a moment I totally freaked out, as this looks like documentation. After reading the code more, I can see it is coming from the DOCUMENTATION variable in my plugin. After playing around a bit more and trying out different values I can see that the default value mentioned in the documentation is becoming the default value in the Python code.

After searching more I can see that the Ansible developers want the documentation string to be the gold standard and the code is parsing it find the default values. In my mind this is more confusing. I would expect the default value to be declared inside of the code.

Parsing the DOCUMENTATION and then finding the default values there in a Python code still does not fit in my brain. Fixed the issue for now, let me see what other surprises are waiting in the future.

September 03, 2021 02:21 AM

Priyanka Saggu

Update: Kubernetes 1.23 Release Enhancements Shadow! #32

Sept 3, 2021

👋 A very quick update!

I’m serving as a shadow on the Enhancements team for the current Kubernetes 1.23 release.

enhancements shadow

Most of the work for Enhancements team, i.e shepherding features for the current Kubernetes release cycle and maintaining the status of Kubernetes Enhancements Proposals or KEP(s), happens right at the beginning of the release.

As of today, we’re already 2 weeks into the current release cycle. The Enhancements team has now reached out to each SIG, to remind everyone of the opt-in process, the deadline, and confirming which enhancements are currently being tracked for each SIG.

If you want to read what all Enhancements KEP(s) are being tracked under the Kubernetes 1.23 release, please check the current Enhancements tracking sheet: https://bit.ly/k8s123-enhancements


To folks who’re interested in learning how Kubernetes releases happens and what goes on behind all the hard work, please consider reading the Release Team Role Handbooks.

You may also want to read about the Release Team Shadow Program.

The Release Team recruits a number of apprentices for each release role, in order to train new release team leads, handle tasks that each lead may not be able to cover, share knowledge about the release process, and help contributors broaden their areas of knowledge and participation.

September 03, 2021 12:00 AM

September 01, 2021

Sandeep Choudhary

Ansible: Roles

Ansible's roles allow grouping the content, which can be reused and easily share with other users. The role consists of vars, files, tasks, and other artifacts in a standard directory structure which have these subdirectories.

  • Tasks
  • Handlers
  • Library
  • Default
  • Vars
  • Files
  • Templates
  • Meta

We can use roles in the playbook or task with the help of Include and Import. To know more about Import/Include, you can check here

  • At Playbook level with the roles
  • At task level with include_roles
  • At task level with import_roles

Playbook Roles at playbook level are imported statically and executes the playbook in this order

  • Pre tasks
  • Roles listed in the playbook
  • Tasks define in the playbook and any handler triggered by the tasks
  • Post tasks define after the role
---
- hosts: webservers
  roles:
    - aws

Task with Include

---
- hosts: all
  tasks:
    - name: Use a dynamic roles
       include_roles:
           name: role_name  

Task with Import

---
- hosts: all
  tasks:
    - name: Use a static roles
       import_role:
           name: role_name  

Passing parameter to the roles

---
- hosts: all
  roles:
    - { role: aws, message: "ec2" }
    - { role: aws, message: "s3" }

Conditionally adding roles

---
- hosts: all
  tasks:
    - name: Include the some_role role
      include_role:
        name: some_role
      when: "ansible_facts['os_family'] == 'Ubuntu'"

Ansible's roles are really helpful to group the similar content and use in the different playbook and give the feasibility to the user to share with other which saves a lot of work. It's worth using them and shares your roles in the community to help others.

Cheers!

#100DaysToOffload #Ansible

September 01, 2021 05:16 AM

August 31, 2021

Jason Braganza

This is Water

My idols have feet of clay, as usual. The book is one of those, do as I say, not as I do books. I learnt about his abusive behaviour about a month after reading the book. While I absolutely loved reading it, I cannot in good conscience suggest you buy it anymore. Hopefully the clip I linked to, should be enough, because what he did share in the talk, is worth emulating, even if he himself, didn’t.

At some point in my mid twenties, before I found Stoicism, I somehow managed to start looking at things in my life, from other points of view.
And I was wondering how I got to be so “adult”, at least in that sense, when I did the rest of my growing up and adulting in my mid thirties. It was not the Catholic faith that I was brought up in, nor had I found the Stoic sensibilities that I’d stumble across much later in life.
I found the answer today as I was cleaning out my book shelves.


Read more… (2 min remaining to read)

by Mario Jason Braganza at August 31, 2021 12:15 AM

August 30, 2021

Sandeep Choudhary

Ansible: Import & Include

In Ansible playbooks are a collection of different tasks. It's a good idea to break the tasks into the different files, which make it easier to include/import these task in different playbooks. Now the question is when to use include or import and how these two are different from each other?

Ansible provides four distributed – Variables – Task – Playbook – Role

Include

Including variables, tasks, or role adds them into the playbook dynamically. This means when Ansible processes these files as they come up they are included in the current playbook as its variable, task, or role. So, these can be affected by the previous tasks.

Playbook's can not be used with the include

Import

Importing task, playbook, or role add them into playbook statically. Ansible pre-processes these files before it runs any task in the playbook, which means these are not affected by other tasks.

Tip: Import variables if you want to use these more than once in the playbook.

Import and Include differ with the way Ansible loads these files into the playbook. So, it is good to use the one which best fits your use case.

Cheers!

#100DaysToOffload #Ansible

August 30, 2021 05:26 AM

August 22, 2021

Bhavin Gandhi

Building RepRap 3D printer

My younger brother goes by the name HemRobotics almost everywhere. I have been working with him on a 3D printer build. We completed the first build of our RepRap machine based on Prusa i3 printer. I will be talking about our experience and learnings from this project. Why did we build a 3D printer? It’s a hacker’s dream to have a 3D printer on their desk. I had seen 3D printers on television when I was in school.

by Bhavin Gandhi (bhavin192@removethis.geeksocket.in) at August 22, 2021 12:05 PM

August 17, 2021

Priyanka Saggu

How to add a new check in KubeLinter? #31

Aug 17, 2021

Contents

Introduction

The following document demonstrates the steps followed while adding a new validation check in the upstream kube-linter project.

KubeLinter is a static analysis tool that checks Kubernetes YAML files and Helm charts to ensure the applications represented in them adhere to best practices.

kube-linter GitHub Project

This document uses the static check latest-tag, as an example to lay down the steps written below.

The latest-tag validation check indicates when a deployment-like kubernetes object (i.e. deployments, statefulsets, etc) is running a container with an invalid container image (by default, flags for a floating image tag like “latest”)

Steps to add a new check


[STEP 1] Setup the project on your local machine

  • Clone the project repository & change directory to the project root

    git clone git@github.com:stackrox/kube-linter.git
      
    cd kube-linter/
    

[STEP 2] Setup a new directory for the new validation check

  • From the root of the project, change directory to the pkg/templates/ folder

    cd pkg/templates
    
  • Create a new directory with the name corresponding to the new check (for example, in this case, latesttag)

    mkdir latesttag
      
    cd latesttag/
    

[STEP 3] Implement the logic for the new check under the new directory

  • Create the following files & folders in the path pkg/templates/latesttag/

    mkdir -p internal/params
      
    touch internal/prarms/params.go
      
    touch template.go template_test.go
    
  • The above files can be understood as (& referred from):

    • internal/params/params.go

      The internal/params/params.go file will contain the paramaters required for the test check.

    • template.go

      The actual implementation logic for the new test check will be written in the template.go file.

    • template_test.go

      And finally, the unit tests to validate the new check logic will be written in template_test.go file

[STEP 4] Make the new check visible to kube-linter

  • From the root of the project, modify the pkg/templates/all/all.go file to include the path to our new test, i.e. pkg/templates/latesttag

    vim pkg/templates/all/all.go
    
  • And modify the file to look like following:

    package all
    
    import (
      // Import all check templates.
      ...
      _ "golang.stackrox.io/kube-linter/pkg/templates/latesttag"
      ...
    )
    

[STEP 5] Include the new check as a kube-linter builtin check

  • From the root of the project, go to the pkg/builtin-checks/yamls folder & make a new yaml file corresponding to the name of the new check, (for example, latest-tag.yaml).

    vim latest-tag.yaml
    
  • And edit the file to look like following:

    name: "latest-tag"
    description: "Indicates when a deployment-like object is running a container with an invalid container image"
    remediation: "Use a container image with a proper image tag satisfying the \"AllowList\" & \"BlockList\" regex patterns."
    scope:
      objectKinds:
        - DeploymentLike
    template: "latest-tag"
    params:
      BlockList: [".*:(latest)$"]
      AllowList: []
    
  • The latest-tag.yaml file does the following:

    • provide self-explanatory string values for name, description & remediation fields
    • define the scope of the check, using the following block

      scope:
        objectKinds:
          - DeploymentLike
      
    • references the new test check template (defined at pkg/templates/latesttag/template.go), for example, template: "latest-tag"
    • provides default paramaeter values required by the check validation logic (defined at pkg/templates/latesttag/internal/params/params.go)

      params:
        BlockList: [".*:(latest)$"]
        AllowList: []
      

(Optional) [STEP 6] Include the new check as a kube-linter default built-in check

  • So far, the new check is just a builtin check, and is not a default check yet (i.e. kube-linter won’t run this check by default unless specified)

    • In order to add the new check as a kube-linter default check:

    Edit the internal/defaultchecks/defaultcheck.go file & add the new check name, to look like the following:

      package defaultchecks
    
      import (
        "golang.stackrox.io/kube-linter/internal/set"
      )
    
      var (
        // List is the list of built-in checks that are enabled by default.
        List = set.NewFrozenStringSet(
          ...
          "latest-tag",
          ...
        )
      )
    

Once everything above is done, the next step is to test the the newly added check!

Steps to verify the new check


[STEP 1] Build the kube-linter binary from the local project source code.

The local kube-linter binary will contain the new check latest-tag

  • Run the following command:

    make generated-srcs
    
  • And the output would look something like:

    ❯ make generated-srcs 
    go generate ./...
    Compiling Go source in ./cmd/kube-linter to bin/darwin/kube-linter
    Compiling Go source in ./cmd/kube-linter to bin/linux/kube-linter
    Compiling Go source in ./cmd/kube-linter to bin/windows/kube-linter.exe
    kube-linter templates list --format markdown > docs/generated/templates.md
    kube-linter checks list --format markdown > docs/generated/checks.md
    

    The above command generate local kube-linter binary (for linux, windows & macOS), along with all other auto-generated check templates & markdown documentation required by the kube-linter command line binary tooling to interact with.

[STEP 2] Create an example static yaml manifest

  • As the new check latest-tag (we’re adding as part of this documentation), is defined for a scope of DeploymentLike object, I’m creating an example deployment manifest yaml to do the testing.

  • At the root of the project, create an examples directory & put a deployment.yaml file under it (as our test deployment yaml manifest):

    mkdir examples
    
    cd examples/
    
    vim deployment.yaml
    

    Edit the deployment.yaml file to look like the following:

    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: nginx-deployment
      labels:
        app: nginx
    spec:
      replicas: 2
      selector:
        matchLabels:
          app: nginx
      template:
        metadata:
          labels:
            app: nginx
        spec:
          containers:
          - name: nginx
            image: example.com/test:v1.0.0
            ports:
            - containerPort: 80
          - name: frontend
            image: quay.io/django:latest
            ports:
            - containerPort: 8000
          - name: key-value-store
            image: docker.io/redis:v1.4.1
            ports:
            - containerPort: 6379
    

[STEP 3] Test the new check with the above example static yaml manifest

  • Execute the following command to run all the kube-linter checks on the test deployment.yaml manifest

    /bin/linux/kube-linter lint examples/deployment.yaml
    
  • In case, you want to run just the newly added latest-tag check, execute the following command:

    /bin/linux/kube-linter lint --do-not-include-builtin-checks --include latest-tag examples/deployment.yaml
    
  • And the output would look something like this:

    ❯ ./bin/linux/kube-linter lint /examples/deployment.yaml 
    KubeLinter 0.2.2-6-gc65dd74013-dirty
    
    examples/deployment.yaml: (object: <no namespace>/nginx-deployment apps/v1, Kind=Deployment) The container "nginx" is using an invalid container image, "example.com/test:v1.0.0". Please use images that satisfies the `AllowList` criteria : ["^(docker.io)"] (check: latest-tag, remediation: Use a container image with a proper image tag satisfying the "AllowList" & "BlockList" regex patterns.)
    
    examples/deployment.yaml: (object: <no namespace>/nginx-deployment apps/v1, Kind=Deployment) The container "frontend" is using an invalid container image, "quay.io/django:latest". Please use images that are not blocked by the `BlockList` criteria : [".*:(latest)$"] (check: latest-tag, remediation: Use a container image with a proper image tag satisfying the "AllowList" & "BlockList" regex patterns.)
    ...
    

[STEP 4] Verify the unit tests & the E2E (end-to-end) tests

  • Run the following command from the root of the project

    make test
      
    make e2e-test
    

[STEP 5] Lint the project

  • Run the following command from the root of the project to lint the project source code

    make lint
    

And finally, when you’re happy with how the new check performs, please raise a PR for the upstream kube-linter project & wait for a review! 🙂

August 17, 2021 12:00 AM

August 13, 2021

Anwesha Das

Finding and installing the security updates on Ubuntu

Keeping the system updated is a daily task of sys admins. To find out the available security updates on a Ubuntu machine use the commands here under. I have done this on a virtual machine Ubuntu 21.04.

apt update

This will update all the package information for the Debian repositories.

grep security /etc/apt/sources.list > /tmp/security.list

Getting the security repositories so we can only check in those. The apt software repositories are defined in the /etc/apt/sources.list file or in separate files under the /etc/apt/sources.list.d/ directory in
Ubuntu and all other Debian based distributions.

Finding and installing the security updates in Ubuntu

sudo apt list --upgradable -oDir::Etc::Sourcelist=/tmp/security.list
Listing... Done
libperl5.32/hirsute-security 5.32.1-3ubuntu2.1 amd64 [upgradable from: 5.32.1-3ubuntu2]
perl-base/hirsute-security 5.32.1-3ubuntu2.1 amd64 [upgradable from: 5.32.1-3ubuntu2]
perl-modules-5.32/hirsute-security 5.32.1-3ubuntu2.1 all [upgradable from: 5.32.1-3ubuntu2]
perl/hirsute-security 5.32.1-3ubuntu2.1 amd64 [upgradable from: 5.32.1-3ubuntu2]

Installing the security upgrades

The next step is to install the security upgrades.

sudo apt-get upgrade -oDir::Etc::Sourcelist=/tmp/security.list -s
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Calculating upgrade... Done
The following packages will be upgraded:
  libperl5.32 perl perl-base perl-modules-5.32
4 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Inst libperl5.32 [5.32.1-3ubuntu2] (5.32.1-3ubuntu2.1 Ubuntu:21.04/hirsute-security [amd64]) [perl:amd64 ]
Inst perl [5.32.1-3ubuntu2] (5.32.1-3ubuntu2.1 Ubuntu:21.04/hirsute-security [amd64]) []
Inst perl-base [5.32.1-3ubuntu2] (5.32.1-3ubuntu2.1 Ubuntu:21.04/hirsute-security [amd64]) []
Conf perl-base (5.32.1-3ubuntu2.1 Ubuntu:21.04/hirsute-security [amd64]) []
Inst perl-modules-5.32 [5.32.1-3ubuntu2] (5.32.1-3ubuntu2.1 Ubuntu:21.04/hirsute-security [all])
Conf libperl5.32 (5.32.1-3ubuntu2.1 Ubuntu:21.04/hirsute-security [amd64])
Conf perl (5.32.1-3ubuntu2.1 Ubuntu:21.04/hirsute-security [amd64])
Conf perl-modules-5.32 (5.32.1-3ubuntu2.1 Ubuntu:21.04/hirsute-security [all])

Stay safe and keep the system secured.

by Anwesha Das at August 13, 2021 02:01 PM

July 28, 2021

Anwesha Das

Upgrade CentOS 7 to CentOS 8 inplace

I wanted to upgrade one of my servers from CentOS 7 to CentOS 8. The following are the steps I followed :

Check the kernel version

Before we start the upgrade process, let us see the kernel version.

# uname -a
Linux centostest 3.10.0-1127.el7.x86_64 #1 SMP Tue Mar 31 23:36:51 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Installing basic packages

Now let us begin with installing the basic packages.

yum install epel-release -y

EPEL provides a set of additional packages for CentOS, RHEL from the Fedora sources.

yum install yum-utils rpmconf

yum-utils is a collection of utility tools and programs for managing yum repositories, installing different packages such as debug packages, source packages.

rpmconf searches for .rpmnew, .rpmsave, and .rpmorigfiles and asks the user what to do with them.

The following cammand will check the configuration files of all packages and ask users for input.

rpmconf -a

Install dnf package manager

dnf is the replacement for yum in the newer version of the OS.

yum install dnf

Remove yum

Remove yum, the default package manager for CentOS 7, and yum configuration file to avoid any conflict with dnf. Since for CentOS 8 dnf is the primary package manager.

dnf -y remove yum yum-metadata-parser

rm -Rf /etc/yum

Getting CentOS 8 release packages

We are all set to upgrade from CentOS 7 to CentOS 8, but we need to upgrade the system before that.

dnf upgrade

This will update packages to their latest versions that are both available and resolvable.
I have installed CentOS 8 release packages and also for EPEL.

dnf install http://mirror.centos.org/centos/8/BaseOS/x86_64/os/Packages/{centos-linux-repos-8-2.el8.noarch.rpm,centos-linux-release-8.4-1.2105.el8.noarch.rpm,centos-gpg-keys-8-2.el8.noarch.rpm}
dnf -y upgrade https://dl.fedoraproject.org/pub/epel/epel-release-latest-8.noarch.rpm
dnf clean all

Removed the old kernel core of CentOS 7 and conflicting packages along with it.

rpm -e `rpm -q kernel`
rpm -e --nodeps sysvinit-tools

CentOS 8 system upgrade

The following packages, dracut-network and rpmconf, conflicts upgrade process, therefore removed them.

dnf remove dracut-network rpmconf

With the following dnf command, we ask to download all the non deltarpms for CentOS 8.

dnf -y --releasever=8 --allowerasing --setopt=deltarpm=false distro-sync

Installing a new kernel

To get the new kernel for CentOS 8, run the following command.

dnf -y install kernel-core

The final step would be to install CenOS 8 minimal packages.

dnf -y groupupdate "Core" "Minimal Install"

After all done:

# cat /etc/os-release
NAME="CentOS Linux"
VERSION="8"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="8"
PLATFORM_ID="platform:el8"
PRETTY_NAME="CentOS Linux 8"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:8"
HOME_URL="https://centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"
CENTOS_MANTISBT_PROJECT="CentOS-8"
CENTOS_MANTISBT_PROJECT_VERSION="8"

Now run reboot to complete the upgrade.

Log back to the server, check the kernel version, and TaDa! We have CentOS 8 running on the system.

# uname -a
Linux centostest 4.18.0-305.10.2.el8_4.x86_64 #1 SMP Tue Jul 20 17:25:16 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Happy upgrading :)

by Anwesha Das at July 28, 2021 02:16 PM

July 21, 2021

Kushal Das

Trouble of zoom and participant name

Last night I was in a panel along with Juan Andrés Guerrero-Saade organized by Aveek Sen, the topic was "Tips on how journalists can avoid getting snooped". You can watch the recording at Youtube.

But this post is not about that. It is about Zoom. Just before logging into the call, I made sure that the name is changed while joining the call, generally my daughter uses the Zoom and her name was mentioned before. I personally have almost zero zoom usage (except 2-3 times in last 1 year). But, after logging into the call, zoom again went back to the older name, and did not allow me to change it during the session. I kept trying during the session without any luck. I don't know why did they do this or why I could not find a way to change my name, but I feel this is really stupid.

July 21, 2021 04:10 AM

July 18, 2021

Nabarun Pal

Giving Back

Over time, I have learnt a lot from the open-source software community, not just in India but from various parts of the world. I have the privilege of learning from the best software engineers in this world. A lot of great mentors have influenced me and helped me in shaping my way into OSS. These interactions were valuable to me, being a self-taught software engineer.

While talking to one junior from my alma mater, I realized there is still a gap between people wanting to get mentored and good mentors in the OSS community. There are outstanding programs like DGPLUG Summer Training which help new contributors learn how to survive in the open-source community by covering in breadth the mechanics. Still, people usually don’t discover on their own. I understand how frustrating it is to get stuck when you are exploring something. Having a mentor/partner with whom you can discuss doubts is vital in that situation.

As a way of giving back to the community that I have learned so much from, I am pledging 4 hours per week of my time to talk with folks who seek mentorship in OSS, Career journey, or, in general, talk about anything common interests.

Just go to https://calendly.com/palnabarun/1-on-1 to schedule a slot.

July 18, 2021 07:10 AM

June 24, 2021

Armageddon

Let's play with Traefik

I've been playing around with containers for a few years now. I find them very useful. If you host your own, like I do, you probably write a lot of nginx configurations, maybe apache.

If that's the case, then you have your own solution to get certificates. I'm also assuming that you are using let's encrypt with certbot or something.

Well, I didn't want to anymore. It was time to consolidate. Here comes Traefik.

Traefik

So Traefik is

an open-source Edge Router that makes publishing your services a fun and easy experience. It receives requests on behalf of your system and finds out which components are responsible for handling them.

Which made me realize, I still need nginx somewhere. We'll see when we get to it. Let's focus on Traefik.

Configuration

If you run a lot of containers and manage them, then you probably use docker-compose.

I'm still using version 2.3, I know I am due to an upgrade but I'm working on it slowly. It's a bigger project… One step at a time.

Let's start from the top, literally.

---
version: '2.3'

services:

Note

Upgrading to version 3.x of docker-compose requires the creation of network to link containers together. It's worth investing into, this is not a docker-compose tutorial.

Then comes the service.

traefik:
  container_name: traefik
  image: "traefik:latest"
  restart: unless-stopped
  mem_limit: 40m
  mem_reservation: 25m

and of course, who can forget the volume mounting.

volumes:
  - "/var/run/docker.sock:/var/run/docker.sock:ro"

Design

Now let's talk design to see how we're going to configuse this bad boy.

I want to Traefik to listen on ports 80 and 443 at a minimum to serve traffic. Let's do that.

command:
  - --entrypoints.web.address=:80
  - --entrypoints.websecure.address=:443

and let's not forget to map them.

ports:
  - "80:80"
  - "443:443"

Next, we would like to redirect http to https always.

- --entrypoints.web.http.redirections.entryPoint.to=websecure
- --entrypoints.web.http.redirections.entryPoint.scheme=https

We are using docker, so let's configure that as the provider.

- --providers.docker

We can set the log level.

- --log.level=INFO

If you want a dashboard, you have to enable it.

- --api.dashboard=true

And finally, if you're using Prometheus to scrape metrics… You have to enable that too.

- --metrics.prometheus=true

Let's Encrypt

Let's talk TLS. You want to serve encrypted traffic to users. You will need an SSL Certificate.

Your best bet is open source. Who are we kidding, you'd want to go with let's encrypt.

Let's configure acme to do just that. Get us certificates. In this example, we are going to be using Cloudflare.

- --certificatesresolvers.cloudflareresolver.acme.email=<your@email.here>
- --certificatesresolvers.cloudflareresolver.acme.dnschallenge.provider=cloudflare
- --certificatesresolvers.cloudflareresolver.acme.storage=./acme.json

warning

Let's Encrypt have set limits on how many certificates you can request per certain amount of time. To test your certificate request and renewal processes, use their staging infrastructure. It is made for such purpose.

Then we mount it, for persistence.

- "./traefik/acme.json:/acme.json"

Let's not forget to add our Cloudflare API credentials as environment variables for Traefik to use.

environment:
  - CLOUDFLARE_EMAIL=<your-cloudflare@email.here>
  - CLOUDFLARE_API_KEY=<your-api-key-goes-here>

Dashboard

Now let's configure Traefik a bit more with a bit of labeling.

First, we specify the host Traefik should listen for to service the dashboard.

labels:
  - "traefik.http.routers.dashboard-api.rule=Host(`dashboard.your-host.here`)"
  - "traefik.http.routers.dashboard-api.service=api@internal"

With a little bit of Traefik documentation searching and a lot of help from htpasswd, we can create a basicauth login to protect the dashboard from public use.

- "traefik.http.routers.dashboard-api.middlewares=dashboard-auth-user"
- "traefik.http.middlewares.dashboard-auth-user.basicauth.users=<user>:$$pws5$$rWsEfeUw9$$uV45uwsGeaPbu8RSexB9/"
- "traefik.http.routers.dashboard-api.tls.certresolver=cloudflareresolver"

Middleware

I'm not going to go into details about the middleware flags configured here but you're welcome to check the Traefik middleware docs.

- "traefik.http.middlewares.frame-deny.headers.framedeny=true"
- "traefik.http.middlewares.browser-xss-filter.headers.browserxssfilter=true"
- "traefik.http.middlewares.ssl-redirect.headers.sslredirect=true"

Full Configuration

Let's put everything together now.

traefik:
  container_name: traefik
  image: "traefik:latest"
  restart: unless-stopped
  mem_limit: 40m
  mem_reservation: 25m
  ports:
    - "80:80"
    - "443:443"
  command:
    - --entrypoints.web.address=:80
    - --entrypoints.websecure.address=:443
    - --entrypoints.web.http.redirections.entryPoint.to=websecure
    - --entrypoints.web.http.redirections.entryPoint.scheme=https
    - --providers.docker
    - --log.level=INFO
    - --api.dashboard=true
    - --metrics.prometheus=true
    - --certificatesresolvers.cloudflareresolver.acme.email=<your@email.here>
    - --certificatesresolvers.cloudflareresolver.acme.dnschallenge.provider=cloudflare
    - --certificatesresolvers.cloudflareresolver.acme.storage=./acme.json
  volumes:
    - "/var/run/docker.sock:/var/run/docker.sock:ro"
    - "./traefik/acme.json:/acme.json"
  environment:
    - CLOUDFLARE_EMAIL=<your-cloudflare@email.here>
    - CLOUDFLARE_API_KEY=<your-api-key-goes-here>
  labels:
    - "traefik.http.routers.dashboard-api.rule=Host(`dashboard.your-host.here`)"
    - "traefik.http.routers.dashboard-api.service=api@internal"
    - "traefik.http.routers.dashboard-api.middlewares=dashboard-auth-user"
    - "traefik.http.middlewares.dashboard-auth-user.basicauth.users=<user>:$$pws5$$rWsEfeUw9$$uV45uwsGeaPbu8RSexB9/"
    - "traefik.http.routers.dashboard-api.tls.certresolver=cloudflareresolver"
    - "traefik.http.middlewares.frame-deny.headers.framedeny=true"
    - "traefik.http.middlewares.browser-xss-filter.headers.browserxssfilter=true"
    - "traefik.http.middlewares.ssl-redirect.headers.sslredirect=true"

nginx

nginx pronounced

[engine x] is an HTTP and reverse proxy server, a mail proxy server, and a generic TCP/UDP proxy server, originally written by Igor Sysoev.

In this example, we're going to assume you have a static blog generated by a static blog generator of your choice and you would like to serve it for people to read it.

So let's do this quickly as there isn't much to tell except when it comes to labels.

nginx:
  container_name: nginx
  image: nginxinc/nginx-unprivileged:alpine
  restart: unless-stopped
  mem_limit: 8m
  command: ["nginx", "-enable-prometheus-metrics", "-g", "daemon off;"]
  volumes:
    - "./blog/:/usr/share/nginx/html/blog:ro"
    - "./nginx/default.conf.template:/etc/nginx/templates/default.conf.template:ro"
  environment:
    - NGINX_BLOG_PORT=80
    - NGINX_BLOG_HOST=<blog.your-host.here>

We are mounting the blog directory from our host to /usr/share/nginx/html/blog as read-only into the nginx container. We are also providing nginx with a template configuration and passing the variables as environment variables as you noticed. It is also mounted as read-only. The configuration template looks like the following, if you're wondering.

server {

    listen       ${NGINX_BLOG_PORT};
    server_name  localhost;

    root   /usr/share/nginx/html/${NGINX_BLOG_HOST};

    location / {
	index  index.html;
	try_files $uri $uri/ =404;
    }
}

Traefik configuration

So, Traefik configuration at this point is a little bit tricky for the first time.

First, we configure the host like we did before.

labels:
  - "traefik.http.routers.blog-http.rule=Host(`blog.your-host.here`)"

We tell Traefik about our service and the port to loadbalance on.

- "traefik.http.routers.blog-http.service=blog-http"
- "traefik.http.services.blog-http.loadbalancer.server.port=80"

We configure the middleware to use configuration defined in the Traefik middleware configuration section.

- "traefik.http.routers.blog-http.middlewares=blog-main"
- "traefik.http.middlewares.blog-main.chain.middlewares=frame-deny,browser-xss-filter,ssl-redirect"

Finally, we tell it about our resolver to generate an SSL Certificate.

- "traefik.http.routers.blog-http.tls.certresolver=cloudflareresolver"

Full Configuration

Let's put the nginx service together.

nginx:
  container_name: nginx
  image: nginxinc/nginx-unprivileged:alpine
  restart: unless-stopped
  mem_limit: 8m
  command: ["nginx", "-enable-prometheus-metrics", "-g", "daemon off;"]
  volumes:
    - "./blog/:/usr/share/nginx/html/blog:ro"
    - "./nginx/default.conf.template:/etc/nginx/templates/default.conf.template:ro"
  environment:
    - NGINX_BLOG_PORT=80
    - NGINX_BLOG_HOST=<blog.your-host.here>
  labels:
    - "traefik.http.routers.blog-http.rule=Host(`blog.your-host.here`)"
    - "traefik.http.routers.blog-http.service=blog-http"
    - "traefik.http.services.blog-http.loadbalancer.server.port=80"
    - "traefik.http.routers.blog-http.middlewares=blog-main"
    - "traefik.http.middlewares.blog-main.chain.middlewares=frame-deny,browser-xss-filter,ssl-redirect"
    - "traefik.http.routers.blog-http.tls.certresolver=cloudflareresolver"

Finale

It's finally time to put everything together !

---
version: '2.3'

services:

  traefik:
    container_name: traefik
    image: "traefik:latest"
    restart: unless-stopped
    mem_limit: 40m
    mem_reservation: 25m
    ports:
      - "80:80"
      - "443:443"
    command:
      - --entrypoints.web.address=:80
      - --entrypoints.websecure.address=:443
      - --entrypoints.web.http.redirections.entryPoint.to=websecure
      - --entrypoints.web.http.redirections.entryPoint.scheme=https
      - --providers.docker
      - --log.level=INFO
      - --api.dashboard=true
      - --metrics.prometheus=true
      - --certificatesresolvers.cloudflareresolver.acme.email=<your@email.here>
      - --certificatesresolvers.cloudflareresolver.acme.dnschallenge.provider=cloudflare
      - --certificatesresolvers.cloudflareresolver.acme.storage=./acme.json
    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock:ro"
      - "./traefik/acme.json:/acme.json"
    environment:
      - CLOUDFLARE_EMAIL=<your-cloudflare@email.here>
      - CLOUDFLARE_API_KEY=<your-api-key-goes-here>
    labels:
      - "traefik.http.routers.dashboard-api.rule=Host(`dashboard.your-host.here`)"
      - "traefik.http.routers.dashboard-api.service=api@internal"
      - "traefik.http.routers.dashboard-api.middlewares=dashboard-auth-user"
      - "traefik.http.middlewares.dashboard-auth-user.basicauth.users=<user>:$$pws5$$rWsEfeUw9$$uV45uwsGeaPbu8RSexB9/"
      - "traefik.http.routers.dashboard-api.tls.certresolver=cloudflareresolver"
      - "traefik.http.middlewares.frame-deny.headers.framedeny=true"
      - "traefik.http.middlewares.browser-xss-filter.headers.browserxssfilter=true"
      - "traefik.http.middlewares.ssl-redirect.headers.sslredirect=true"

  nginx:
    container_name: nginx
    image: nginxinc/nginx-unprivileged:alpine
    restart: unless-stopped
    mem_limit: 8m
    command: ["nginx", "-enable-prometheus-metrics", "-g", "daemon off;"]
    volumes:
      - "./blog/:/usr/share/nginx/html/blog:ro"
      - "./nginx/default.conf.template:/etc/nginx/templates/default.conf.template:ro"
    environment:
      - NGINX_BLOG_PORT=80
      - NGINX_BLOG_HOST=<blog.your-host.here>
    labels:
      - "traefik.http.routers.blog-http.rule=Host(`blog.your-host.here`)"
      - "traefik.http.routers.blog-http.service=blog-http"
      - "traefik.http.services.blog-http.loadbalancer.server.port=80"
      - "traefik.http.routers.blog-http.middlewares=blog-main"
      - "traefik.http.middlewares.blog-main.chain.middlewares=frame-deny,browser-xss-filter,ssl-redirect"
      - "traefik.http.routers.blog-http.tls.certresolver=cloudflareresolver"

Now we're all set to save it in a docker-compose.yaml file and

docker-compose up -d

If everything is configured correctly, your blog should pop-up momentarily. Enjoy !

by Elia el Lazkani at June 24, 2021 08:30 PM

June 21, 2021

Armageddon

Playing with containers and Tor

As my followers well know, by now, I am a tinkerer at heart. Why do I do things ? No one knows ! I don't even know.

All I know, all I can tell you is that I like to see what can I do with the tools I have at hand. How can I bend them to my will. Why, you may ask. The answer is a bit complicated; part of who I am, part of what I do as a DevOps. End line is, this time I was curious.

I went down a road that taught me so much more about containers, docker, docker-compose and even Linux itself.

The question I had was simple, can I run a container only through Tor running in another container?

Tor

I usually like to start topics that I haven't mentioned before with definitions. In this case, what is Tor, you may ask ?

What is Tor?

Tor is free software and an open network that helps you defend against traffic analysis, a form of network surveillance that threatens personal freedom and privacy, confidential business activities and relationships, and state security.

Although that home page is obscure because it was replaced by the new design of the website. Although I love what Tor has done with all the services they offer, don't get me wrong. But giving so much importance on the browser only and leaving the rest for dead when it comes to website, I have to say, I'm a bit sad.

Anyway, let's share the love for Tor and thank them for the beautiful project they offered humanity.

Now that we thanked them, let's abuse it.

Tor in a container

The task I set to discover relied on Tor being containerized. The first thing I do is, simply, not re-invent the wheel. Let's find out if someone already took that task.

With a litte bit of search, I found the dperson/torproxy docker image. It isn't ideal but I believe it is written to be rebuilt.

Can we run it ?

docker run -it -p 127.0.0.1:8118:8118 -d dperson/torproxy
curl -Lx http://localhost:8118 http://jsonip.com/

And this is definitely not your IP. Don't take my word for it! Go to http://jsonip.com/ in a browser and see for yourself.

Now that we know we can run Tor in a container effectively, let's kick it up a notch.

docker-compose

I will be testing and making changes as I go along. For this reason, it's a good idea to use docker-compose to do this.

Compose is a tool for defining and running multi-container Docker applications. With Compose, you use a YAML file to configure your application’s services. Then, with a single command, you create and start all the services from your configuration.

Now that we saw what the docker team has to say about docker-compose, let's go ahead and use it.

First, let's implement what we just ran ad-hoc in docker-compose.

---
version: '3.9'
services:
  torproxy:
    image: dperson/torproxy
    container_name: torproxy
    restart: unless-stopped

Air-gapped container

The next piece of the puzzle is to figure out if and how can we create an air-gapped container.

It turns out, we can create an internal network in docker that has no access to the internet.

First, the air-gapped container.

air-gapped:
  image: ubuntu
  container_name: air-gapped
  restart: unless-stopped
  command:
    - bash
    - -c
    - sleep infinity
  networks:
    - no-internet

Then comes the network.

networks:
  no-internet:
    driver: bridge
    internal: true

Let's put it all together in a docker-compose.yaml file and run it.

docker-compose up -d

Keep that terminal open, and let's put the hypothesis to the test and see if rises up to be a theory.

docker exec air-gapped apt-get update

Aaaaand…

Err:1 http://archive.ubuntu.com/ubuntu focal InRelease
  Temporary failure resolving 'archive.ubuntu.com'
Err:2 http://security.ubuntu.com/ubuntu focal-security InRelease
  Temporary failure resolving 'security.ubuntu.com'
Err:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease
  Temporary failure resolving 'archive.ubuntu.com'
Err:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease
  Temporary failure resolving 'archive.ubuntu.com'
Reading package lists...
W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/focal/InRelease  Temporary failure resolving 'archive.ubuntu.com'
W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/focal-updates/InRelease  Temporary failure resolving 'archive.ubuntu.com'
W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/focal-backports/InRelease  Temporary failure resolving 'archive.ubuntu.com'
W: Failed to fetch http://security.ubuntu.com/ubuntu/dists/focal-security/InRelease  Temporary failure resolving 'security.ubuntu.com'
W: Some index files failed to download. They have been ignored, or old ones used instead.

looks like it's real peeps, hooray !

Putting everything together

Okay, now let's put everything together. The list of changes we need to make are minimal. First, I will list them, then I will simply write them out in docker-compose.

  • Create an internet network for the Tor container
  • Attach the internet network to the Tor container
  • Attach the no-internet network to the Tor container so that our air-gapped container can access it.

Let's get to work.

---
version: '3.9'
services:

  torproxy:
    image: dperson/torproxy
    container_name: torproxy
    restart: unless-stopped
    networks:
      - no-internet
      - internet

  air-gapped:
    image: ubuntu
    container_name: air-gapped
    restart: unless-stopped
    command:
      - bash
      - -c
      - sleep infinity
    networks:
      - no-internet

networks:
  no-internet:
    driver: bridge
    internal: true
  internet:
    driver: bridge
    internal: false

Run everything.

docker-compose up -d

Yes, this will run it in the background and there is no need for you to open another terminal. It's always good to know both ways. Anyway, let's test.

let's exec into the container.

docker exec -it air-gapped bash

Then we configure apt to use our torproxy service.

echo 'Acquire::http::Proxy "http://torproxy:8118/";' > /etc/apt/apt.conf.d/proxy
echo "export HTTP_PROXY=http://torproxy:8118/" >> ~/.bashrc
echo "export HTTPS_PROXY=http://torproxy:8118/" >> ~/.bashrc
export HTTP_PROXY=http://torproxy:8118/
export HTTPS_PROXY=http://torproxy:8118/
apt-get update
apt-get upgrade -y
DEBIAN_FRONTEND=noninteractive apt-get install -y curl

Harvesting the fruits of our labour

First, we always check if everything is set correctly.

While inside the container, we check the environment variables.

env | grep HTTP

You should see.

HTTPS_PROXY=http://torproxy:8118/
HTTP_PROXY=http://torproxy:8118/

Then, we curl our IP.

curl https://jsonip.com/

And that is also not your IP.

It works !

Conclusion

Is it possible to route a container through another Tor container ?

The answer is obviously Yes and this is the way to do it. Enjoy.

by Elia el Lazkani at June 21, 2021 09:30 PM

June 10, 2021

Bhavin Gandhi

Removing Comments and Disqus

After moving away from Google Fonts, Google Analytics, I also removed Disqus comments from my website. Instead of implementing an alternative commenting system, I decided to remove it altogether. I will be covering why I did it, and how I’m showing the old comments from Disqus statically on the website. Why am I removing the comments? This was not a very easy decision to take. But finally I took the decision to remove the commenting system, and here are the reasons:

by Bhavin Gandhi (bhavin192@removethis.geeksocket.in) at June 10, 2021 02:23 PM

April 19, 2021

Saptak Sengupta

Opting out of Google FLoC network

Recently, Google announced their new ad-tracking and surveillance tool called Federated Learning of Cohorts (FLoC). This is a new alternative to the third-party cookie tracking that is otherwise widely used for advertising business.

EFF has written more about the issues with using Google FLoC and also created a website where you can test if you are already a victim of their FLoC tests.

Google will track any user visiting your website even if it doesn't have Google analytics or any other services related to Google. One easy way for users visiting websites to opt out of this is to not use Google Chrome and use browsers like Firefox, etc. However, website maintainers can also help against this new tracking technology by opting out of the FLoC network.

Permissions-Policy Header

So the main way of opting out of this surveillance technology is to add a HTTP response header to their websites.

The HTTP response header is

Permissions-Policy: interest-cohort()

The FLoC technology uses interest-cohort to check for an allowlist. By default, everything is allowed as long as the user is visiting from a browser which supports InterestCohort API. However, by mentioning interest-cohort() in the Permissions-Policy header, the website is opting out from allowing any origin (including the current page) from being tracked using FLoC. Hence the FLoC feature is turned off for the website, even if the user is visiting your website from a Google Chrome browser.

NGINX

To add the above header, go to your nginx configuration file and add the following inside the server block:

server {
    ...

    add_header Permissions-Policy "interest-cohort=()";
    
    ...
}

If you have different nginx confs for multiple websites running via nginx, you have to do the above in all the server blocks or nginx configuration files.

Then run nginx -t to test that everything is correct in your updated nginx configuration. Then, restart nginx by running the command service nginx restart (or any other command that you might use based on your OS to restart nginx)

If you are using any other web server or proxy server, you can check this link: https://paramdeo.com/blog/opting-your-website-out-of-googles-floc-network

April 19, 2021 10:26 AM

April 03, 2021

Pradyun Gedam

OSS Work update #10

I’m trying to post these roughly once a month. Here’s the Feb post.

Work I did (1 Mar 2021 - 31 Mar 2021)

Technical

  • Started writing a PEP 517 build backend for Sphinx themes.
  • Started writing sphinx-basic-ng, an attempt at modernising the Sphinx theme ecosystem.
  • Released new versions of sphinx-autobuild.
  • More updates to Furo.
  • Made progress on installer’s implementation.
  • Made progress on pip’s documentation rewrite.
  • Started work on rewriting ScriptTest, to pay down pip’s technical debt.

Communication

  • Talking to relevant folks about toml on PyPI (and moving it to… a specific GitHub repository).
  • Finally sat down and did an “open source responsibility audit”.
  • My open social weekends is still a thing, and has been great so far!
  • Still collaborating on designing a lockfile format for Python.
  • Added to The Executable Book Project’s GitHub organisation! ^>^
  • Chatted with a few people about the state of Python and pip on Debian.

General notes

Looking back, I think I picked up a couple of new projects based on random brain waves I had! That’s perfectly timed, because I’ve decided to pivot away from my earlier approach of “yay, more responsibility!”.

What next?

This is getting increasingly harder to decide on, as my free time chunks are becoming smaller and I’m picking up bigger projects. :)

Technical

  • Sphinx Theme PEP 517 stuff: Make the initial release.
  • sphinx-basic-ng: Make the first usable release.
  • pip: Clear some of the backlog on the pull request front.
  • pip: More progress on the documentation rewrite.

Communication

  • Spend more time looking into the Python lockfile standardisation effort.
  • Write a blog post, on automated code formatting.
  • Find more speaking opportunities, to talk about things that aren’t Python packaging!

Other commitments

A full time job, that pays my bills. :)

April 03, 2021 12:00 AM

March 03, 2021

Robin Schubert

How to make your phone battery last for one week and longer with this simple trick

Sure, it sounds like a click-bait headline, but I'm serious. I've bought my Android phone, a Google Nexus X5, second-hand from ebay. It's a great device, popular with developers but it's not known for its long lasting battery. In fact, I had to charge my phone at least once a day, if not more often.

But not anymore. During the last three weeks I charged it twice, no more. I didn't just turn it off. What I did is simple: I stopped using it. I turned off the WiFi, the mobile data connection and as a result I don't have much reason to look at it very often. Sometimes not even once a day. I do get phone calls from time to time, and I even play some puzzle games while I brush my teeth, but that's it.

Why no smart phone?

I would say I was an average smart phone user. I do some of that social networking thingies, message with friends and family and if I don't know the answer to a random question I ask duckduckgo. I've not been a poweruser, I don't have a zillion followers on any platform, and I don't take pictures of me or my food or my cats (don't have any) all the time. But I did use my phone regularly to check my email, my calendar or via termux work remotely on several Linux servers.

I cannot give any reliable numbers about the amount of time I've spent staring on my smart phone, but I can tell that I took it everywhere. I took it to the toilet to read my mastodon timeline and I took it with me when I went to bed to read my RSS feeds. I had it lying next to me on the table during meals or at work, so I could peek at incoming messages.

After a couple of weeks without using my smart phone now, I can tell that I miss none of that. I integrated some things into a more or less regular schedule, like checking feeds and mastodon on the laptop only once a day before I go to bed. It saves a lot of time and I never had the feeling I would miss anything. However, I find much more time to read books or play some guitar now, which I enjoy very much.

I spending a huge amount of my day in front of a screen anyway, due to my job and hobbies, but I feel that dropping the phone is absolutely beneficial for my health and also I could be the better role model for my children.

How no smart phone?

That leaves us with the question: What apps/services to drop and how to replace those that I'm not willing to drop?

The obvious

The easiest part was to port the most phone actions to the laptop. I did read mails on the phone a lot, though I didn't reply very often. Sometimes I just flagged some as important and left the work for later. Doing email on my laptop only resulted in more consistency and structure for me, and additionally saved quite some time and stress in my off-work time.

Randomly browsing the web or looking up stuff on Wikipedia, just like reading feeds and timelines was also easy to replace and led to more structure and much less distraction. Interestingly, I don't think that I'm spending more time on the laptop now, but I never took any measure to verify that ;)

I used my phone as second factor for some two-factor authentication logins. For now I also replaced that with the OTP module of my password manager pass. It still is a valid second factor, as I need the laptop additionally to the password, however, I think I will replace that with a security key in future.

I still enjoy Spotify sometimes. To make that work without my phone, I installed spotifyd on my raspberry pi, which I use as media center. The daemon just serves as the player, while I'm controlling the playlist from my laptop (using a beautiful TUI written in Rust, called spt).

The messaging

I'm using Signal as my main messaging app, but also joined some Matrix rooms on different servers and even follow some Mattermost conversations. Since I never seriously used IRC on the phone, I instead tried to integrate the messaging to my favorite IRC client weechat, with quite decent success.

I'm running signald on my weechat server and fixed some on the signal-weechat plugin and message happily ever since. Signald is a java app that subscribes to my phone number and provides a socket that can be used to interact with the API. Is it as secure as the official Signal app? Nope. But that's not my threat model ;) It's way better than the crappy electron app, in my opinion.

After I configured that, I realized that there are also plugins available for matrix and Mattermost (the latter of which is in pre-alpha, but hey). Using weechat for all my messaging feels great and to me is a big improvement over the phone (and over web- and electron clients). I know that there may be security drawbacks, not in transportation but in the way data is stored on my server. Luckily I can address these issues and I can do that independent of any app.

The rest

I thought that I could just drop everything that I cannot port to laptop. However, I hardly found anything I had to give up. I still use the phone to take pictures sometimes (never took many) and use the audio recorder occasionally. I play less random games.

The only thing I miss is the calendar functionality. I do have an excellent calendar on the laptop, or course (khal). But sometimes I would like to quickly glance at my upcoming events or schedule an appointment without being on the laptop. I will need to think how to do that without going fully analog on this one again. I'm having an eye on the Mudita Pure which I think would support me in exactly the way I chose to take.

by Robin Schubert at March 03, 2021 12:00 AM

February 27, 2021

Pradyun Gedam

OSS Work update #9

Alrighty! Let’s start doing this again. The plan is to get back to doing these roughly once a month again.

Work I did (1 Jan 2021 - 26 Feb 2021)

Technical

  • Published the last version of pip that supports Python 2! 🎉
  • Published a few releases of Furo – a Sphinx theme I wrote.
  • Made some (unreleased) changes to sphinx-autobuild.
  • Made some more progress on installer – a reusable library for Python (wheel) package installation.
  • Rewrote get-pip.py and the generation pipeline for it.
  • Started a rewrite of pip’s documentation. I’d love to get some feedback on this.

Communication

  • I’m experimenting with a new thing: social weekends!
  • I presented 2 talks at FOSDEM: in the Python devroom and Open Source Design devroom. Shout-out to Bernard Tyers, for all the help and the bazillion reminders to make sure I do all the things on time. :)
  • Collaborating on designing a lockfile format for Python, that can hopefully be standardised for interoperability.

General notes

Onboarding in a new company, relocating internationally, settling into a new space has been… well, it’s all been a very interesting learning experience.

Given the fairly strict lockdown and the percentage of people wearing masks in my locality, I’ve spent a lots of time indoors. Looking forward to the social weekends experiment I’m doing.

What next?

Technical

  • pip: Work on the documentation rewrite, hopefully to get it ready in time for the next release.
  • pip: Clear some of the backlog on the pull request front.
  • pip: General discussions for new features and enhancements.
  • TOML: Work on writing that the compliance test suite.
  • TOML: Bring toml for Python back from the dead.
  • Furo: Make the first stable release.
  • Start work on the other Sphinx theme I have in mind.

Communication

  • Spend more time looking into the Python lockfile standardisation effort.
  • Catch up on the Python-on-Debian saga, and see how I can contribute constructively.

Other commitments

Oh, I have a full time job at Bloomberg now. :)

February 27, 2021 12:00 AM

February 26, 2021

Saptak Sengupta

Anonymous Chat using OnionShare

Let's dive in a little deeper into the feature.

Why do we need an anonymous chat?

A common question that we got during developing this feature is what's the use of an anonymous chat room since we already have end-to-end encrypted messaging apps. It leaves a lot fewer traces.

The way we achieve this is very simple. There is no form of storage whatsoever in OnionShare chat mode. The chat is not persistent. The chat server stores no information at all (not even the usernames of people chatting). So once the chat server is closed, and the Tor Browser tab with the chat client is closed, there is no data (or metadata) related to chat that remains, even in the person's system who started the server. Hence, it leaves much less trace compared to other chat applications.

A good example of the above as mentioned by Micah in his blog is:

If, for example, you send a message to a Signal group, a copy of your message ends up on each device (the devices, and computers if they set up Signal Desktop of each member of the group). Even if disappearing messages is turned on it’s hard to confirm all copies of the messages are actually deleted from all devices, and from any other places (like notifications databases) they may have been saved to. OnionShare chat rooms don’t store any messages anywhere, so the problem is reduced to a minimum.

Given that the OnionShare chat feature works over the onion network, so it also has the additional anonymity feature. Also, adding to the anonymity feature, OnionShare chat doesn't need any form of signing in. Hence, people chatting can stay anonymous, and everything happens inside the tor network. One can just start a chat server, share the link via some disposable way, and then wait for the other people to join while maintaining anonymity.

Because it's an onion service, there is no way for an attacker to eavesdrop on the messages. The closest they can get is if they run a malicious Tor rendezvous node that's chosen for the onion service, they'll be able to spy on encrypted onion traffic. So, there's no capturing ciphertext to decrypt later on.

So what happens under the hood?

The chat feature is dependent on flask-socketio and eventlet for the WebSocket server implementation, and socket.io client js for the frontend implementation of the chat client. So when a chat server is started, the WebSocket is started in a namespace "/chat". Whenever a new user joins the link, they are given a randomly generated username and they are added to the room "default". There is only one room, and the actual name of the room can be set from the OnionShare settings-related code, but it doesn't really impact anything in the implementation. Both the room name and the randomly generated username are stored in a flask session. But that information is also completely gone once the chat server is stopped. The room and username information are only there to emit the messages properly.

You can also change the randomly generated username to a username (or pseudo username) of your choice for that particular session.

There are two main types of messages:

  1. status messages - these are sent from the client to the server only when a new user joins or someone updates their username. The status message is then broadcasted to all the other connected clients, who will then see it as a form of a status message in the chat window.

Onionshare Chat window with status messages for user joining and changing username

  1. user messages - these are sent when a user sends a message. All messages are broadcasted, so in case you share the link to multiple users, there is no concept of private message and everyone connected to the room can view your messages. Hence, sharing the onion link securely is important.

Onionshare Chat window with status messages for user joining and changing username

All of these WebSocket communication happens over the Tor onion services. OnionShare in itself doesn't implement any encryption algorithm to the chat and heavily relies on the Tor onion service's encryptions for the same. The message from the client to the OnionShare server is E2EE as it goes via Tor's onion connection. Then the OnionShare server broadcasts the message to all the other clients connected to the chat room through their E2EE onion connection, over WebSockets.

So what now?

I feel, as of now, the OnionShare anonymous chat is great if you quickly want to have an anonymous, secure, non-persistent conversation with someone or a group of people. It is also great if a whistleblower wants to share some details over chat with a journalist and then remove all traces of that conversation completely. But I feel if someone needs to host a chat server for a long time where people can connect anonymously, this is probably not the best solution for that.

There are still some issues that we will hopefully improve in the next releases. Firstly, we need to try and make it a bit more asynchronous. Right now, if left inactive for a long time, the Tor connection over WebSocket is sometimes dropped which isn't a great user experience. We are working on improving that.

Also, we will improve the UI for the chat more to give a better user experience.

With the new tabs feature, one can have all different modes (even multiple servers of same mode) running at the same time. So you can have a chat server and share mode server running at the same time. All the modes are very independent of each other and hence don't affect one another in any way.

I hope you all enjoy the new chat feature and leave feedbacks/suggestions on how to improve it. You can also read more about this and other features at docs.onionshare.org

February 26, 2021 09:52 AM

February 21, 2021

Sayan Chowdhury

Love Boredom

I was talking to Jason today, discussing aspects of life and building habits. Last week was pretty rough, devoid of motivation to maintain my habits. To which he said to me, “to fall in love with boredom” and passed me a copule post from James Clear. How to Fall in Love With Boredom and Unlock Your Mental Toughness How to Stay Focused When You Get Bored Working Toward Your Goals Lessons on Success and Deliberate Practice from Mozart, Picasso, and Kobe Bryant The first blog was one I could relate to in an instant.

February 21, 2021 06:30 PM

September 10, 2020

Nabarun Pal

My journey in the Kubernetes Release Team: Part 1

During this period last year, I got interested in how a new Kubernetes version is released and what goes on behind it. After some searching, I found that all of the process and the roles are well documented in the Release Team Role Handbooks.

rt-handbooks-repo

I read through all of them to understand the process, why there are so many roles and what responsibilities each team is entrusted with. All of that sounded pretty interesting to me. All the teams did an amazing work and were equally crucial for the well functioning of a Kubernetes Release cycle. I got specifically interested with the Enhancements and CI Signal teams. I started to dig how can I lend my hand to the efforts.

Shadow Roles

With the role handbooks, I got to know of the Release Team Shadow Program which is aimed at mentoring new contributors and training them to be the next leads of the Kubernetes Release Team. The shadows are expected to learn from the leads and fill in wherever necessary. You can think of these positions as “trainee/intern” roles at your workplace. This was just a primer on the program. You can read more on the link.

Okay. I know where to start with. But how?

Turns out it requires a marginal amount of effort, bucket loads of curiosity and time commitment to apply for the Shadow Program. The Release Team at the start of every release cycle pushes out a public form for inviting applications to the shadow roles.

Taking the plunge

I took the initiative and filled the form with my interests and thoughts. And few days later I, along with other shadows, was welcomed by MrBobbyTables to my first involvement with the Release Team. 🎉

1.17 Introduction

The next few months were just like a roller coaster ride. The team that I was shadowing for was the Enhancements Team and our work was to shepherd features for the Kubernetes release and maintain the status of Kubernetes Enhancements Proposals, aka, KEP(s).

The role involved understanding each outstanding KEP, pinging respective OWNERS if the enhancement would be graduating in the current release cycle, and keeping track whether the enhancements are satisfying the requirements for the release.

Here were a few takeaways that I took while working on the team:

  • Knowledge of what is involved when adding a feature into Kubernetes!!!
  • Reading through an enourmous amount of KEPs, I got to know about the features themselves
  • Communicating effectively with others and breaking the ice
  • A lot of GitHub triage skills and tricks
  • Tricks of wrangling data on a spreadsheet 😉

I will write about the complete lifecycle of a Kubernetes Enhancement Proposal in a future article.

We released Kubernetes 1.17 : The Chillest Release (Yes! That is the release theme 😃) after all the efforts of the 1.17 release team. The last release of the year is usually the most chilled out and is a bit short due to the December vacation.

The whole team became akin to a family for me spending all that effort into ensuring a smooth release. We, the Enhancements Team (Bob, Jeremy, Anna, Kristin), even got together at KubeCon San Diego to meet up physically.

1.17-rt-meet

The Next Steps

After Kubernetes 1.17, I signed up again for the Kubernetes 1.18 team for Enhancements to get more exposure to the KEP landscape. This time Jeremy was leading the Enhancements Team. It was fun again to work with the enhancements team (Jeremy, Kirsten, Heba, John), all the same just this time there were new shadows along with me compared to last time.

This release cycle was mostly the same for me other than I was a bit more involved than the last time having served on the team previously. And, after all the hard work of the 1.18 release team, we were treated to a quarky Kubernetes 1.18 ⭐.

Graduating to be the Enhancements Lead 🚀

Aaaaaand after a splendid 1.18, Jeremy nominated me to be the Enhancements Lead for Kubernetes 1.19 Release Team. I was stoked to get the opportunity and at the same time scared if I can do full justice to the responsibility bestowed upon me. The knowledge of the role while working with Bob and Jeremy in the previous release teams gave me the confidence that I can fulfill the responsibilities of the role.

1.19 Nomination

This release cycle eventually became special in many ways. We were hit by a deadly pandemic which changed a lot of things in our life. The release cycle was extended to 5 months instead of the usual 12 weeks cadence. The pandemic and various other factors shaved off quite a bit of the usual bandwidth the community had previously. These times were very crucial for the whole world and the team didn’t want to put undue additional pressure on the amazing contributors that we have.

The shadows that I selected for the Enhancements Team spanned 12 and a half hours of timezone and created an amazing round the earth coverage for the team. This meant no team member had to toil in their odd times of the day. I take this opportunity to thank again all the enhancements shadows - Kirsten, Harsha, Miroslaw and John for their efforts even in such hard times.

After those tense & tough 5 months firefighting a lot of issues, the 1.19 release team released Kubernetes 1.19: Accentuate the Paw-sitive. You can read about the release on the Kubernetes Blog and the upcoming Kubernetes 1.19 Release Webinar where I will be presenting along with the 1.19 Release Lead Taylor and the 1.19 Communications Lead Max.

With that, my watch ended over the Release Enhancements Team and it was time to hand over the baton to next lead. ☮

I was very happy to nominate Kirsten to succeed me as the next Enhancements Lead of the Release Team. 🎉

1.20 Nomination

Along with that it was time for me to graduate to my next role. I look forward to working with Jeremy, Savitha and Daniel for Kubernetes 1.20. I will be shadowing the Release Lead for Kubernetes 1.20. 😎

1.20 Onboarding

How can you get involved? 🙌

The last release of the year, Kubernetes 1.20, is going to be published in December. The Release Team is looking out for folks for the shadow roles. All you need to do is go ahead, read the role handbooks, figure out which role interests you the best and then fill the form. We ask every prospect to fill the form because we want to know if the release team would be a good fit for you and to find the right role for you in the Release Team.

The applications will be open until End of Day Friday, September 11, 2020 Pacific Time.

Still in doubt? 🔍

I would say just go ahead and volunteer for the shadow roles. 🚢

Feel free to contact me on Twitter at @theonlynabarun, or on the Kubernetes Slack, in case you have anything to ask.

Quick References


Update since the original version

I wrote this article over 6 months back and since then led the Kubernetes 1.21 Release Team. I plan to write about my experience leading the Release Team in the near future. Do subscribe to the RSS feed for updates.

September 10, 2020 03:28 AM

August 23, 2020

Abhilash Raj

Concurrency bugs in Go

I recently read this paper titled, Understanding Real-World Concurrency Bugs in Go (PDF), that studies concurrency bugs in Golang and comments on the new primitives for messages passing that the language is often known for.

I am not a very good Go programmer, so this was an informative lesson in various ways to achieve concurrency and synchronization between different threads of execution. It is also a good read for experienced Go developers as it points out some important gotchas to look out for when writing Go code. The fact that it uses real world examples from well known projects like Docker, Kubernetes, gRPC-Go, CockroachDB, BoltDB etc. makes it even more fun to read!

The authors analyzed a total of 171 concurrency bugs from several prominent Go open source projects and categorized them in two orthogonal dimensions, one each for the cause of the bug and the behavior. The cause is split between two major schools of concurrency

Along the cause dimension, we categorize bugs into those that are caused by misuse of shared memory and those caused by misuse of message passing

and the behavior dimension is similarly split into

we separate bugs into those that involve (any number of ) goroutines that cannot proceed (we call themblocking bugs) and those that do not involve any blocking (non-blocking bugs)

Interestingly, they chose the behavior to be blocking instead of deadlock since the former implies that atleast one thread of execution is blocked due to some concurrency bug, but the rest of them might continue execution, so it is not a deadlock situation.

Go has primitive shared memory protection mechanisms like Mutex, RWMutex etc. with a caveat

Write lock requests in Go have ahigher privilege than read lock requests.

as compared to pthread in C. Go also has a new primitive called sync.Once that can be used to guarantee that a function is executed only once. This can be useful in situations where some callable is shared across multiple threads of execution but it shouldn't be called more than once. Go also has sync.WaitGroups , which is similar to pthread_join to wait for various threads of executioun to finish executing.

Go also uses channels for the message passing between different threads of executions called Goroutunes. Channels can be buffered on un-buffered (default), the difference between them being that in a buffered channel the sender and receiver don't block on each other (until the buffered channel is full).

The study of the usage patterns of these concurrency primitives in various code bases along with the occurence of bugs in the codebase concluded that even though message passing was used at fewer places, it accounted for a larger number of bugs(58%).

Implication 1:With heavier usages of goroutines and newtypes of concurrency primitives, Go programs may potentiallyintroduce more concurrency bugs

Also, interesting to note is this observation in tha paper

Observation 5:All blocking bugs caused by message passing are related to Go’s new message passing semantics like channel. They can be difficult to detect especially when message passing operations are used together with other synchronization mechanisms

The authors also talk about various ways in which Go runtime can detect some of these concurrency bugs. Go runtime includes a deadlock detector which can detect when there are no goroutunes running in a thread, although, it cannot detect all the blocking bugs that authors found by manual inspection.

For shared memory bugs, Go also includes a data race detector which can be enbaled by adding -race option when building the program. It can find races in memory/data shared between multiple threads of execution and uses happened-before algorithm underneath to track objects and their lifecycle. Although, it can only detect a part of the bugs discovered by the authors, the patterns and classification in the paper can be leveraged to improve the detection and build more sophisticated checkers.

by Abhilash Raj at August 23, 2020 12:59 AM

August 08, 2020

Farhaan Bukhsh

Url Shortner in Golang

TLDR; Trying to learn new things I tried writing a URL shortner called shorty. This is a first draft and I am trying to approach it from first principle basis. Trying to break down everything to the simplest component.

I decided to write my own URL shortner and the reason for doing that was to dive a little more into golang and to learn more about systems. I have planned to not only document my learning but also find and point our different ways in which this application can be made scalable, resilient and robust.

A high level idea is to write a server which takes the big url and return me a short url for the same. I have one more requirement where I do want to provide a slug i.e a custom short url path for the same. So for some links like https://play.google.com/store/apps/details?id=me.farhaan.bubblefeed, I want to have a url like url.farhaan.me/linktray which is easy to remember and distribute.

The way I am thinking to implement this is by having two components, I want a CLI interface which talks to my Server. I don’t want a fancy UI for now because I want it to be exclusively be used through terminal. A Client-Server architecture, where my CLI client sends a request to the server with a URL and an optional slug. If a slug is present URL will have that slug in it and if it doesn’t it generates a random string and make the URL small. If you see from a higher level it’s not just a URL shortner but also a URL tagger.

The way a simple url shortner works:

Flow Diagram

A client makes a request to make a given URL short, server takes the URL and stores it to the database, server then generates a random string and maps the URL to the string and returns a URL like url.farhaan.me/<randomstring>.

Now when a client requests to url.farhaan.me/<randomstring>, it goest to the same server, it searches the original URL and redirects the request to a different website.

The slug implementation part is very straightforward, where given a word, I might have to search the database and if it is already present we raise an error but if it isn’t we add it in the database and return back the URL.

One optimization, since it’s just me who is going to use this, I can optimize my database to see if the long URL already exists and if it does then no need to create a new entry. But this should only happen in case of random string and not in case of slugs. Also this is a trade off between reducing the redundancy and latency of a request.

But when it comes to generating a random string, things get a tiny bit complicated. This generation of random strings, decides how many URLs you can store. There are various hashing algorithms that I can use to generate a string I can use md5, base10 or base64. I also need to make sure that it gives a unique hash and not repeated ones.

Unique hash can be maintained using a counter, the count either can be supplied from a different service which can help us to scale the system better or it can be internally generated, I have used database record number for the same.

If you look at this on a system design front. We are using the same Server to take the request and generate the URL and to redirect the request. This can be separated into two services where one service is required to generate the URL and the other just to redirect the URL. This way we increase the availability of the system. If one of the service goes down the other will still function.

The next step is to write and integrate a CLI system to talk to the server and fetch the URL. A client that can be used for an end user. I am also planning to integrate a caching mechanism in this but not something out of the shelf rather write a simple caching system with some cache eviction policy and use it.

Till then I will be waiting for the feedback. Happy Hacking.

I now have a Patreon open so that you folks can support me to do this stuff for longer time and sustain myself too. So feel free to subscribe to me and help me keeping doing this with added benefits.

by fardroid23 at August 08, 2020 01:49 PM

July 20, 2020

Farhaan Bukhsh

Link Tray

TLDR; Link Tray is a utility we recently wrote to curate links from different places and share it with your friends. The blogpost has technical details and probably some productivity tips.

Link Bubble got my total attention when I got to know about it, I felt it’s a very novel idea, it helps to save time and helps you to curate the websites you visited. So on the whole, and believe me I am downplaying it when I say Link Bubble does two things:

  1. Saves time by pre-opening the pages
  2. Helps you to keep a track of pages you want to visit

It’s a better tab management system, what I felt weird was building a whole browser to do that. Obviously, I am being extremely naive when I am saying it because I don’t know what it takes to build a utility like that.

Now, since they discontinued it for a while and I never got a chance to use it. I thought let me try building something very similar, but my use case was totally different. Generally when I go through blogs or articles, I open the links mentioned in a different tab to come back to them later. This has back bitten me a lot of time because I just get lost in so many links.

I thought if there is a utility which could just capture the links on the fly and then I could quickly go through them looking at the title, it might ease out my job. I bounced off the same idea across to Abhishek and we ended up prototyping LinkTray.

Our first design was highly inspired by facebook messenger but instead of chatheads we have links opened. If you think about it the idea feels very beautiful but the design is “highly” not scalable. For example if you have as many as 10 links opened we had trouble in finding our links of interest which was a beautiful design problems we faced.

We quickly went to the whiteboard and put up a list of requirements, first principles; The ask was simple:

  1. To share multiple links with multiple people with least transitions
  2. To be able to see what you are sharing
  3. To be able to curate links (add/remove/open links)

We took inspiration from an actual Drawer where we flick out a bunch of links and go through them. In a serendipitous moment the design came to us and that’s how link tray looks like the way it looks now.

Link Tray

Link Tray was a technical challenge as well. There is a plethora of things I learnt about the Android ecosystem and application development that I knew existed but never ventured into exploring it.

Link Tray is written in Java, and I was using a very loosely maintained library to get the overlay activity to work. Yes, the floating activity or application that we see is called an overlay activity, this allows the application to be opened over an already running application.

The library that I was using doesn’t have support for Android O and above. To figure that out it took me a few nights 😞 , also because I was hacking on the project during nights 😛 . After reading a lot of GitHub issues I figured out the problem and put in the support for the required operating system.

One of the really exciting features that I explored about Android is Services. I think I might have read most of the blogs out there and all the documentation available and I know that I still don't know enough. I was able to pick enough pointers to make my utility to work.

Just like Uncle Bob says make it work and then make it better. There was a persistent problem, the service needs to keep running in the background for it to work. This was not a functional issue but it was a performance issue for sure and our user of version 1.0 did have a problem with it. People got mislead because there was constant notification that LinkTray is running and it was annoying. This looked like a simple problem on the face but was a monster in the depth.

Architecture of Link Tray

The solution to the problem was simple stop the service when the tray is closed, and start the service when the link is shared back to link tray. Tried, the service did stop but when a new link was shared the application kept crashing. Later I figured out the bound service that is started by the library I am using is setting a bound flag to True but when they are trying to reset this flag , they were doing at the wrong place, this prompted me to write this StackOverflow answer to help people understand the lifecycle of service. Finally after a lot of logs and debugging session I found the issue and fixed it. It was one of the most exciting moment and it help me learn a lot of key concepts.

The other key learning, I got while developing Link Tray was about multi threading, what we are doing here is when a link is shared to link tray, we need the title of the page if it has and favicon for the website. Initially I was doing this on the main UI thread which is not only an anti-pattern but also a usability hazard. It was a network call which blocks the application till it was completed, I learnt how to make a network call on a different thread, and keep the application smooth.

Initially approach was to get a webview to work and we were literally opening the links in a browser and getting the title and favicon out, this was a very heavy process. Because we were literally spawning a browser to get information about links, in the initial design it made sense because we were giving an option to consume the links. Over time our design improved and we came to a point where we don’t give the option to consume but to curate. Hence we opted for web scraping, I used custom headers so that we don’t get caught by robot.txt. And after so much of effort it got to a place where it is stable and it is performing great.

It did take quite some time to reach a point where it is right now, it is full functional and stable. Do give it a go if you haven’t, you can shoot any queries to me.

Link to Link Tray: https://play.google.com/store/apps/details?id=me.farhaan.bubblefeed

Happy Hacking!

by fardroid23 at July 20, 2020 02:30 AM

June 07, 2020

Kuntal Majumder

Transitioning to Windows

So, recently I started using windows for work. Why? There are a couple of reasons, one that I needed to use MSVC, that is the Microsoft Visual C++ toolchain and the other being, I wasn’t quite comfortable to ifdef stuff for making it work on GCC aka, the GNU counterpart of MSVC.

June 07, 2020 02:38 PM

May 09, 2020

Kuntal Majumder

Krita Weekly #14

After an anxious month, I am writing a Krita Weekly again and probably this would be my last one too, though I hope not. Let’s start by talking about bugs. Unlike the trend going about the last couple of months, the numbers have taken a serious dip.

May 09, 2020 04:12 PM

May 06, 2020

April 11, 2020

Shakthi Kannan

Using Docker with Ansible

[Published in Open Source For You (OSFY) magazine, October 2017 edition.]

This article is the eighth in the DevOps series. In this issue, we shall learn to set up Docker in the host system and use it with Ansible.

Introduction

Docker provides operating system level virtualisation in the form of containers. These containers allow you to run standalone applications in an isolated environment. The three important features of Docker containers are isolation, portability and repeatability. All along we have used Parabola GNU/Linux-libre as the host system, and executed Ansible scripts on target Virtual Machines (VM) such as CentOS and Ubuntu.

Docker containers are extremely lightweight and fast to launch. You can also specify the amount of resources that you need such as CPU, memory and network. The Docker technology was launched in 2013, and released under the Apache 2.0 license. It is implemented using the Go programming language. A number of frameworks have been built on top of Docker for managing these cluster of servers. The Apache Mesos project, Google’s Kubernetes, and the Docker Swarm project are popular examples. These are ideal for running stateless applications and help you to easily scale them horizontally.

Setup

The Ansible version used on the host system (Parabola GNU/Linux-libre x86_64) is 2.3.0.0. Internet access should be available on the host system. The ansible/ folder contains the following file:

ansible/playbooks/configuration/docker.yml

Installation

The following playbook is used to install Docker on the host system:

---
- name: Setup Docker
  hosts: localhost
  gather_facts: true
  become: true
  tags: [setup]

  tasks:
    - name: Update the software package repository
      pacman:
        update_cache: yes

    - name: Install dependencies
      package:
        name: "{{ item }}"
        state: latest
      with_items:
        - python2-docker
        - docker

    - service:
        name: docker
        state: started

    - name: Run the hello-world container
      docker_container:
        name: hello-world
        image: library/hello-world

The Parabola package repository is updated before proceeding to install the dependencies. The python2-docker package is required for use with Ansible. Hence, it is installed along with the docker package. The Docker daemon service is then started and the library/hello-world container is fetched and executed. A sample invocation and execution of the above playbook is shown below:

$ ansible-playbook playbooks/configuration/docker.yml -K --tags=setup
SUDO password: 

PLAY [Setup Docker] *************************************************************

TASK [Gathering Facts] **********************************************************
ok: [localhost]

TASK [Update the software package repository] ***********************************
changed: [localhost]

TASK [Install dependencies] *****************************************************
ok: [localhost] => (item=python2-docker)
ok: [localhost] => (item=docker)

TASK [service] ******************************************************************
ok: [localhost]

TASK [Run the hello-world container] ********************************************
changed: [localhost]

PLAY RECAP **********************************************************************
localhost                  : ok=5    changed=2    unreachable=0    failed=0   

With verbose ’-v’ option to ansible-playbook, you will see an entry for LogPath, such as /var/lib/docker/containers//-json.log. In this log file you will see the output of the execution of the hello-world container. This output is the same when you run the container manually as shown below:

$ sudo docker run hello-world

Hello from Docker!

This message shows that your installation appears to be working correctly.

To generate this message, Docker took the following steps:
 1. The Docker client contacted the Docker daemon.
 2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
 3. The Docker daemon created a new container from that image which runs the
    executable that produces the output you are currently reading.
 4. The Docker daemon streamed that output to the Docker client, which sent it
    to your terminal.

To try something more ambitious, you can run an Ubuntu container with:
 $ docker run -it ubuntu bash

Share images, automate workflows, and more with a free Docker ID:
 https://cloud.docker.com/

For more examples and ideas, visit:
 https://docs.docker.com/engine/userguide/

Example

A Deep Learning (DL) Docker project is available (https://github.com/floydhub/dl-docker) with support for frameworks, libraries and software tools. We can use Ansible to build the entire DL container from the source code of the tools. The base OS of the container is Ubuntu 14.04, and will include the following software packages:

  • Tensorflow
  • Caffe
  • Theano
  • Keras
  • Lasagne
  • Torch
  • iPython/Jupyter Notebook
  • Numpy
  • SciPy
  • Pandas
  • Scikit Learn
  • Matplotlib
  • OpenCV

The playbook to build the DL Docker image is given below:

- name: Build the dl-docker image
  hosts: localhost
  gather_facts: true
  become: true
  tags: [deep-learning]

  vars:
    DL_BUILD_DIR: "/tmp/dl-docker"
    DL_DOCKER_NAME: "floydhub/dl-docker"

  tasks:
    - name: Download dl-docker
      git:
        repo: https://github.com/saiprashanths/dl-docker.git 
        dest: "{{ DL_BUILD_DIR }}"

    - name: Build image and with buildargs
      docker_image:
         path: "{{ DL_BUILD_DIR }}"
         name: "{{ DL_DOCKER_NAME }}"
         dockerfile: Dockerfile.cpu
         buildargs:
           tag: "{{ DL_DOCKER_NAME }}:cpu"

We first clone the Deep Learning docker project sources. The docker_image module in Ansible helps us to build, load and pull images. We then use the Dockerfile.cpu file to build a Docker image targeting the CPU. If you have a GPU in your system, you can use the Dockerfile.gpu file. The above playbook can be invoked using the following command:

$ ansible-playbook playbooks/configuration/docker.yml -K --tags=deep-learning

Depending on the CPU and RAM you have, it will take considerable amount of time to build the image with all the software. So be patient!

Jupyter Notebook

The built dl-docker image contains Jupyter notebook which can be launched when you start the container. An Ansible playbook for the same is provided below:

- name: Start Jupyter notebook
  hosts: localhost
  gather_facts: true
  become: true
  tags: [notebook]

  vars:
    DL_DOCKER_NAME: "floydhub/dl-docker"

  tasks:
    - name: Run container for Jupyter notebook
      docker_container:
        name: "dl-docker-notebook"
        image: "{{ DL_DOCKER_NAME }}:cpu"
        state: started
        command: sh run_jupyter.sh

You can invoke the playbook using the following command:

$ ansible-playbook playbooks/configuration/docker.yml -K --tags=notebook

The Dockerfile already exposes the port 8888, and hence you do not need to specify the same in the above docker_container configuration. After you run the playbook, using the ‘docker ps’ command on the host system, you can obtain the container ID as indicated below:

$ sudo docker ps
CONTAINER ID        IMAGE                    COMMAND               CREATED             STATUS              PORTS                NAMES
a876ad5af751        floydhub/dl-docker:cpu   "sh run_jupyter.sh"   11 minutes ago      Up 4 minutes        6006/tcp, 8888/tcp   dl-docker-notebook

You can now login to the running container using the following command:

$ sudo docker exec -it a876 /bin/bash

You can then run an ‘ifconfig’ command to find the local IP address (“172.17.0.2” in this case), and then open http://172.17.0.2:8888 in a browser on your host system to see the Jupyter Notebook. A screenshot is shown in Figure 1:

Jupyter Notebook

TensorBoard

TensorBoard consists of a suite of visualization tools to understand the TensorFlow programs. It is installed and available inside the Docker container. After you login to the Docker container, at the root prompt, you can start Tensorboard by passing it a log directory as shown below:

# tensorboard --logdir=./log

You can then open http://172.17.0.2:6006/ in a browser on your host system to see the Tensorboard dashboard as shown in Figure 2:

TensorBoard

Docker Image Facts

The docker_image_facts Ansible module provides useful information about a Docker image. We can use it to obtain the image facts for our dl-docker container as shown below:

- name: Get Docker image facts
  hosts: localhost
  gather_facts: true
  become: true
  tags: [facts]

  vars:
    DL_DOCKER_NAME: "floydhub/dl-docker"

  tasks:
    - name: Get image facts
      docker_image_facts:
        name: "{{ DL_DOCKER_NAME }}:cpu"

The above playbook can be invoked as follows:

$ ANSIBLE_STDOUT_CALLBACK=json ansible-playbook playbooks/configuration/docker.yml -K --tags=facts 

The ANSIBLE_STDOUT_CALLBACK environment variable is set to ‘json’ to produce a JSON output for readability. Some important image facts from the invocation of the above playbook are shown below:

"Architecture": "amd64", 
"Author": "Sai Soundararaj <saip@outlook.com>", 

"Config": {

"Cmd": [
   "/bin/bash"
], 

"Env": [
   "PATH=/root/torch/install/bin:/root/caffe/build/tools:/root/caffe/python:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin", 
   "CAFFE_ROOT=/root/caffe", 
   "PYCAFFE_ROOT=/root/caffe/python", 
   "PYTHONPATH=/root/caffe/python:", 
   "LUA_PATH=/root/.luarocks/share/lua/5.1/?.lua;/root/.luarocks/share/lua/5.1/?/init.lua;/root/torch/install/share/lua/5.1/?.lua;/root/torch/install/share/lua/5.1/?/init.lua;./?.lua;/root/torch/install/share/luajit-2.1.0-beta1/?.lua;/usr/local/share/lua/5.1/?.lua;/usr/local/share/lua/5.1/?/init.lua", 
   "LUA_CPATH=/root/torch/install/lib/?.so;/root/.luarocks/lib/lua/5.1/?.so;/root/torch/install/lib/lua/5.1/?.so;./?.so;/usr/local/lib/lua/5.1/?.so;/usr/local/lib/lua/5.1/loadall.so", 
   "LD_LIBRARY_PATH=/root/torch/install/lib:", 
   "DYLD_LIBRARY_PATH=/root/torch/install/lib:"
], 

"ExposedPorts": {
   "6006/tcp": {}, 
   "8888/tcp": {}
}, 

"Created": "2016-06-13T18:13:17.247218209Z", 
"DockerVersion": "1.11.1", 

"Os": "linux", 

"task": { "name": "Get image facts" }

You are encouraged to read the ‘Getting Started with Docker’ user guide available at http://docs.ansible.com/ansible/latest/guide_docker.html to know more about using Docker with Ansible.

April 11, 2020 06:30 PM

January 19, 2020

Rahul Jha

"isn't a title of this post" isn't a title of this post

[NOTE: This post originally appeared on deepsource.io, and has been posted here with due permission.]

In the early part of the last century, when David Hilbert was working on stricter formalization of geometry than Euclid, Georg Cantor had worked out a theory of different types of infinities, the theory of sets. This theory would soon unveil a series of confusing paradoxes, leading to a crisis in the Mathematics community  regarding the stability of the foundational principles of the math of that time.

Central to these paradoxes was the Russell’s paradox (or more generally, as we’d talk about later, the Epimenides Paradox). Let’s see what it is.

In those simpler times, you were allowed to define a set if you could describe it in English. And, owing to mathematicians’ predilection for self-reference, sets could contain other sets.

Russell then, came up with this:

\(R\)  is a set of all the sets which do not contain themselves.

The question was "Does \(R \) contain itself?" If it doesn’t, then according to the second half of the definition it should. But if it does, then it no longer meets the definition.

The same can symbolically be represented as:

Let \(R = \{ x \mid x \not \in x \} \), then \(R \in R \iff R \not \in R \)

Cue mind exploding.

“Grelling’s paradox” is a startling variant which uses adjectives instead of sets. If adjectives are divided into two classes, autological (self-descriptive) and heterological (non-self-descriptive), then, is ‘heterological’ heterological? Try it!

Epimenides Paradox

Or, the so-called Liar Paradox was another such paradox which shred apart whatever concept of ‘computability’ was, at that time - the notion that things could either be true or false.

Epimenides was a Cretan, who made one immortal statement:

“All Cretans are liars.”

If all Cretans are liars, and Epimenides was a Cretan, then he was lying when he said that “All Cretans are liars”. But wait, if he was lying then, how can we ‘prove’ that he wasn’t lying about lying? Ein?


This is what makes it a paradox: A statement so rudely violating the assumed dichotomy of statements into true and false, because if you tentatively think it’s true, it backfires on you and make you think that it is false. And a similar backfire occurs if you assume that the statement is false. Go ahead, try it!

If you look closely, there is one common culprit in all of these paradoxes, namely ‘self-reference’. Let’s look at it more closely.

Strange Loopiness

If self-reference, or what Douglas Hofstadter - whose prolific work on the subject matter has inspired this blog post - calls ‘Strange Loopiness’ was the source of all these paradoxes, it made perfect sense to just banish self-reference, or anything which allowed it to occur. Russell and Whitehead, two rebel mathematicians of the time, who subscribed to this point of view, set forward and undertook the mammoth exercise, namely “Principia Mathematica”, which we as we will see in a little while, was utterly demolished by Gödel’s findings.

The main thing which made it difficult to ban self-reference was that it was hard to pin point where exactly did the self-reference occur. It may as well be spread out over several steps, as in this ‘expanded’ version of Epimenides:

The next statement is a lie.

The previous statement is true.

Russell and Whitehead, in P.M. then, came up with a multi-hierarchy set theory to deal with this. The basic idea was that a set of the lowest ‘type’ could only contain ‘objects’ as members (not sets). A set of the next type could then only either contain objects, or sets of lower types. This, implicitly banished self-reference.

Since, all sets must have a type, a set ‘which contains all sets which are not members of themselves’ is not a set at all, and thus you can say that Russell’s paradox was dealt with.

Similarly, if an attempt is made towards applying the expanded Epimenides to this theory, it must fail as well, for the first sentence to make a reference to the second one, it has to be hierarchically above it - in which case, the second one can’t loop back to the first one.

Thirty one years after David Hilbert set before the academia to rigorously demonstrate that the system defined in Principia Mathematica was both consistent (contradiction-free) and complete (i.e. every true statement could be evaluated to true within the methods provided by P.M.), Gödel published his famous Incompleteness Theorem. By importing the Epimenides Paradox right into the heart of P.M., he proved that not just the axiomatic system developed by Russell and Whitehead, but none of the axiomatic systems whatsoever were complete without being inconsistent.

Clear enough, P.M. lost it’s charm in the realm of academics.

Before Gödel’s work too, P.M. wasn’t particularly loved as well.

Why?

It isn’t just limited to this blog post, but we humans, in general, have a diet for self-reference - and this quirky theory severely limits our ability to abstract away details - something which we love, not only as programmers, but as linguists too - so much so, that the preceding paragraph, “It isn’t … this blog … we humans …” would be doubly forbidden because the ‘right’ to mention ‘this blog post’ is limited only to something which is hierarchically above blog posts, ‘metablog-posts’. Secondly, me (presumably a human) belonging to the class ‘we’ can’t mention ‘we’ either.

Since, we humans, love self-reference so much, let’s discuss some ways in which it can be expressed in written form.

One way of making such a strange loop, and perhaps the ‘simplest’ is using the word ‘this’. Here:

  • This sentence is made up of eight words.
  • This sentence refers to itself, and is therefore useless.
  • This blog post is so good.
  • This sentence conveys you the meaning of ‘this’.
  • This sentence is a lie. (Epimenides Paradox)

Another amusing trick for creating a self-reference without using the word ‘this sentence’ is to quote the sentence inside itself.

Someone may come up with:

The sentence ‘The sentence contains five words’ contains five words.

But, such an attempt must fail, for to quote a finite sentence inside itself would mean that the sentence is smaller than itself. However, infinite sentences can be self-referenced this way.

The sentence
    "The sentence
        "The sentence
                                ...etc
                                ...etc
        is infinitely long"
    is infinitely long"
is infinitely long"

There’s a third method as well, which you already saw in the title - the Quine method. The term ‘Quine’ was coined by Douglas Hofstadter in his book “Gödel Escher, Bach” (which heavily inspires this blog post). When using this, the self-reference is ‘generated’ by describing a typographical entity, isomorphic to the quine sentence itself. This description is carried in two parts - one is a set of ‘instructions’ about how to ‘build’ the sentence, and the other, the ‘template’ contains information about the construction materials required.

The Quine version of Epimenides would be:

“yields falsehood when preceded by it’s quotation” yields falsehood when preceded by it’s quotation

Before going on with ‘quining’, let’s take a moment and realize how awfully powerful our cognitive capacities are, and what goes in our head when a cognitive payload full of self-references is delivered - in order to decipher it, we not only need to know the language, but also need to work out the referent of the phrase analogous to ‘this sentence’ in that language. This parsing depends on our complex, yet totally assimilated ability to handle the language.

The idea of referring to itself is quite mind-blowing, and we keep doing it all the time — perhaps, why it feels so ‘easy’ for us to do so. But, we aren’t born that way, we grow that way. This could better be realized by telling someone much younger “This sentence is wrong.”. They’d probably be confused - What sentence is wrong?. The reason why it’s so simple for self-reference to occur, and hence allow paradoxes, in our language, is well, our language. It allows our brain to do the heavy lifting of what the author is trying to get through us, without being verbose.

Back to Quines.

Reproducing itself

Now, that we are aware of how ‘quines’ can manifest as self-reference, it would be interesting to see how the same technique can be used by a computer program to ‘reproduce’ itself.

To make it further interesting, we shall choose the language most apt for the purpose - brainfuck:

>>>>+++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++++>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++++>++++++++++++++++++++++++++++++++++++++++++++++>++++++++++++++++++++++++++++++++++++++++++++++>++++++++++++++++++++++++++++++++++++++++++++++>++++++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++++>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++++>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++++>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++>++++++++++++++++++++++++++++++++++++++++++++++>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++>++++++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++++>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++++>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++>++++++++++++++++++++++++++++++++++++++++++++++>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++>+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++>++++++++[->++++++++<]>--....[-]<<[<]<<++++++++[->+++++>++++++++<<]>+++>-->[[-<<<+>.>>]<.[->+<]<[->+<]>>>]<<<<[<]>[.>]

Running that program above produces itself as the output. I agree, it isn’t the most descriptive program in the world, so written in Python below, is the nearest we can go to describe what’s happening inside those horrible chains of +’s and >’s:

THREE_QUOTES = '"' * 3

def eniuq(template): print(
  f'{template}({THREE_QUOTES}{template}{THREE_QUOTES})')

eniuq("""THREE_QUOTES = '"' * 3

def eniuq(template): print(
  f'{template}({THREE_QUOTES}{template}{THREE_QUOTES})')

eniuq""")

The first line generates """ on the fly, which marks multiline strings in Python.

Next two lines define the eniuq function, which prints the argument template twice - once, plain and then surrounded with triple quotes.

The last 4 lines cleverly call this function so that the output of the program is the source code itself.

Since we are printing in an order opposite of quining, the name of the function is ‘quine’ reversed -> eniuq (name stolen from Hofstadter again)

Remember the discussion about how self-reference capitalizes on the processor? What if ‘quining’ was a built-in feature of the language, providing what we in programmer lingo call ‘syntactic sugar’?

Let’s assume that an asterisk, * in the brainfuck interpreter would copy the instructions before executing them, what would then be the output of the following program?

*

It’d be an asterisk again. You could make an argument that this is silly, and should be counted as ‘cheating’. But, it’s the same as relying on the processor, like using “this sentence” to refer to this sentence - you rely on your brain to do the inference for you.

What if eniuq was a builtin keyword in Python? A perfect self-rep was then just be a call away:

eniuq('eniuq')

What if quine was a verb in the English language? We could reduce a lot of explicit cognitive processes required for inference. The Epimenides paradox would then be:

“yields falsehood if quined” yields falsehood if quined

Now, that we are talking about self-rep, here’s one last piece of entertainment for you.

The Tupper’s self-referential formula

This formula is defined through an inequality:

\({1 \over 2} < \left\lfloor \mathrm{mod}\left(\left\lfloor {y \over 17} \right\rfloor 2^{-17 \lfloor x \rfloor - \mathrm{mod}(\lfloor y\rfloor, 17)},2\right)\right\rfloor\)

If you take that absurd thing above, and move around in the cartesian plane for the coordinates \(0 \le x \le 106, k \le y \le k + 17\), where \(k\) is a 544 digit integer (just hold on with me here), color every pixel black for True, and white otherwise, you'd get:

This doesn't end here. If \(k\) is now replaced with another integer containing 291 digits, we get yours truly:

January 19, 2020 06:30 PM