Programming Sucks

[Linking to this for all of us who make our living digging tunnels under Mordor …]

Every friend I have with a job that involves picking up something heavier than a laptop more than twice a week eventually finds a way to slip something like this into conversation: “Bro,1you don’t work hard. I just worked a 4700-hour week digging a tunnel under Mordor with a screwdriver.”  They have a point. Mordor sucks, and it’s certainly more physically taxing to dig a tunnel than poke at a keyboard unless you’re an ant. But, for the sake of the argument, can we agree that stress and insanity are bad things? Awesome. Welcome to programming.

“Rounding Corrections”, by Sandra Haynes, and “The Floor”, by Melissa Hall

Passing these along from the folks at io9 on planet Gizmodo:

What might life be like in a future where every citizen was guaranteed a universal basic income? Two very good short stories imagine the sociological and cultural changes that might happen, in poignant and chilling ways. One of them was good enough to earn a sizable cash prize and you can read it now.


Meltdown and Spectre: What they are, who’s affected, and what’s being done about them

ArsTechnica has a concise, well-written article on two recently-disclosed vulnerabilities found in nearly all Intel processors and some AMD and ARM processors.   Particularly worrisome is the impact these flaws can have on virtualised environments, which includes cloud-based virtual machines and their hypervisors.  Remedying these vulnerabilities will ultimately require replacing the processors.  However, near-term software-only solutions have been developed and are already being deployed.   This article gives a good, moderately technical overview of the what, who, and how of the flaws and the fixes.

A Gentle Introduction to Quantum Computing

A very good, truly gentle intro to quantum computing.  A basic understanding of probability and complex numbers is required.  But, if you’re truly interested in gaining a basic understanding of QC’s mathematics, you’ll likely already be familiar with those.

ABSTRACT: Quantum Computing is a new and exciting field at the intersection of mathematics, computer science and physics. It concerns a utilization of quantum mechanics to improve the efficiency of computation. Here we present a gentle introduction to some of the ideas in quantum computing. The paper begins by motivating the central ideas of quantum mechanics and quantum computation with simple toy models. From there we move on to a formal presentation of the small fraction of (finite dimensional) quantum mechanics that we will need for basic quantum computation. Central notions of quantum architecture (qubits and quantum gates) are described. The paper ends with a presentation of one of the simplest quantum algorithms: Deutsch’s algorithm. Our presentation demands neither advanced mathematics nor advanced physics.

Noson Yanofsky et al.


Where did THAT come from?

All too often Python programmers ask themselves, “Who took the last cup of coffee and didn’t start another pot brewing!?”  Probably more often they’ll wonder “Where the hell is my proggy loading that function (or library or module) from?”   Whereas one can usually track down the SOB who left the empty pot and go all BobP* on him, it is often harder, depending on the platform, to answer the latter question.

Recently I found myself with several virtualenv paths as well as two or three actual installations (talking Python, here) along with several environment variables (PATH, LD_LIBRARY_PATH, PYTHONPATH, DYLD_LIBRARY_PATH, DYLD_FRAMEWORK_PATH, etc.) that were, to be perfectly blunt, FUBAR.   The code I was working on needed to load a modified version of the SQLITE3 module, which itself loaded a shared object (.so) file somewhere within its bowels.    No matter how I changed the several *PATH variables, I kept getting results that indicated the modified shared library simply wasn’t being referenced.   I could have spent a great deal of time trying various permutations of the paths in each of the PATH variables.   Having no desire to make a career (let alone a life-long persuit) of this, I decided to take a somewhat more sane approach.   Surely, there must be a way to tell which module python actually loads when it encounters a line like:

import sqlite3

Enter the modulefinder module (section 30.6 of the Python Library Reference).  This contains a class by the same name that gathers information from import statements encountered in the course of executing a script.  Its report method can be invoked after the script has finished and will list the modules imported  and the actual files where those modules were found.    So, to find out which sqlite3 module was being imported, I first tried:

from modulefinder import ModuleFinder
mf = ModuleFinder()
mf.run_script('import sqlite3')

which rewards me with:

Traceback (most recent call last):
  File "", line 1, in 
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/", line 112, in run_script
    fp = open(pathname, READ_MODE)
IOError: [Errno 2] No such file or directory: 'import sqlite3'

What modulefinder really wants is a script that contains the import statements you’re looking to trace.   So, I put

import sqlite3

into a file named and tried again:


and … Success!

Name                      File
----                      ----
m __main__        
m _sqlite3                  /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload/
m datetime                  /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload/
P sqlite3                   /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/sqlite3/
m sqlite3.dbapi2            /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/sqlite3/
m time                      /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload/

This canned report is nice, but you might want to use ModuleFinder‘s results
programatically, in which case you’ll want to refer to its modules attribute, which is
a Python dict object mapping module names to the files from whcih they were actually imported.

Beyond Python

Although this doesn’t tell me where my modified SQLITE3 library is being loaded, it does bring me a step closer.
I can now tell where python is picking up its sqlite3 module along with several other modules, some of which I can readily dismiss ( and It’s obvious that sqlite3 is a Python package from the final part of its path (…/sqlite3/ as well as from the capital “P” (for “package”) to the left of the module’s name in the report. Looking inside this file we find just one (executable) line of Python:

from dbapi2 import *

From the ModuleFinder report we already knew this module was being imported, so we continue digging there:

from _sqlite3 import *

which, again, confirms what we already knew from the report. But, that’s all we find in this file, aside from a few class declarations that are nothing more than wrappers and dressing for the contents of _sqlite3. If we go fossicking about inside the source code for (which can be found under the Modules subdirectory of the Python source tree) we’ll find a very simple and typical Python extension module that wraps the functions found in the SQLITE3 library itself. This seems to be a bit of a dead end. We still don’t know where the actual location of the SQLITE3 library (i.e. libsqlite3.dylib) that the extension module is referencing. For this, we’ll need a different tool: the ctypes module.

Ctypes let’s us load and easily access any dynamically linked library (DLL, .so, .dyld, etc) object and, in the case of function objects, call them from within python code. It also contains a very useful function called find_library that will … well … find shared libraries. It uses the environment’s *PATH variables the same way the dynamic linker would use them (i.e., searching these in the same order) for the specified library. So, we look for sqlite3:

from ctypes.util import find_library


and get:


Looking at my DYLD_LIBRARY_PATH environment variable, I have only /opt/local/lib in the path. I can either replace the libsqlite3.dyld file in this directory with my own, modified version, or I can prepend this environment variable with the path where the modified library can be found:

export DYLD_LIBRARY_PATH="/home/nick/tryit/sqlite-3.6.23/.libs:$DYLD_LIBRARY_PATH"

and then use find_library as before

from ctypes.util import find_library


which yields


just as I’d expect.


There is always the temptation to automate processes such as these. I could have tied this all together into a nice neat little function that would take the module name as its argument and return the path, if any, of the associated dynamic library. If I were trying to develop a large scale code scanning tool, say, for doing a security audit of the code, I might do just that. Here, I just wanted to find out which of several possible dynamic libraries was being loaded by importing a single, particular module. A few lines of code are all that’s needed in this case. I’ll leave the fancier tool development to the intrepid reader. Right now, I have to go make some more coffee … before I get yelled at. (Again.)