upcoming talks: new Django, also concurrency

In development are my next three talks.  The first is a short one (~20 mins) on "New Stuff in Django 1.7" which highlights things added or extended over the last two years.  Next is the major feature for 1.7, "Django Migrations in 1.7".  TLDR: it's like a simpler, nicer South.
The third talk is on "Plentiful Perplexities in Parallel Python."  There are at least five different ways to do have multiple things be running at once -- I'll go over them in terms of ease-of-use, generality, and reliability/ease of debugging.  There's a lot of non-intuitive features and side effects. I'll go over the most common.
I'm always looking for ideas for new talks and classes.  Please email me!

new Twitter account

A red letter day!
soon in the Land of Whiskey
this is #myfirstTweet

https://twitter.com/JohnTellsAll

devops: per-process swap usage

awk '/Name/{name=$2}; /VmSwap/{printf "%20s\t%s %s\n",name,$2,$3}END{ print ""}' /proc/*/status

inspired from: adnans

Output:

     chromium-browse 19664 kB

     chromium-browse 6128 kB

        avahi-daemon 320 kB

        avahi-daemon 220 kB

              colord 644 kB

                smbd 840 kB

ref: fio I/O performance/stress tool

"fio is an I/O tool meant to be used both for benchmark and stress/hardware verification. It has support for 19 different types of I/O engines"



fio – Freecode

StackOverflow: achievements unlocked

I'm now one of the StackOverflow "Top Python Answerers" of all time!  I'm also a Top Answerer for Multiprocessing.

strace to the rescue, or, program uses mysterious config defaults

Strace is a powerful tool that lets you debug other programs, and find where config files are found.

Programs have bugs.  They're written for one platform and ported to another.  In any case sometimes things don't work as they should.

I manage this website using the rather nice FTP client lftp.  It can automatically "mirror" a local directory on my lappytop onto the web server. This means I don't manage the server directly, I only push files up, so if something happens I can switch to another server with very little hassle.

Today I noticed I couldn't store the password to the FTP account. The documented command bookmark edit brought up an editor, but it didn't have my saved bookmark!  On my system it's misconfigured.  Bookmarks work, as I can do open jta to connect to this site.

To find out where lftp is storing bookmarks, I used the wonderful tool strace.  It lets you run a subcommand, and monitor exactly what is going on from a system perspective.  It logs system calls.

In this case, I knew I wanted to scan for the open() system call, when the program opens up the bookmark file.  Here's the command I used to get lftp to cough up the information:

$ strace -o z -e trace=open lftp -c 'open jta'
Password:
cd: Login failed: 530 Login incorrect. 

The above command ran lftp, read the bookmark file, and tried to connect (open) to the JTA ("John Tells All") FTP bookmark.  I exited out because I just wanted to see the strace log to find the bookmark file.

I guessed the bookmark file would have the letters 'ook' in it:

$ egrep ook z
open("/home/johnm/.local/share/lftp/bookmarks", O_RDONLY) = 4

The above trace shows that the application is storing bookmarks in ~/.local/share/lftp/bookmarks  The open() command returned a number which means the file was found and opened correctly.

Success!  I can now edit this file and add my own bookmarks, including passwords.

See also: How to Bookmark Passwords with Lftp

code: search Pypi for all package versions

sample code to query the global Python package repository, PyPi, directly. Stack Overflow

#!/usr/bin/env python

'''
pversions.py -- search for package version from PyPi
'''
# adapted from pip.commands.SearchCommand

import sys, xmlrpclib

pnames = sys.argv[1:]
if not pnames:
    sys.exit('Usage: pversions (packagename)...')

pypi = xmlrpclib.ServerProxy('https://pypi.python.org/pypi')
for packagename in (pname.lower() for pname in pnames):
    print packagename,':'
    exact_hits = (
        hit for hit in pypi.search({'name': packagename})
        if hit['name'].lower() == packagename
    )
    print ', '.join( (hit['version'] for hit in exact_hits) )