new Twitter account

A red letter day!
soon in the Land of Whiskey
this is #myfirstTweet

https://twitter.com/JohnTellsAll

devops: per-process swap usage

awk '/Name/{name=$2}; /VmSwap/{printf "%20s\t%s %s\n",name,$2,$3}END{ print ""}' /proc/*/status

inspired from: adnans

Output:

     chromium-browse 19664 kB

     chromium-browse 6128 kB

        avahi-daemon 320 kB

        avahi-daemon 220 kB

              colord 644 kB

                smbd 840 kB

ref: fio I/O performance/stress tool

"fio is an I/O tool meant to be used both for benchmark and stress/hardware verification. It has support for 19 different types of I/O engines"



fio – Freecode

StackOverflow: achievements unlocked

I'm now one of the StackOverflow "Top Python Answerers" of all time!  I'm also a Top Answerer for Multiprocessing.

strace to the rescue, or, program uses mysterious config defaults

Strace is a powerful tool that lets you debug other programs, and find where config files are found.

Programs have bugs.  They're written for one platform and ported to another.  In any case sometimes things don't work as they should.

I manage this website using the rather nice FTP client lftp.  It can automatically "mirror" a local directory on my lappytop onto the web server. This means I don't manage the server directly, I only push files up, so if something happens I can switch to another server with very little hassle.

Today I noticed I couldn't store the password to the FTP account. The documented command bookmark edit brought up an editor, but it didn't have my saved bookmark!  On my system it's misconfigured.  Bookmarks work, as I can do open jta to connect to this site.

To find out where lftp is storing bookmarks, I used the wonderful tool strace.  It lets you run a subcommand, and monitor exactly what is going on from a system perspective.  It logs system calls.

In this case, I knew I wanted to scan for the open() system call, when the program opens up the bookmark file.  Here's the command I used to get lftp to cough up the information:

$ strace -o z -e trace=open lftp -c 'open jta'
Password:
cd: Login failed: 530 Login incorrect. 

The above command ran lftp, read the bookmark file, and tried to connect (open) to the JTA ("John Tells All") FTP bookmark.  I exited out because I just wanted to see the strace log to find the bookmark file.

I guessed the bookmark file would have the letters 'ook' in it:

$ egrep ook z
open("/home/johnm/.local/share/lftp/bookmarks", O_RDONLY) = 4

The above trace shows that the application is storing bookmarks in ~/.local/share/lftp/bookmarks  The open() command returned a number which means the file was found and opened correctly.

Success!  I can now edit this file and add my own bookmarks, including passwords.

See also: How to Bookmark Passwords with Lftp

code: search Pypi for all package versions

sample code to query the global Python package repository, PyPi, directly. Stack Overflow

#!/usr/bin/env python

'''
pversions.py -- search for package version from PyPi
'''
# adapted from pip.commands.SearchCommand

import sys, xmlrpclib

pnames = sys.argv[1:]
if not pnames:
    sys.exit('Usage: pversions (packagename)...')

pypi = xmlrpclib.ServerProxy('https://pypi.python.org/pypi')
for packagename in (pname.lower() for pname in pnames):
    print packagename,':'
    exact_hits = (
        hit for hit in pypi.search({'name': packagename})
        if hit['name'].lower() == packagename
    )
    print ', '.join( (hit['version'] for hit in exact_hits) )

code: sync files using FTP

My webhost doesn't support rsync, only FTP.  For grins I wrote a script to scan through interesting local files and push them up to my webhost.  Alas the progress wasn't as trivial as I'd hoped, so I abandoned the effort and used lftp the "mirror" command.

The code shows a number of best practices:

• don't store passwords/keys, pass them in via environment variables
• make an iterator of files, add to it using chain()
• Python3 subprocess.check_output() is a little odd
• logging, like voting, should be done early and often
• code ensures FTP connection always closed cleanly

https://github.com/shavenwarthog/johntellsall/blob/master/class/django-queryset/ftpsync.py