Thursday, June 20, 2013

prevent multiple copies of script running at same time

Single-process servers (and cron jobs) are easier to write.  If a script fires up, then starts processing a job that another copy of the same script is working on, then bad things happen.

The traditional solution to this is a bit of a hack.  Script fires up, writes its unique process id (PID) to a magic file somewhere.  When the script dies we delete the file.  This works somewhat, but fails if the script gets impolitely killed -- the "lock" file still exists but the associated script is no longer running.

The following trick is useful. When the script fires up it registers itself with a global list of strings held by the OS. If the code can't register itself, it knows the program is already running, and should exit.  When a script dies or is killed, the OS itself de-registers it, thus the script can run again.

Alas this trick is Linux-only.

#!/usr/bin/env python

import os, sys, time

def get_lock(process_name=None):
    prevent multiple processes running at the same time
    import socket
    if not process_name:
        process_name = os.path.basename(sys.argv[0])
    lock_socket = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
        lock_socket.bind('\0' + process_name)
        return lock_socket
    except socket.error:
    return None

def main():
    sys.process_lock = get_lock()
    print sys.process_lock
    if sys.process_lock:

if __name__=='__main__':


Monday, June 17, 2013

automating web tests with Ghost

Developing reliable software for the web is a challenging proposition. A site's code organization is complex, with libraries interacting with each other in an exponentially expanding number of possibilities.  It's quite easy to make a minor change in one part of the site that breaks another part.

Manual testing isn't an option. If you have a rarely-used but critical part of the site's workflow, like "email me my forgotten password", it would affect users if the functionality breaks. With manual testing you might never know your users are getting a poor experience.

One technique for making reliable web software is using automated tests.  In addition to the lower-level unit tests of code, it's valuable to have a scriptable "browser" that is able to click through your site and check various features.

Install and verify Ghost

Here's how to install and use Ghost, a Python library that emulates a Webkit browser. The browser is comparable to Apple Safari or Google Chrome.  Instructions are for Ubuntu.

sudo apt-get install python-pyside
sudo pip install

Verify that things were installed correctly:
python -c 'from ghost import Ghost'

Example headless web scripting

Here's a sample program.  It loads the Duck Duck Go search page, then types in a search query. It then clicks the submit button, waits for the search results to load, and prints out the HTML of the first result. Finally it captures a snapshot of the page!

from ghost import Ghost

ghost = Ghost()'')
ghost.fill("#search_form_homepage", {'q': 'beer'})
result, _resources = ghost.evaluate(
print result

Here's the above in a Gist for your use: Ghost example with Duck Duck Go


Wednesday, June 5, 2013

Retrotechtacular: How I wrote Pitfall for the Atari 2600

I adored writing assembly language for the Apple back in, err, 1984?  Somehow it felt very "clean", things would be completely inscrutable until you tweaked just the right bit and then everything would pop in to place.  Then the program was rock-solid, doing exactly what you told it incredibly efficiently and quickly.  Of course it never did exactly what you wanted, but that was the fun of it.

The "C" language had some of the clarity of assembly, and the advantage it would crash hard if you asked it to do something silly.  Writing a device driver in C++ (what luxury!) was fun because it would do different things depending on if I was writing debug statements or not.  The console was 9600 baud, and the code was timing-dependent, so writing code so you could see what you were doing... changed what you were doing!   As a perk, if you did something unexpected the machine would hard-lock, requiring a long and luxurious 40-second reboot.  What fun!

More common languages are too "fluffy", they always do things; there's no time consequence for writing loose code.  Ah well.  I adore Python and even though working with it for many years I still really enjoy it and learn new things.

I imagine this guy knows *exactly* what I'm talking about:
Retrotechtacular: How I wrote Pitfall for the Atari 2600