Functional Programming with Python

Excellent explanation of how to use functional programming to make your Python code smaller and easier to debug

Functional Programming with Python

oneliner: reset Django administrator password

$ ./manage.py shell
 
>>> from django.contrib.auth.models import User
>>> a=User.objects.all()[0]; a.set_password('beerbeer'); a.save()

correcting NetworkManager

Ubuntu uses the NetworkManager (NM) daemon to regulate a computer's network connections, most importantly the wireless.  Sometimes it gets confused, and sometimes upstream routers lie to it.

Every now and then someone lies to NM, telling it about Domain Name Server (DNS) machines that don't work.  The symptom is that even though your local LAN works, and you can ping external IPs, web pages generally don't work.  They stall and give you the Unhappy Chrome message.

To test DNS:

dig yahoo.com

If it comes back nearly immediately with verbiage in the "ANSWER SECTION", then your nameserver is fine and working properly.  If it takes a few seconds, then something is confused.

One fix is to overwrite NetworkManager's settings when it does anything.  This is kind of impolite, but what the hell.

As root, copy this script to /etc/NetworkManager/dispatcher.d/50fix-dns and mark it executable with "chmod +x ".

#!/bin/bash
# Fix DNS set by misguided routers
#
# JM 7/2013
cat << EOF > /etc/resolv.conf
# DONT EDIT
# see $0
nameserver 4.2.2.2
EOF

Concurrency/99Bottles

simple concurrency program, written in ~10 different styles in Python!  Includes coroutines, generators, Greenlet, Gevent, and Twisted.

Concurrency/99Bottles - Python Wiki

prevent multiple copies of script running at same time

Single-process servers (and cron jobs) are easier to write.  If a script fires up, then starts processing a job that another copy of the same script is working on, then bad things happen.

The traditional solution to this is a bit of a hack.  Script fires up, writes its unique process id (PID) to a magic file somewhere.  When the script dies we delete the file.  This works somewhat, but fails if the script gets impolitely killed -- the "lock" file still exists but the associated script is no longer running.

The following trick is useful. When the script fires up it registers itself with a global list of strings held by the OS. If the code can't register itself, it knows the program is already running, and should exit.  When a script dies or is killed, the OS itself de-registers it, thus the script can run again.

Alas this trick is Linux-only.


#!/usr/bin/env python

import os, sys, time


def get_lock(process_name=None):
    """
    prevent multiple processes running at the same time
    """
    # http://stackoverflow.com/questions/788411/check-to-see-if-python-script-is-running
    import socket
    if not process_name:
        process_name = os.path.basename(sys.argv[0])
    lock_socket = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
    try:
        lock_socket.bind('\0' + process_name)
        return lock_socket
    except socket.error:
        pass
    return None


def main():
    sys.process_lock = get_lock()
    print sys.process_lock
    if sys.process_lock:
        time.sleep(30)


if __name__=='__main__':
    main()


Gist: https://gist.github.com/shavenwarthog/5824515

automating web tests with Ghost

Developing reliable software for the web is a challenging proposition. A site's code organization is complex, with libraries interacting with each other in an exponentially expanding number of possibilities.  It's quite easy to make a minor change in one part of the site that breaks another part.

Manual testing isn't an option. If you have a rarely-used but critical part of the site's workflow, like "email me my forgotten password", it would affect users if the functionality breaks. With manual testing you might never know your users are getting a poor experience.

One technique for making reliable web software is using automated tests.  In addition to the lower-level unit tests of code, it's valuable to have a scriptable "browser" that is able to click through your site and check various features.

Install and verify Ghost

Here's how to install and use Ghost, a Python library that emulates a Webkit browser. The browser is comparable to Apple Safari or Google Chrome.  Instructions are for Ubuntu.

sudo apt-get install python-pyside
sudo pip install Ghost.py

Verify that things were installed correctly:

python -c 'from ghost import Ghost'

Example headless web scripting

Here's a sample program.  It loads the Duck Duck Go search page, then types in a search query. It then clicks the submit button, waits for the search results to load, and prints out the HTML of the first result. Finally it captures a snapshot of the page!

#!/usr/bin/python
from ghost import Ghost

ghost = Ghost()
ghost.open('http://duckduckgo.com/') 

ghost.wait_for_selector('input[name=q]')
ghost.fill("#search_form_homepage", {'q': 'beer'})
ghost.fire_on("#search_form_homepage",
              "submit",
              expect_loading=True) 

ghost.wait_for_selector('#r1-0')
result, _resources = ghost.evaluate(
    "document.getElementById('r1-0').innerHTML;")
print result

ghost.capture_to('beer.png')

Here's the above in a Gist for your use: Ghost example with Duck Duck Go

Resources:

• Steven Fruchter's Ghost tutorial
• Ghost.py home page

Retrotechtacular: How I wrote Pitfall for the Atari 2600

I adored writing assembly language for the Apple back in, err, 1984?  Somehow it felt very "clean", things would be completely inscrutable until you tweaked just the right bit and then everything would pop in to place.  Then the program was rock-solid, doing exactly what you told it incredibly efficiently and quickly.  Of course it never did exactly what you wanted, but that was the fun of it.

The "C" language had some of the clarity of assembly, and the advantage it would crash hard if you asked it to do something silly.  Writing a device driver in C++ (what luxury!) was fun because it would do different things depending on if I was writing debug statements or not.  The console was 9600 baud, and the code was timing-dependent, so writing code so you could see what you were doing... changed what you were doing!   As a perk, if you did something unexpected the machine would hard-lock, requiring a long and luxurious 40-second reboot.  What fun!

More common languages are too "fluffy", they always do things; there's no time consequence for writing loose code.  Ah well.  I adore Python and even though working with it for many years I still really enjoy it and learn new things.

I imagine this guy knows *exactly* what I'm talking about:
Retrotechtacular: How I wrote Pitfall for the Atari 2600