Testing Patterns

Testing code is hard.

As a developer, I want to write cool code to solve interesting, real-world problems. I don't want to be "smart" and write "smart tests" that are inscrutable, rather than spending time on writing more (feature) code.

Here an article that summarizes my experience. Namely, mocks are evil, and tests should be dumb. If tests are complex enough to require their own framework, it means you need to take a close look at where you're spending your rare brainpower: on features, or unit tests?

No company ever was able to take over a market and satisfy customers by having more/better unit tests.

• A decade of unit testing (Karl Seguin): have super-focused unit tests that focus on calculation, not interaction with other functions; integration tests are a must; ignore code coverage [!]; more logging is better than having more tests.

Improving Linux networking performance [LWN.net]

Improving Linux networking performance [LWN.net]

Docker/Nginx Tip: run standalone

Instead of using Docker to tweak your Nginx config, it's possible to run Nginx directly so it won't become a daemon and make your container exit.  To wit:

nginx -g 'daemon off;'

Docker/Fig tip: easily watch cluster logs

When using Fig to run a cluster of Docker containers, it's handy to have a console, and also a separate terminal with information so you can see what's going on.  That is, an interactive console, plus all the logs from all the containers.

Use Fig to start your cluster, then interactively your containers as needed. As you modify stuff, the services emit logging messages, which you can see in the second terminal.

here's the code for the second terminal. It uses Fig to get logs for all the containers.  When the config is changes and the cluster is restarted, this command loops again to reconnect and get all the new log messages. In this way you can just do you work and this terminal will always have the latest logs.

Watch Docker cluster logs, even after restart

for ((;;)) ; do fig logs ; done

tl;dr: infinite loops ftw

extra Docker networking goodies

I work a great deal with Docker, connecting webservers and databases together, hiding everything behind a caching proxy like Nginx.  Docker is a wonderful tool, providing isolation and semi-magical capabilities like hiding an entire "cluster" of machines behind a single container, a single process.

However containers are not much fun. They're generally so stripped down as to make debugging issues rather more awkward than normal.  The following questions are common when something doesn't work:

• is my process running on the correct port?
• what processes are running?
• which services are up, on which ports?

With a default Docker container, answers to these questions aren't obvious.

Solution: add the following to all of your Dockerfiles:

RUN apt-get update && apt-get install -y apt-file curl net-tools procps psmisc
# 'psmisc' provides /bin/fuser; procps=>/bin/ps; net-tools=>netstat

Here's an example Nginx Dockerfile which 1) lets me verify Nginx is running on the right port and happy, and 2) when I hop into the container with "docker exec", I can use normal tools to see what's going on.

FROM nginx:1.7
RUN apt-get update && apt-get install -y apt-file curl net-tools procps psmisc
# 'psmisc' provides /bin/fuser; procps=>/bin/ps; net-tools=>netstat
ADD index.html /usr/share/nginx/html/index.html
ADD nginx.conf /etc/nginx.conf

OSX Docker Tip: set datetime

OSX doesn't directly support the awesomeness that is Docker containers. It uses an intermediate Linux virtual machine, and that runs containers for you, on your behalf.

I started getting weird Amazon S3 errors. My project would work locally (in OSX), and would work inside a container (Linux), but the S3 filesystem would throw errors.

It turns out Amazon doesn't like it if you request an asset two months in the past... The Mac system had its clock set correctly, but the VirtualBox Linux VM had drifted.

The following hops into the VM, then sets the clock based on a global Network Time Protocol server. You should only have do this a couple times a year.

boot2docker sudo ntpclient -sh pool.ntp.org

tiny "Hello World" webserver

I'm writing a Nginx configuration that will wrap password protection on top of a bunch of dependent web servers. For testing, I want a few "hello world" type web servers, each of one gives a different message, but is really tiny.

The following is what I came up with -- great for testing.

# USAGE:
# python ./hello.py free beer 

import SimpleHTTPServer
import StringIO
import SocketServer
import sys

PORT = 8000

class MyHandler(SimpleHTTPServer.SimpleHTTPRequestHandler):
    MESSAGE = ' '.join(sys.argv[1:])
    
    def send_head(self):
        self.send_response(200)
        self.send_header("Content-type", 'text/plain')
        self.end_headers()
        return StringIO.StringIO(self.MESSAGE)
    
httpd = SocketServer.TCPServer(("", PORT), MyHandler)

print "serving at port", PORT
httpd.serve_forever()