Friday, March 13, 2015

Wednesday, February 4, 2015

networking with the Ant-ernet

Harvester ants and the Internet protocol TCP both have the same goal: maximize transfer of bits/food in as efficient a manner as possible. It turns out they use the same protocol!

The Independent Discovery of TCP/IP, By Ants

Friday, January 30, 2015

Testing Patterns

Testing code is hard.

As a developer, I want to write cool code to solve interesting, real-world problems. I don't want to be "smart" and write "smart tests" that are inscrutable, rather than spending time on writing more (feature) code.

Here an article that summarizes my experience. Namely, mocks are evil, and tests should be dumb. If tests are complex enough to require their own framework, it means you need to take a close look at where you're spending your rare brainpower: on features, or unit tests?

No company ever was able to take over a market and satisfy customers by having more/better unit tests.
  • A decade of unit testing (Karl Seguin): have super-focused unit tests that focus on calculation, not interaction with other functions; integration tests are a must; ignore code coverage [!]; more logging is better than having more tests.

Thursday, January 22, 2015

Improving Linux networking performance [LWN.net]

Improving Linux networking performance [LWN.net]

Docker/Nginx Tip: run standalone

Instead of using Docker to tweak your Nginx config, it's possible to run Nginx directly so it won't become a daemon and make your container exit.  To wit:

nginx -g 'daemon off;'

Docker/Fig tip: easily watch cluster logs

When using Fig to run a cluster of Docker containers, it's handy to have a console, and also a separate terminal with information so you can see what's going on.  That is, an interactive console, plus all the logs from all the containers.

Use Fig to start your cluster, then interactively your containers as needed. As you modify stuff, the services emit logging messages, which you can see in the second terminal.

here's the code for the second terminal. It uses Fig to get logs for all the containers.  When the config is changes and the cluster is restarted, this command loops again to reconnect and get all the new log messages. In this way you can just do you work and this terminal will always have the latest logs.

Watch Docker cluster logs, even after restart


for ((;;)) ; do fig logs ; done

tl;dr: infinite loops ftw

extra Docker networking goodies

I work a great deal with Docker, connecting webservers and databases together, hiding everything behind a caching proxy like Nginx.  Docker is a wonderful tool, providing isolation and semi-magical capabilities like hiding an entire "cluster" of machines behind a single container, a single process.

However containers are not much fun. They're generally so stripped down as to make debugging issues rather more awkward than normal.  The following questions are common when something doesn't work:

  • is my process running on the correct port?
  • what processes are running?
  • which services are up, on which ports?

With a default Docker container, answers to these questions aren't obvious.

Solution: add the following to all of your Dockerfiles:

RUN apt-get update && apt-get install -y apt-file curl net-tools procps psmisc
# 'psmisc' provides /bin/fuser; procps=>/bin/ps; net-tools=>netstat

Here's an example Nginx Dockerfile which 1) lets me verify Nginx is running on the right port and happy, and 2) when I hop into the container with "docker exec", I can use normal tools to see what's going on.

FROM nginx:1.7
RUN apt-get update && apt-get install -y apt-file curl net-tools procps psmisc
# 'psmisc' provides /bin/fuser; procps=>/bin/ps; net-tools=>netstat
ADD index.html /usr/share/nginx/html/index.html
ADD nginx.conf /etc/nginx.conf