TIP: work with Amazon AWS ECS Container Service Docker containers

When running Amazon EC2 Container Service (ECS), have these two commands running in Tmux on your instances:

tail -f $(ls -1t /var/log/ecs/ecs-agent.log.* | head -1)

sudo tail -f /var/log/upstart/docker.log


In another Tmux tab, this command lets you look at the logs from your running Docker container:

docker logs -f $(docker ps --filter name=myname,status=running --quiet)

TIP: fix bizarre Docker problems

docker rm -fv $(docker ps -aq)

Sometimes testing produces strange Docker-related problems.  The above command kills all containers, including the stopped ones.  The "-v" also destroys each container's private storage volumes.  This is great because Docker likes to run differently different times you run it -- because it has local state.  Local state is evil!  Run the above command every now and then so Docker starts from a clean, state-less, state.

In Docker, stopping a container is different from killing it.  The former stops the running container/process, but leaves the private volume.  The latter zaps the volume.

Thanks Sam!

talk: Platonic Solids of Quality

(This page is bit.ly/jta-platonic )

Slides: John Tells All: Platonic Solids of Quality

Video: Platonic Solids of Quality, John Tells All

Detailed outline on Workflowy

Thanks to Don Westland for the A/V, Marcel for organizing and the picture, DataScience Inc for the hosting, and Keith B. for the presentation and the smooooth scotch.

TIP: auto-fix your Python code!

The script autopep8 will automatically fix and reindent your Python code!

The following shows a "diff" of what PEP8 changes should be made, and then we apply the changes:

$ autopep8 -d collect.py
@@ -29,5 +29,5 @@

-if __name__=='__main__':
+if __name__ == '__main__':
     main()

$ autopep8 -i collect.py

reach your next-level DevOps Career

I often get questions about how to get into development or how to progress in a dev/webdev/devops career.  Here's my most recent "From Dev to DevOps" ideas:

- become very familiar with doing container-centric development.  At the moment "Docker Compose" will get you comfortable with the terminology and mechanics of containers.  It's extremely useful for local dev work. It's also very similar to Docker Cloud, and leads pretty directly to Kubernetes and Amazon EC2 Container Service (ECS).

- learn one or more orchestration softwares.  Chef is the biggest, Puppet is cool, Salt Stack is the new hotness.  My favorite is Ansible: it's human-readable, flexible, fast, and obvious.  The other day I wrote a "create or update a distributed S3 bucket, and verify it has the correct permissions" in 7 lines of code!

- Amazon AWS is a pain in the ass to learn. However, there are tons of really excellent talks, so you can teach yourself quite a bit. You don't have to do Amazon, but it's what 95% of the startups use.

- Invest in O'reilly Safari Online ($40/month?) to read all their books and watch all their videos.  I'll post my favorites in another article. Start with "Agile Testing" and "Continuous Delivery."

- DevOps can really accelerate development by creating and maintaining a strong Continuous Integration system and Testing culture in general.  Run don't walk to read "Agile Testing" by Lisa Crispin. I steal her stuff for about half of my talks. (If you saw my "Python Testing" talk last year, there was a bit in there borrowed from the book).

- tell your current employer that your career is moving into DevOps (/ Amazon). They're getting a good deal in you if you stay there and contribute, so they'll start investing in you by sending you to conferences and/or training you up.

- Run don't walk to go to all Meetups you're even vaguely interested in.  I do Unix Users Group (UUASC, heavy servers) and LA DevOps (these are listed on the sidebar.) My favorite is the Socal Python group. It's super friendly and wide-ranging.

- enjoy the ride!