Wednesday, August 22, 2018

resources for learning Golang for professional developers

I'm re-learning Go. I've been a professional developer for a long time and have delivered production software in 10+ languages, so mastering one more language is really, really not a big deal :)

However, learning in 2018 can be bumpy, there's tons of noise and distractions. My normal practice for learning is "buy three books and build a project". Books are written and edited by experienced people, have a through-line to aid understanding, and have a consistent tone. Books have errors, that are fixed with published errata. Books are great.

My current best spot for tech books is O'reilly Safari Books. It's $40/month, and has zillions of books and training videos! (225 are on Golang)

If you're a professional developer you don't need to spend two chapters on variables and types and for loops and such. An excellent "get dropped in the weeds and learn" tool is a Cookbook -- a dense catalog of small, real-world programs. For Golang, I'm reading "Go Cookbook" by Aaron Torres, Packt Publishing. I've read other Packt books and find they're clear and dense enough to chew into. (The Python Cookbook by Jones and Beazley is spectacular. After programming Python for 20 years I still found new things to learn every few pages.)

Here are other, high quality, free resources for learning Go if you're already a developer:

https://golang.org/doc/effective_go.html - valuable introduction to the "Go"-ness of the language, compared to C.

http://www.javiersaldana.com/tech/2015/01/22/loose-coupling-in-go-lang.html - Saldana has a number of small practical idiomatic Go programs, used as a discussion of larger programming design patterns.

Friday, July 20, 2018

TIP: Bash has “global search and replace”!

TIP: Bash has “global search and replace”! It works with the history mechanism. Example, the bangbang (!!) command repeats the previous command:

$ !! # repeat previous command

Adding a colon (:) and then a letter or two will modify the command before running it.  A useful modifier is "p" for printing.  That is:

$ !!:p # repeat previous command, but just :p-print it

This is useful because you can use up-arrow to now go to the previous command and edit it interactively.

For non-interactive editing, you can do global search and replace!  Example: use the "repeat previous command" command, bangbang (!!). Then modify it (:), then say "global search" (gs).  To do this to find "one" and replace it with "two", use this command:

$ !!:gs/one/two

In my real-world case, I'd already run a command to deploy my Development server with Terraform. The specific command is:

AWS_PROFILE=development terraform plan -var-file=../config/development.tfvars

This command was in my shell history.  I want to recall this command (bangbang, aka !!), then search and replace "development" with "staging" to use Terraform to deploy to my staging environment.

Command:

$ !!:gs/development/staging

Got me:

AWS_PROFILE=staging terraform plan -var-file=../config/staging.tfvars

Resources:


Bash scripting cheatsheet

running My Traceroute (aka Matt's traceroute) (MTR) on macOS


Mtr is a wonderful program that combines ping and traceroute. It shows you each hop along a path to another host on the internet, and how long each hop takes.  It's my #1 go-to tool to debug wifi / networking / DNS issues. And, it's pretty!

Anyway it requires extra privileges, so it's a bit fiddly to run. Even worse, The Internet Is Wrong on this topic, there's lots of bad advice.

Here's how to install and run mtr on a macOS machine:

brew install mtr

PATH=$PATH:/usr/local//Cellar/mtr/0.92/sbin sudo mtr 8.8.8.8

The "8.8.8.8" is a magic IP. Easy to remember, it's a public DNS router that our friends at Google make available to the public. You can use any IP or domain name here. I use the all-8s IP, because sometimes my DNS isn't working, so pinging a raw IP will tell me if my DNS is acting up, and if so, which one.

Here's what it looks like:



If you press d, it switches displays to more visual. This lets the "bad actors" in the network jump out:


Since this is a terminal-based CLI program it's easy to install and run on a server. Maybe your local network is good, but the server's network or DNS is acting up -- mtr will make issues really easy to see and fix!

Thursday, July 19, 2018

tech book recommendations

Recently I was asked about Python books covering Object Oriented programming for someone coming from another language. Here are some resources:

- I recommend subscribing to Safari Books Online. They have jillions of books and videos, including "Python Beyond The Basics - Object Oriented Programming" (high-ranked video). It's $40/month. If you're a professional Dev or DevOps, or trying to be, it's an easy investment.

- the class section of "Modern Python Cookbook" (book) has tons of real-world Python idioms: designing classes with lots vs little processing, classes with __slots__, and advanced class design... Actually this book looks great I'm going to read it.

- David Beazley's "Python Cookbook" is great, overflowing with real-world problems, solutions, and discussion.

- in the Los Angeles region all the Lynda.com tech resources are freeee with an LA public library card.

- the "Cookbook" books I find are good for someone coming from another language, the books don't spend 100 pages talking about dictionaries and strings and so forth. It turns out there's tons of Cookbooks nowadays: data visualization, machine learning, testing, you name it!

Friday, July 6, 2018

Kafka on macOS

Generally I run everything in Docker: it's less fiddly, and I can do a clean uninstall very easily. Alas Docker networking is different, and changes every few months as Docker makes things easier... by changing the networking.

As of July 2018 here's the easiest way I've found to run Kafka on macOS:

brew install kafka kafkacat zookeeper
brew services start zookeeper
brew services start kafka

Once Kafka is up, list out the brokers:

$ kafkacat -L -b localhost
Metadata for all topics (from broker -1: localhost:9092/bootstrap):
 1 brokers:
  broker 0 at 192.168.0.133:9092
 0 topics:

Now let's go to the mysterious directory that has tons of good tools. Create a topic, then use "kafkacat" again to verify our new topic has been created:

$ cd /usr/local/Cellar/kafka/*/libexec/bin
./kafka-topics.sh --create  --topic example-topic --zookeeper localhost:2181 --partitions 1 --replication-factor 1

$ kafkacat -L -b localhost
Metadata for all topics (from broker -1: localhost:9092/bootstrap):
 1 brokers:
  broker 0 at 192.168.0.133:9092
 1 topics:
  topic "example-topic" with 1 partitions:
    partition 0, leader 0, replicas: 0, isrs: 0

Woot! My thanks to springheeledjak and pkafel!

Saturday, June 23, 2018

Quality Code in Practice

Developers like to write code. They like to type fast, to work fast, and deploy fast. This is fine. We all like this, it's fun!
A more high-level view is that as a business, as a team we want to deliver value fast. This is harder: which value to what audience are we serving? If we deliver buggy code to the client, no one is happy. If we make some users happy but our core audience is not served, then we haven't really delivered on our goals. Our goal as professionals, as entrepreneurs, is to find/create an audience, and serve that audience by giving them value.
In practice, "clean" code helps us work fast and also deliver value to our audience. Clean in this case meaning free of bugs, clearly designed, and of a design which directly meets the needs of our audience.
Automated workflow tools can help us move forward. They run very quickly, giving us positive feedback that what we write is legible and missing the most egregious errors. The basic categories of tools are 1) surface-level syntax, 2) language-centric analysis, 3) project-based unit and other tests.
For a recent project, this is our top-level Makefile.  It lets us as developers get rapid feedback on the quality of our project. Also, having a single file lets these tasks be repeatable by a Continuous Integration system, so we can be confident things don't break if we had too much coffee and committed something without running tests.

all:

auto-fix:
git ls-files '*.py' \
| xargs autopep8 --in-place --aggressive

lint:
flake8

clean:
find . -name '*.pyc' | xargs rm

run:
cd cj ; ./manage.py run

test:
pytest

Monday, June 18, 2018

Easy Docker

This talk was given at the LA Django meetup in DTLA on 6/19/2018. My thanks to Anthony at Gridspace for sponsoring! Also thanks to Marcel for inviting me again.

This page is http://bit.ly/jta-easy-docker

The slides are in Googledocs, source code is in Github => https://github.com/johntellsall/easy-docker