CI Tip: block insecure changes to your source code

As a dev or devops person, it's pretty easy to accidentally expose AWS or other secrets by putting them into "temporary" code or environment files. Once a secret appears in version control, it should no longer be trusted, go ahead and rotate the key.

Here's how to write a checker, so that secrets won't make it into source code!  This is perfect for a Git commit hook. The important feature to use is Perl-style regular expressions, which allow you to say "match this exactly N number of times". Also, the "-w" flag to Ack lets us match word boundaries. Thus, if something is a word, and exactly some number of characters long, we flag it.

Example:

$ echo 'I love gin' | ack -w '[a-z]{3}'
I love gin

This example finds words of exactly three letters. The word "gin" is found, as it's exactly three characters. The second word "love" is not flagged, even though it has 3 characters included in it, because we added the "-w" flag to only find words. "Gin" has a non-letter on either side, so it's a word, so we match it.

To catch AWS keys, search for "words" of exactly 20 characters, uppercase letters and numbers only:

$ ack -lw '[0-9A-Z]{20}'
env-sales
web-services/src/config/configResolver.ts
.env

Ah: two local environment files have AWS keys, which is okay. These are not checked in to Git. However, a key snuck in to another file, in source code!  Now that Ack alerted me to a mislaid key, I can go fix it and make our system more secure. Win!

Sublime tips and tricks

https://generalassemb.ly/blog/sublime-text-3-tips-tricks-shortcuts/

Things I learned:

- ⌘-SHIFT-J = select everything that's similarly indented

- CTRL-⌘-UP = move selection up (vs dragging the region with the mouse)

- CTRL-SHIFT-K = delete entire current line. I used to do ⌘-L, Delete dozens of times a day. Now there's another way!

TIP: you can exclude file types when searching for stuff in Git!

Git isn't just a database, it's a comprehensive searching system XX. You can exclude file types when searching for stuff in Git!

Example: the Development branch broke some time between two Git commits. I want to find the error but don't want to look at huge piles of changes. I don’t think the front end Vue files broke the build, so let's find changes excluding those files, so I can find the error.

First, I list the files between the two commits, to see if it’s roughly what I want:

git diff --stat c7b5d2d^..2de4986 -- ':(exclude)*.vue'

Looks good. Next, I list the actual content by zapping the "--stat" argument:

^--stat^

Or: git diff c7b5d2d^..2de4986 -- ':(exclude)*.ts'

I found the problem! Yay, Git!

More info:
- Git pathspec
- Bash "Event Designators" (the upcaret to search and replace previous line)

console.log macro in Sublime Text

Here's how to make a selection macro in Sublime Text. 

Type the word "beer", select it, then press a key to turn it into a console.log statement!

Here's how to do it:

1. Create the console.log template:

Click Tools > Developer > New Snippet...

Paste the following:

<![CDATA[
console.log("$SELECTION");
]]>
log

Save the above as log.sublime-snippet

2. Create a keybinding to the snippet:

Click Sublime Text > Preferences > Key Bindings

Paste this bit into the middle of the keymap:

 {

  "keys": ["ctrl+shift+l"], 

  "command": "insert_snippet", 

  "args": { "name": "Packages/User/log.sublime-snippet" } },

Save it.

To use, type beer in your program. Double-click to select it, then hit your "log this message" keyboard command ctrl+shift+l.

Boom!

console.log("beer");

quality interactive tests with Lynx, the server-side browser

Working on a web page test on a server, but can't view the page in a normal web browser? It's easy enough to transfer the HTML to your local computer, or... use a text-mode browser!

The Lynx browser is tiny, fast, and supports a good part of modern web pages. It's a great tool for quickly getting a rough idea of what your page looks like.

Example: my Flask app refuses to handle authentication correctly. So, I write a API test to log in to myserver, capture the response, then write it to a server-side HTML file.  Next I use Lynx to view it so I can see what it's doing!

Here's my Registration test:

def test_post(client):
    "validate page that uses POST"
    rv = client.post('/auth/register', data=dict(
        email='test@example.com',
        username='test',
        password='test',
    ))
    if 1:
        with open('zoot.html', 'w') as pagef:
            pagef.write(rv.data)

Run the test

pytest

Next, use the server-side browser to view my output:

lynx -dump -stdin < zoot.html

Output:

   (BUTTON) Toggle navigation [1]test
     * [2]Home

     * [3]Log In

Register

   Email test@example.com____
   Username test________________
   Password ____________________

   Passwords must match.
   Confirm password ____________________

   This field is required.
   Register

Note the "Passwords must match" bit. Ah, there's my bug. In the POST I'd forgotten to send both the password and validating password2 fields.  Bug fixed, thanks to Lynx!

Terraform and iTerm2 FTW

1) install imgcat from iTerm2 page
2) brew install graphviz

Now, when you're working on a complex set of Terraform dependencies, you can display them directly in your terminal!

terraform graph | dot -Tpng | imgcat

(Code from the excellent book Terraform: Up and Running by Yevgeniy Brikman.)

pretty titles (and cats) in your terminal!

A little-known feature of iTerm2 is that you can insert images directly into your (previously text only) terminal!  This can be useful (render calculation as plot directly in terminal) or we can use it for silly purposes.

Cats. Who on the interwebs doesn't love cats?

Setup

brew install imagemagick
copy iTerm2's imgcat script into your path

Test: display built-in image

convert rose: - | imgcat /dev/stdin

Pretty Title

convert -background lightblue -fill blue  -font Tahoma -size 165x70  label:Anthony    z.png && imgcat z.png

Good Job, Bub!

curl -s http://thinga.com/assets/uploads/images/Lil-Bub-pic-02.jpg | imgcat /dev/stdin

Note the "Tahoma" font used above is probably only on macOS.  Choose another font that's installed into ImageMagick.  What fonts are those, you say?  Here you go:

List ImageMagick Fonts (Awk FTW)

$ convert -list font | awk '/Font:/ {print $2} {next}' | pr -4t
AndaleMono   BigCaslonM     Herculanum      TimesNewRomanBI
AppleChancery   BrushScriptI     HoeflerTextOrname TimesNewRomanI
AppleMyungjo   Chalkduster     Impact       Trebuchet
Arial   ComicSans     InaiMathi       TrebuchetMSB
ArialB   ComicSansMSB     Kokonor       TrebuchetMSBI
ArialBI   CourierNew     Krungthep       TrebuchetMSI
ArialBk   CourierNewB     MicrosoftSansSeri Verdana
ArialI   CourierNewBI     PlantagenetCherok VerdanaB
ArialNarrow   CourierNewI     Sathu       VerdanaBI
ArialNarrowB   GB18030Bitmap     Silom       VerdanaI
ArialNarrowBI   Georgia     Skia       Webdings
ArialNarrowI   GeorgiaB     Tahoma       Wingdings
ArialRoundedB   GeorgiaBI     TahomaB       Wingdings2
ArialUnicode   GeorgiaI     TimesNewRoman     Wingdings3
Ayuthaya   Gurmukhi     TimesNewRomanB    Zapfino