Fast Parallel Downloading (for apt-get and aptitude) #3

"aptitude" has more features, but is as low-hassle and scriptable as "apt-get". For instance, you can say "this package was automatically installed", which means if you don't need it any longer, it'll be automatically removed. Another perk is a direct way of asking "why was/wasn't this package installed?", to help sort out sometimes confusing package dependencies.

Alas, it doesn't have the "print-uris" trick which I relied upon in doing quick upgrades. I've found a simple workaround: use both aptitude and apt-get. The following script updates the list of available packages. It uses apt-get to download -- but not install -- all updates. Lastly, it uses aptitude to upgrade the packages, using the freshly-downloaded files. A "safe" upgrade won't delete a package to let another package be installed correctly, it just does installs.

For a one-line version, see morch's a previous comment.

cd /var/cache/apt/archives/
aptitude update
apt-get -y --print-uris -u upgrade | grep -o -e "http://[^\']+" \
| xargs -r -l3 -P5 wget -nv
aptitude -y safe-upgrade

Bits of Evidence

Slideshow adding Science to "Computer Science". Includes references to a lot of myths like "best programmers are 45x better than the worst ones."

Bits of Evidence

View more presentations from Greg Wilson.

turning batch MapReduce into infinitely streaming data wrangling

Pipelining and Real-time Analytics with MapReduce Online

Comparison of Scrum, XP, Kanban, and other Agile Methodologies

State of Agile

Empirical Data behind Software Development Techniques

Our friends at Microsoft Research have published four papers testing--and measuring--development ideas. One big surprise was that a company's organization "can predict software failure-proneness with a precision and recall of 85 percent. " It's more important than using assertions or increasing test coverage!

Other results: test coverage isn't a direct measure of software quality, and assertions are useful.

I was glad to see some numbers behind what I've experienced, that Test Driven Development increases quality. "What the research team found was that the TDD teams produced code that was 60 to 90 percent better in terms of defect density than non-TDD teams. They also discovered that TDD teams took longer to complete their projects—15 to 35 percent longer." Hmm.

Exploding Software-Engineering Myths

Unit testing in Coders at Work

Very interesting essay on how master programmers (including Jamie Zawinski) write. Includes a TDD and traditional Sodoku solvers, with completely different implementations.

Unit testing in Coders at Work

Fast Parallel Downloading (for apt-get) #2

This is how to upgrade your Ubuntu system, downloading several packages in parallel. See previous article for details:

cd /var/cache/apt/archives/
apt-get -y --print-uris -u dselect-upgrade >debs.list
egrep -o -e "http://[^\']+" debs.list | xargs -l3 -P5 wget -nv
apt-get -u dselect-upgrade