Thursday, March 24, 2016

UPDATED: quickly download lots of Python packages

This trick downloads Python packages up to 9x faster than normal:

egrep -o '^([A-Za-z].*==[^ ]+)' requirements.txt | xargs -n1 -P9 pip download

After things are downloaded, actually build and install the packages:

pip install -r requirements.txt

EDIT: original code gave `egrep: Invalid range end` -- fixed. Also added "-n1" to xargs so it'll download in parallel, vs sequentially.

No comments:

Post a Comment