How to speed up an already cached pip install?

Posted by Maxime R. on Stack Overflow See other posts from Stack Overflow or by Maxime R.
Published on 2012-09-13T15:17:20Z Indexed on 2012/09/15 9:38 UTC
Read the original article Hit count: 279

Filed under:
|
|

I frequently have to re-create virtual environments from a requirements.txt and I am already using $PIP_DOWNLOAD_CACHE. It still takes a lot of time and I noticed the following:

Pip spends a lot of time between the following two lines:

Downloading/unpacking SomePackage==1.4 (from -r requirements.txt (line 2))
  Using download cache from $HOME/.pip_download_cache/cached_package.tar.gz

Like ~20 seconds on average to decide it's going to use the cached package, then the install is fast. This is a lot of time when you have to install dozens of packages (actually enough to write this question).

What is going on in the background? Are they some sort of integrity checks against the online package?

Is there a way to speed this up?

edit: Looking at:

time pip install -v Django==1.4

I get:

real    1m16.120s
user    0m4.312s
sys     0m1.280s

The full output is here http://pastebin.com/e4Q2B5BA. Looks like pip is spending his time looking for a valid download link while it already has a valid cache of http://pypi.python.org/packages/source/D/Django/Django-1.4.tar.gz.

Is there a way to look for the cache first and stop there if versions match?

© Stack Overflow or respective owner

Related posts about python

Related posts about virtualenv