When trying to use remotezip with PyPi, I discovered that while their server supports range requests, it does not support using a negative offset to get {content-length - bytes}-{content-length}.
When raise_for_status is called, an HTTP 501 error is returned (it could theoretically also be a 405) and remotezip aborts.
|
def _request(self, kwargs): |
|
if self._session: |
|
res = self._session.get(self._url, stream=True, **kwargs) |
|
else: |
|
res = requests.get(self._url, stream=True, **kwargs) |
|
res.raise_for_status() |
Reproducer:
from remotezip import RemoteZip
url = "https://files.pythonhosted.org/packages/71/6d/95777fd66507106d2f8f81d005255c237187951644f85a5bd0baeec8a88f/paramiko-2.12.0-py2.py3-none-any.whl"
with RemoteZip(url) as wzip:
wzip.extract('METADATA')
I was going to do a pull request, but not being super good with Python myself, I found it was not obviously fixible (to me at least) with a simple try/catch since it's going through constructors, etc.
Checking for a 501 or 405 error before raise_for_status and falling back to getting self.__file_size by a separate http request for content-length should fix this.
Something like:
def _request(self, kwargs):
if self._session:
res = self._session.get(self._url, stream=True, **kwargs)
else:
res = requests.get(self._url, stream=True, **kwargs)
if res._status_code == 501 or 405:
(do whatever needs to be done so that
self._file_size = requests.get(self._url, stream=True).headers['Content-Length']
)
res.raise_for_status()
Edit: I also submitted a bug/feature request to PyPI/warehoise about this on their server; I don't anticipate they will implement it quickly, but if they do, the given reproducer may not work.
When trying to use remotezip with PyPi, I discovered that while their server supports range requests, it does not support using a negative offset to get {content-length - bytes}-{content-length}.
When raise_for_status is called, an HTTP 501 error is returned (it could theoretically also be a 405) and remotezip aborts.
python-remotezip/remotezip.py
Lines 184 to 189 in 2afca9a
Reproducer:
I was going to do a pull request, but not being super good with Python myself, I found it was not obviously fixible (to me at least) with a simple try/catch since it's going through constructors, etc.
Checking for a 501 or 405 error before raise_for_status and falling back to getting self.__file_size by a separate http request for content-length should fix this.
Something like:
Edit: I also submitted a bug/feature request to PyPI/warehoise about this on their server; I don't anticipate they will implement it quickly, but if they do, the given reproducer may not work.