Skip to content

Unhandled HTTPErrors When Negative Offset Not Supported #15

@StDymphna

Description

@StDymphna

When trying to use remotezip with PyPi, I discovered that while their server supports range requests, it does not support using a negative offset to get {content-length - bytes}-{content-length}.

When raise_for_status is called, an HTTP 501 error is returned (it could theoretically also be a 405) and remotezip aborts.

def _request(self, kwargs):
if self._session:
res = self._session.get(self._url, stream=True, **kwargs)
else:
res = requests.get(self._url, stream=True, **kwargs)
res.raise_for_status()

Reproducer:

from remotezip import RemoteZip

url = "https://files.pythonhosted.org/packages/71/6d/95777fd66507106d2f8f81d005255c237187951644f85a5bd0baeec8a88f/paramiko-2.12.0-py2.py3-none-any.whl"
  with RemoteZip(url) as wzip:
      wzip.extract('METADATA')

I was going to do a pull request, but not being super good with Python myself, I found it was not obviously fixible (to me at least) with a simple try/catch since it's going through constructors, etc.

Checking for a 501 or 405 error before raise_for_status and falling back to getting self.__file_size by a separate http request for content-length should fix this.

Something like:

def _request(self, kwargs): 
     if self._session: 
         res = self._session.get(self._url, stream=True, **kwargs) 
     else: 
         res = requests.get(self._url, stream=True, **kwargs) 
if res._status_code == 501 or 405:
    (do whatever needs to be done so that
    self._file_size = requests.get(self._url, stream=True).headers['Content-Length']
    )
res.raise_for_status()

Edit: I also submitted a bug/feature request to PyPI/warehoise about this on their server; I don't anticipate they will implement it quickly, but if they do, the given reproducer may not work.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions