问题描述:
Traceback (most recent call last):
File "C:\python\iCrawler.py", line 44, in
<module>
crawler(sites)
File "C:\python\iCrawler.py", line 24, in
crawler
resp,content =
h.request(site,'GET')
File
"C:\python\lib\site-packages\httplib2\__init__.py", line 1089, in
request
(response, content) =
self._request(conn, authority, uri, request_uri, method, body,
headers, redirections, cachekey)
File
"C:\python\lib\site-packages\httplib2\__init__.py", line 874, in
_request
(response, content) =
self._conn_request(conn, request_uri, method, body, headers)
File
"C:\python\lib\site-packages\httplib2\__init__.py", line 859, in
_conn_request
content =
_decompressContent(response, content)
File
"C:\python\lib\site-packages\httplib2\__init__.py", line 318, in
_decompressContent
content =
zlib.decompress(content)
zlib.error: Error -3 while decompressing data: incorrect
header check
解决办法:
修改:File
"C:\python\lib\site-packages\httplib2\__init__.py", line 318, in
_decompressContent
改为:content =
zlib.decompress(content,16+zlib.MAX_WBITS)
结果:失败
改为:content =
zlib.decompress(content, -zlib.MAX_WBITS)
结果:成功!
原因:大概是传输的网页是经过压缩的,在解压缩的过程中,zlib处理网页头部是发生错误。
加载中,请稍候......