ó ¾÷Xc@@sdZddlmZddlmZddlZddlZddlZddlZddlZddl Z ddl m Z ddl m Z ddl mZdd lmZejddkrÓddd „Zndd l mZedd d „Zd„ZdS(s(Utilities for file download and caching.i(tabsolute_import(tprint_functionN(turlopen(tURLError(t HTTPErrori(tProgbarcC@sdddd„}t||ƒ}t|dƒ1}x'||d|ƒD]}|j|ƒqCWWdQXdS(sâReplacement for `urlretrive` for Python 2. Under Python 2, `urlretrieve` relies on `FancyURLopener` from legacy `urllib` module, known to have issues with proxy management. # Arguments url: url to retrieve. filename: where to store the retrieved data locally. reporthook: a hook function that will be called once on establishment of the network connection and once after each block read thereafter. The hook will be passed three arguments; a count of blocks transferred so far, a block size in bytes, and the total size of the file. data: `data` argument passed to `urlopen`. i cs@sˆ|jƒjdƒjƒ}t|ƒ}d}xT|j|ƒ}|d7}|sc||||ƒPn|r|||||ƒn|Vq0dS(NsContent-Lengthii(tinfotgettstriptinttread(tresponset chunk_sizet reporthookt total_sizetcounttchunk((s5/tmp/pip-build-isqEY4/keras/keras/utils/data_utils.pyt chunk_read$s  twbR N(tNoneRtopentwrite(turltfilenameR tdataRR tfdR((s5/tmp/pip-build-isqEY4/keras/keras/utils/data_utils.pyt urlretrieves (RtdatasetscC@sÚtjjtjjddƒƒ}tj|tjƒsNtjjddƒ}ntjj||ƒ}tjj|ƒs…tj|ƒn|r­tjj||ƒ}|d}ntjj||ƒ}t} tjj|ƒr|d k rt ||ƒs t dƒt } q qnt } | rt d|ƒd } d d„} d} y‘y#t ||tj| d | ƒƒWngtk r›} t| j|| j| jƒƒ‚n4tk rÎ} t| j|| j| jƒƒ‚nXWn>ttfk r} tjj|ƒr tj|ƒn‚nXd } n|rÖtjj|ƒsÒt d ƒtj|d ƒ}y|jd |ƒWn`ttfk rÄ} tjj|ƒr¾tjj|ƒr®tj|ƒq¾tj|ƒn‚nX|j ƒn|S|S(sñDownloads a file from a URL if it not already in the cache. Passing the MD5 hash will verify the file after download as well as if it is already present in the cache. # Arguments fname: name of the file origin: original URL of the file untar: boolean, whether the file should be decompressed md5_hash: MD5 hash of the file for verification cache_subdir: directory being used as the cache # Returns Path to the downloaded file t~s.kerass/tmps.tar.gzsBA local file was found, but it seems to be incomplete or outdated.sDownloading data fromcS@s0|dkrt|ƒ}n|j||ƒdS(N(RRtupdate(Rt block_sizeRtprogbar((s5/tmp/pip-build-isqEY4/keras/keras/utils/data_utils.pyt dl_progressgs s!URL fetch failure on {}: {} -- {}RsUntaring file...sr:gztpathN(!tosR!t expandusertjointaccesstW_OKtexiststmakedirstFalseRt validate_filetprinttTrueRt functoolstpartialRt ExceptiontformatterrnotreasonRtcodetmsgtKeyboardInterrupttremovettarfileRt extractalltisfiletshutiltrmtreetclose(tfnametorigintuntartmd5_hasht cache_subdirt datadir_basetdatadirt untar_fpathtfpathtdownloadRR t error_msgtettfile((s5/tmp/pip-build-isqEY4/keras/keras/utils/data_utils.pytget_file:sd!      $)   cC@sgtjƒ}t|dƒ }|jƒ}|j|ƒWdQXt|jƒƒt|ƒkr_tStSdS(sÒValidates a file against a MD5 hash. # Arguments fpath: path to the file being validated md5_hash: the MD5 hash being validated against # Returns Whether the file is valid trbN( thashlibtmd5RR Rtstrt hexdigestR,R)(RER@thashertftbuf((s5/tmp/pip-build-isqEY4/keras/keras/utils/data_utils.pyR*s  (t__doc__t __future__RRR-R7R"tsysR:RLtsix.moves.urllib.requestRtsix.moves.urllib.errorRRtutils.generic_utilsRt version_infoRRR)RJR*(((s5/tmp/pip-build-isqEY4/keras/keras/utils/data_utils.pyts$      $T