˽Xc@`sHdZddlmZmZmZddlZddlZddlZddlZddl m Z ddl Z ddl Z ddlmZmZddlZddlmZddlmZmZdd lmZmZmZmZmZmZmZm Z m!Z!dd l"m#Z#dd l$m%Z%m&Z&m'Z'm(Z(m)Z)m*Z*m+Z+m,Z,m-Z-m.Z.m/Z/m0Z0m1Z1m2Z2m3Z3m4Z4m5Z5dd l6m7Z7d efdYZ8dZ9de:fdYZ;dZ<e'eddZ=e'eddZ>e'eddZ?e'eddZ@de:fdYZAde:fdYZBdS(s Classes for read / write of matlab (TM) 5 files The matfile specification last found here: http://www.mathworks.com/access/helpdesk/help/pdf_doc/matlab/matfile_format.pdf (as of December 5 2008) i(tdivisiontprint_functiontabsolute_importN(tBytesIO(tasbytestasstr(t string_typesi(t native_codet swapped_code( t MatFileReadert docfillertmatdimst read_dtypet arr_to_charstarr_dtype_numbert MatWriteErrort MatReadErrortMatReadWarning(t VarReader5(t MatlabObjecttMatlabFunctiontMDTYPESt NP_TO_MTYPESt NP_TO_MXTYPESt miCOMPRESSEDtmiMATRIXtmiINT8tmiUTF8tmiUINT32t mxCELL_CLASStmxSTRUCT_CLASStmxOBJECT_CLASSt mxCHAR_CLASStmxSPARSE_CLASStmxDOUBLE_CLASSt mclass_info(tZlibInputStreamtMatFile5Readerc B`szeZdZed eeeeeed dZdZdZ dZ dZ edZ d dZ dZRS( s Reader for Mat 5 mat files Adds the following attribute to base class uint16_codec - char codec to use for uint16 char arrays (defaults to system default codec) Uses variable reader that has the following stardard interface (see abstract class in ``miobase``:: __init__(self, file_reader) read_header(self) array_from_header(self) and added interface:: set_stream(self, stream) read_full_tag(self) c C`s_tt|j||||||||| s@tj} n| |_d|_d|_dS(sInitializer for matlab 5 file format reader %(matstream_arg)s %(load_args)s %(struct_arg)s uint16_codec : {None, string} Set codec to use for uint16 char arrays (e.g. 'utf-8'). Use system default codec if None N( tsuperR%t__init__tsystgetdefaultencodingt uint16_codectNonet _file_readert_matrix_reader( tselft mat_streamt byte_ordert mat_dtypet squeeze_metchars_as_stringstmatlab_compatibletstruct_as_recordt verify_compressed_data_integrityR*((s3/tmp/pip-build-7oUkmx/scipy/scipy/io/matlab/mio5.pyR's  cC`sH|jjd|jjd}|jjd|dkrDdpGdS(s4 Guess byte order. Sets stream pointer to 0 i~iitIMt(R/tseektread(R.tmi((s3/tmp/pip-build-7oUkmx/scipy/scipy/io/matlab/mio5.pytguess_byte_orderscC`s~i}t|jdd}t|j|}|djjd|d<|dd?}|dd@}d ||f|d <|S( s Read in mat 5 file header tdtypest file_headert descriptions t __header__tversioniis%d.%dt __version__(RR0R R/titemtstrip(R.thdictt hdr_dtypethdrtv_majortv_minor((s3/tmp/pip-build-7oUkmx/scipy/scipy/io/matlab/mio5.pytread_file_headerscC`s"t||_t||_dS(sa Run when beginning read of variables Sets up readers from parameters in `self` N(RR,R-(R.((s3/tmp/pip-build-7oUkmx/scipy/scipy/io/matlab/mio5.pytinitialize_readscC`s|jj\}}|dks0tdn|jj|}|tkrt|j|}|jj||j }|jj\}}nt }|jj|j|t kst d|n|jj |}||fS(s Read header, return header, next position Header has to define at least .name and .is_global Parameters ---------- None Returns ------- header : object object that can be passed to self.read_var_array, and that has attributes .name and .is_global next_position : int position in stream of next variable isDid not read any bytess$Expecting miMATRIX type here, got %d(R,t read_full_tagt ValueErrorR/ttellRR$R-t set_streamR6tFalseRt TypeErrort read_header(R.tmdtypet byte_counttnext_poststreamtcheck_stream_limittheader((s3/tmp/pip-build-7oUkmx/scipy/scipy/io/matlab/mio5.pytread_var_headers    cC`s|jj||S(s Read array, given `header` Parameters ---------- header : header object object with fields defining variable header process : {True, False} bool, optional If True, apply recursive post-processing during loading of array. Returns ------- arr : array array with post-processing applied or not according to `process`. (R-tarray_from_header(R.RYtprocess((s3/tmp/pip-build-7oUkmx/scipy/scipy/io/matlab/mio5.pytread_var_arraysc C`st|tr|g}n|d k r6t|}n|jjd|j|j}g|d>> import scipy.io BytesIO is from the ``io`` module in python 3, and is ``cStringIO`` for python < 3. >>> mat_fileobj = BytesIO() >>> scipy.io.savemat(mat_fileobj, {'b': np.arange(10), 'a': 'a string'}) >>> varmats = varmats_from_mat(mat_fileobj) >>> sorted([name for name, str_obj in varmats]) ['a', 'b'] iR>R?(R%R:RRtitemsizeR;RLRKRORdRZRReRtwriteRk( tfile_objtrdrthdr_lentraw_hdrRoRpt named_matststart_positionRHReRUtvar_strtout_obj((s3/tmp/pip-build-7oUkmx/scipy/scipy/io/matlab/mio5.pytvarmats_from_matNs,)            tEmptyStructMarkercB`seZdZRS(s= Class to indicate presence of empty matlab struct on output (R~RR(((s3/tmp/pip-build-7oUkmx/scipy/scipy/io/matlab/mio5.pyRscC`sut|tjr|S|dkr&dSt|doPt|doPt|d}| rt|drtd|jjD}t}n|r(g}g}x\|jD]N\}}t|t r|ddkr|j |t f|j |qqW|r!tj t |g|StSntj|}|jjt tjfkrq|jd krq||krqdS|S( s Convert input object ``source`` to something we can write Parameters ---------- source : object Returns ------- arr : None or ndarray or EmptyStructMarker If `source` cannot be converted to something we can write to a matfile, return None. If `source` is equivalent to an empty dictionary, return ``EmptyStructMarker``. Otherwise return `source` converted to an ndarray with contents for writing to matfile. tkeystvaluestitemst__dict__cs`s0|]&\}}|jds||fVqdS(t_N(t startswith(t.0tkeytvalue((s3/tmp/pip-build-7oUkmx/scipy/scipy/io/matlab/mio5.pys s it _0123456789N((RbtnptndarrayR+thasattrtdictRRRhRRktobjecttarrayttupleRt asanyarraytdtypettypetobject_R{(tsourcet is_mappingRRtfieldRtnarr((s3/tmp/pip-build-7oUkmx/scipy/scipy/io/matlab/mio5.pyt to_writeables2  R>R?ttag_fullt tag_smalldatat array_flagst VarWriter5cB`seZdZejdeZeed||d<|jdd|d<|j|dS(Nitbyte_count_mdtypeRRtdata((Rtzerost NDT_TAG_SMALLRR(R.RRTRUttag((s3/tmp/pip-build-7oUkmx/scipy/scipy/io/matlab/mio5.pyRscC`sotjdt}||d<||d<|j||j||d}|rk|jjdd|ndS(NRTRUit((RRt NDT_TAG_FULLRRR(R.RRTRURtbc_mod_8((s3/tmp/pip-build-7oUkmx/scipy/scipy/io/matlab/mio5.pyRs     ic C`s|j}|j}|jj|_|j|jtjdt }t |d|d>B|d>B} || d>B|d<||d<|j||j tj |d d tj |}|d kr|j|td n|j |td |_t|_d S(s Write header for given data options shape : sequence array shape mclass - mat5 matrix class is_complex - True if matrix is complex is_logical - True if matrix is logical nzmax - max non zero elements for sparse arrays We get the name and the global flag from the object, and reset them to defaults after we've used them t data_typeiRUiiit flags_classtnzmaxRti4R`iN((RRRROt _mat_tag_posRtmat_tagRRtNDT_ARRAY_FLAGSRRRtasarrayRRRQ( R.R{Ryt is_complexRwRReRjtaftflags((s3/tmp/pip-build-7oUkmx/scipy/scipy/io/matlab/mio5.pyt write_headers$        cC`sy|jj}|jj|||d}|dkrHtdn||jd<|j|j|jj|dS(Niii s-Matrix too large to save with Matlab 5 formatRUI(RROR:RRR(R.t start_postcurr_posRU((s3/tmp/pip-build-7oUkmx/scipy/scipy/io/matlab/mio5.pytupdate_matrix_tag.s  cC`s#||_||_|j|dS(s Write variable at top level of mat file Parameters ---------- arr : array_like array-like object to create writer for name : str, optional name as it will appear in matlab workspace default is empty string is_global : {False, True}, optional whether variable will be global on load into matlab N(RRR(R.RReRj((s3/tmp/pip-build-7oUkmx/scipy/scipy/io/matlab/mio5.pyt write_top9s  cC`s_|jj}tjj|r?|j||j|dSt|}|dkrvt d|t |fnt |t r|j |nt |trtdn|tkr|jn|jjr|j|nf|jjr|j|nJ|jjdkrA|jr(d}nd}|j||n |j||j|dS( s Write `arr` to stream at top and sub levels Parameters ---------- arr : array_like array-like object to create writer for Ns'Could not convert %s (type %s) to arraysCannot write matlab functionstUtStUTF8tascii(RR(RROtscipytsparsetissparset write_sparseRRR+RRRRbRt write_objectRRRtwrite_empty_structRtfieldst write_structt hasobjectt write_cellstkindRt write_chart write_numeric(R.Rt mat_tag_posRtcodec((s3/tmp/pip-build-7oUkmx/scipy/scipy/io/matlab/mio5.pyRMs4            cC`s|jjdk}|jjdk}yt|jjd}WnVtk r|rd|jd}n'|r||jd}n|jd}t}nX|jt||j |d|d||r|j |j |j |j n |j |dS( Ntctbitc128ti1tf8RRw( RRRRtKeyErrortastypeR"RR RRtrealtimag(R.RtimagftlogifRy((s3/tmp/pip-build-7oUkmx/scipy/scipy/io/matlab/mio5.pyRus&  Rc C`s<|jdks$tj|dkrgd tj|jdg}|j|t|j|tddSt |}|j }|j|t|j j dkr%|jr%tj |}tjdd dt||d|jj}|jj|}tjdt|fdd d|}n|j|d tdS( s5 Write string array `arr` with given `codec` iR`iNRR{RtbuffertS1RT(i((RRtalltmaxtndimRR RRR R{RRtproductRRtTtcopyRDtencodeRmR(R.RRR{tn_charstst_arrtst((s3/tmp/pip-build-7oUkmx/scipy/scipy/io/matlab/mio5.pyRs$$    c C`s|j}|j|jjdk}|jjdk}|j}|jt||jtd|d|d|dkr|dn||j |j j d|j |j j d|j |j j|r|j |j jnd S( s Sparse matrices are 2D RRRRwRiiRN(ttocsct sort_indicesRRtnnzRR RR!RtindicesRtindptrRRR(R.RtARRwtnz((s3/tmp/pip-build-7oUkmx/scipy/scipy/io/matlab/mio5.pyRs   cC`sV|jt||jttj|jd}x|D]}|j|q;WdS(NR(RR RRRt atleast_2dtflattenR(R.RRtel((s3/tmp/pip-build-7oUkmx/scipy/scipy/io/matlab/mio5.pyRs  cC`sR|jdt|jtjddtj|jtjgdtjdS(NiR(ii(RRRRRtint32tint8(R.((s3/tmp/pip-build-7oUkmx/scipy/scipy/io/matlab/mio5.pyRscC`s-|jt||jt|j|dS(N(RR RRt _write_items(R.R((s3/tmp/pip-build-7oUkmx/scipy/scipy/io/matlab/mio5.pyRsc C`sg|jjD]}|d^q }tg|D]}t|^q-d}|jr[dp^d}||krtd|dn|jtj|gdd|jtj|dd|d t tj |j d }x0|D](}x|D]}|j ||qWqWdS( Niii@i s+Field names are restricted to %d charactersRRsS%dRTR( RtdescrRRmRRNRRRRR R R( R.Rtft fieldnamest fieldnametlengtht max_lengthRR ((s3/tmp/pip-build-7oUkmx/scipy/scipy/io/matlab/mio5.pyRs#)   cC`sR|jt||jt|jtj|jdddt|j |dS(smSame as writing structs, except different mx class, and extra classname element after header RRRTN( RR RRRRRt classnameRR(R.R((s3/tmp/pip-build-7oUkmx/scipy/scipy/io/matlab/mio5.pyRs (N(R~RRRRRRRR'RRR+RRRRQRRRRRRRRRRRR(((s3/tmp/pip-build-7oUkmx/scipy/scipy/io/matlab/mio5.pyRs.      &  (  +     tMatFile5WritercB`sAeZdZeeededdZdZddZRS(s Class for writing mat5 files trowcC`sU||_||_||_|r-||_n g|_||_||_d|_dS(s< Initialize writer for matlab 5 format files Parameters ---------- %(do_compression)s %(unicode_strings)s global_vars : None or sequence of strings, optional Names of variables to be marked as global for matlab %(long_fields)s %(oned_as)s N(Rtdo_compressionRt global_varsRRR+t_matrix_writer(R.RRRRRR((s3/tmp/pip-build-7oUkmx/scipy/scipy/io/matlab/mio5.pyR's       cC`s~tjd t}dtjtjf|ds6B       @p E 10