ó ˽÷Xc@`s:ddlmZmZmZddlZddlZddlZddlmZddl Z ddl m Z ddddgZ ejd ƒZejd ƒZejd ƒZejd ƒZejd ƒZejdƒZejdƒZejdƒZejdƒZdefd„ƒYZdefd„ƒYZd„Zd„Zd„Zd„Zd„Zd„Z d„Z!d„Z"d„Z#d„Z$d„Z%d„Z&d „Z'd!„Z(d"„Z)de*fd#„ƒYZ+d$„Z,d%„Z-d&„Z.d'„Z/d(„Z0e1e0_2e3d)kr6ddl4Z4e4j5d*Z6e0e6ƒndS(+i(tdivisiontprint_functiontabsolute_importN(tpartial(tnexttMetaDatatloadarfft ArffErrortParseArffErrors^\s*@s^%s^\s+$s^@\S*s^@[Dd][Aa][Tt][Aa]s*^@[Rr][Ee][Ll][Aa][Tt][Ii][Oo][Nn]\s*(\S*)s/^@[Aa][Tt][Tt][Rr][Ii][Bb][Uu][Tt][Ee]\s*(..*$)s'(..+)'\s+(..+$)s(\S+)\s+(..+$)cB`seZRS((t__name__t __module__(((s5/tmp/pip-build-7oUkmx/scipy/scipy/io/arff/arffread.pyR7scB`seZRS((R R (((s5/tmp/pip-build-7oUkmx/scipy/scipy/io/arff/arffread.pyR;scC`sÖ|jƒjƒ}|ddkr&dS|tdƒ dkr@dS|tdƒ dkrZdS|tdƒ dkrtdS|tdƒ dkrŽdS|tdƒ dkr¨dS|td ƒ d krÂd Std |ƒ‚d S( s`Given an arff attribute value (meta data), returns its type. Expect the value to be a name.it{tnominaltrealtnumerictintegertstringt relationaltdatesunknown attribute %sN(tlowertstriptlenR(tattrtypet uattribute((s5/tmp/pip-build-7oUkmx/scipy/scipy/io/arff/arffread.pyt parse_typeEs cC`s |jdƒS(s5If attribute is nominal, returns a list of the valuest,(tsplit(t attribute((s5/tmp/pip-build-7oUkmx/scipy/scipy/io/arff/arffread.pyt get_nominal\scC`s\t|ƒg}|djƒddkr8tdƒ‚n|jg|D] }|^qEƒ|S(s4Read each line of the iterable and put it in a list.iR s0This looks like a sparse ARFF: not supported yet(RRt ValueErrortextend(tofiletdatati((s5/tmp/pip-build-7oUkmx/scipy/scipy/io/arff/arffread.pytread_data_listas  cC`s]t|ƒg}d}|djƒddkr>tdƒ‚nx|D]}|d7}qEW|S(s5Read the whole file to get number of data attributes.iiR s0This looks like a sparse ARFF: not supported yet(RRR(RR tlocR!((s5/tmp/pip-build-7oUkmx/scipy/scipy/io/arff/arffread.pyt get_ndatajs cC`s t|ƒ}td„|DƒƒS(s Given a string containing a nominal type definition, returns the string len of the biggest component. A nominal type is defined as seomthing framed between brace ({}). Parameters ---------- atrv : str Nominal type definition Returns ------- slen : int length of longest component Examples -------- maxnomlen("{floup, bouga, fl, ratata}") returns 6 (the size of ratata, the longest nominal value). >>> maxnomlen("{floup, bouga, fl, ratata}") 6 cs`s|]}t|ƒVqdS(N(R(t.0R!((s5/tmp/pip-build-7oUkmx/scipy/scipy/io/arff/arffread.pys Žs(t get_nom_valtmax(tatrvtnomtp((s5/tmp/pip-build-7oUkmx/scipy/scipy/io/arff/arffread.pyt maxnomlenus cC`sZtjdƒ}|j|ƒ}|rJtd„|jdƒjdƒDƒƒStdƒ‚dS(s©Given a string containing a nominal type, returns a tuple of the possible values. A nominal type is defined as something framed between braces ({}). Parameters ---------- atrv : str Nominal type definition Returns ------- poss_vals : tuple possible values Examples -------- >>> get_nom_val("{floup, bouga, fl, ratata}") ('floup', 'bouga', 'fl', 'ratata') s{(.+)}cs`s|]}|jƒVqdS(N(R(R%R!((s5/tmp/pip-build-7oUkmx/scipy/scipy/io/arff/arffread.pys ©siRs(This does not look like a nominal stringN(tretcompiletmatchttupletgroupRR(R(t r_nominaltm((s5/tmp/pip-build-7oUkmx/scipy/scipy/io/arff/arffread.pyR&‘s &cC`stjdƒ}|j|ƒ}|r|jdƒjƒ}d}d|krf|jddƒ}d}n|jddƒ}d}d|kr¥|jdd ƒ}d }nd |krÌ|jd d ƒ}d }nd|kró|jddƒ}d}nd|kr|jddƒ}d}nd|krA|jddƒ}d}nd|ksYd|krhtdƒ‚n|dkrƒtdƒ‚n||fStdƒ‚dS(Ns%[Dd][Aa][Tt][Ee]\s+[\"']?(.+?)[\"']?$ityyyys%YtYtyys%ytMMs%mtMtdds%dtDtHHs%Hthtmms%MR1tsss%StstztZs6Date type attributes with time zone not supported, yets"Invalid or unsupported date formatsInvalid or no date format(R+R,R-R/RtNonetreplaceR(R(tr_dateR1tpatternt datetime_unit((s5/tmp/pip-build-7oUkmx/scipy/scipy/io/arff/arffread.pytget_date_format®s>              cC`stjd„|ƒS(sWSkip header. the first next() call of the returned iterator will be the @data linecS`stj|ƒ S(N(t r_datametaR-(tx((s5/tmp/pip-build-7oUkmx/scipy/scipy/io/arff/arffread.pytÚs(t itertoolst dropwhile(R((s5/tmp/pip-build-7oUkmx/scipy/scipy/io/arff/arffread.pytgo_dataÖscC`s×|jƒ}tj|ƒ}|rŸ|jdƒ}tj|ƒr`t|ƒ\}}t|ƒ}q¯tj|ƒrt|ƒ\}}t|ƒ}q¯t dƒ‚nt d|ƒ‚|dkrÊt dƒ‚n|||fS(shParse a raw string in header (eg starts by @attribute). Given a raw string attribute, try to get the name and type of the attribute. Constraints: * The first line must start with @attribute (case insensitive, and space like characters before @attribute are allowed) * Works also if the attribute is spread on multilines. * Works if empty lines or comments are in between Parameters ---------- attribute : str the attribute string. Returns ------- name : str name of the attribute value : str value of the attribute next : str next line to be parsed Examples -------- If attribute is a string defined in python as r"floupi real", will return floupi as name, and real as value. >>> iterable = iter([0] * 10) # dummy iterator >>> tokenize_attribute(iterable, r"@attribute floupi real") ('floupi', 'real', 0) If attribute is r"'floupi 2' real", will return 'floupi 2' as name, and real as value. >>> tokenize_attribute(iterable, r" @attribute 'floupi 2' real ") ('floupi 2', 'real', 0) ismulti line not supported yetsFirst line unparsable: %sRs'relational attributes not supported yet( Rt r_attributeR-R/t r_comattrvalttokenize_single_commaRt r_wcomattrvalttokenize_single_wcommaR(titerableRtsattrtmattrR(tnamettypet next_item((s5/tmp/pip-build-7oUkmx/scipy/scipy/io/arff/arffread.pyttokenize_attributeàs)  cC`s€tj|ƒ}|rfy.|jdƒjƒ}|jdƒjƒ}Wqvtk rbtdƒ‚qvXntd|ƒ‚||fS(Niis Error while tokenizing attributes Error while tokenizing single %s(RMR-R/Rt IndexErrorR(tvalR1RTRU((s5/tmp/pip-build-7oUkmx/scipy/scipy/io/arff/arffread.pyRN!s cC`s€tj|ƒ}|rfy.|jdƒjƒ}|jdƒjƒ}Wqvtk rbtdƒ‚qvXntd|ƒ‚||fS(Niis Error while tokenizing attributes Error while tokenizing single %s(ROR-R/RRXR(RYR1RTRU((s5/tmp/pip-build-7oUkmx/scipy/scipy/io/arff/arffread.pyRP0s c C`st|ƒ}xtj|ƒr-t|ƒ}qWd}g}x½tj|ƒsùtj|ƒ}|rêtj|ƒ}|r¤t||ƒ\}}}|j||fƒqöt j|ƒ}|rË|j dƒ}nt d|ƒ‚t|ƒ}q=t|ƒ}q=W||fS(s&Read the header of the iterable ofile.isError parsing line %sN( Rt r_commentR-R@RFt r_headerlineRLRWtappendt r_relationR/R( RR!trelationt attributesR1tisattrRTRUtisrel((s5/tmp/pip-build-7oUkmx/scipy/scipy/io/arff/arffread.pyt read_header?s& cC`s!d|krtjSt|ƒSdS(svgiven a string x, convert it to a float. If the stripped string is a ?, return a Nan (missing value). Parameters ---------- x : str string to convert Returns ------- f : float where float can be nan Examples -------- >>> safe_float('1') 1.0 >>> safe_float('1\n') 1.0 >>> safe_float('?\n') nan t?N(tnptnantfloat(RG((s5/tmp/pip-build-7oUkmx/scipy/scipy/io/arff/arffread.pyt safe_floatas cC`sR|jƒ}||kr|S|dkr,|Stdt|ƒt|ƒfƒ‚dS(NRcs%s value not in %s(RRtstr(tvaluetpvaluetsvalue((s5/tmp/pip-build-7oUkmx/scipy/scipy/io/arff/arffread.pyt safe_nominal~s    cC`sm|jƒjdƒjdƒ}|dkr:tjd|ƒStjj||ƒ}tj|ƒjd|ƒSdS(Nt't"RctNaTsdatetime64[%s](RRdt datetime64tdatetimetstrptimetastype(Rit date_formatRDtdate_strtdt((s5/tmp/pip-build-7oUkmx/scipy/scipy/io/arff/arffread.pyt safe_dateˆs  cB`sDeZdZd„Zd„Zd„Zd„Zd„Zd„ZRS(sySmall container to keep useful informations on a ARFF dataset. Knows about attributes names and types. Examples -------- :: data, meta = loadarff('iris.arff') # This will print the attributes names of the iris.arff dataset for i in meta: print i # This works too meta.names() # Getting attribute type types = meta.types() Notes ----- Also maintains the list of attributes in order, i.e. doing for i in meta, where meta is an instance of MetaData, will return the different attribute names in the order they were defined. cC`s¹||_i|_g|_x—|D]\}}t|ƒ}|jj|ƒ|dkrr|t|ƒf|j|>> from scipy.io import arff >>> from cStringIO import StringIO >>> content = """ ... @relation foo ... @attribute width numeric ... @attribute height numeric ... @attribute color {red,green,blue,yellow,black} ... @data ... 5.0,3.25,blue ... 4.5,3.75,green ... 3.0,4.00,red ... """ >>> f = StringIO(content) >>> data, meta = arff.loadarff(f) >>> data array([(5.0, 3.25, 'blue'), (4.5, 3.75, 'green'), (3.0, 4.0, 'red')], dtype=[('width', '>> meta Dataset: foo width's type is numeric height's type is numeric color's type is nominal, range is ('red', 'green', 'blue', 'yellow', 'black') treadtrtN(thasattrtopent _loadarfftclose(tfR((s5/tmp/pip-build-7oUkmx/scipy/scipy/io/arff/arffread.pyRÓsF  c`s*yt|ƒ\}}Wn/tk rG}dt|ƒ}t|ƒ‚nXt}x5|D]-\}}t|ƒ}|dkrUt}qUqUWt||ƒ} itd6td6td6} it d6t d6t d6} g} g‰|sÕx|D]ê\}}t|ƒ}|dkrSt |ƒ\} }| j |d|fƒˆj t t d| d |ƒƒqä|d krªt|ƒ}| j |d |fƒt|ƒ}ˆj t td |ƒƒqä| j || |fƒˆj t ƒqäWn td ƒ‚tˆƒ‰d‡‡fd†}||ƒ}tj|| ƒ}|| fS(Ns'Error while parsing header, error was: RR RRRsdatetime64[%s]RtRDR sS%dRjs*String attributes not supported yet, sorryRc3`s†ttˆƒƒ}xm|D]e}tj|ƒstj|ƒrCqn|j|ƒ}tg|D]}ˆ|||ƒ^q\ƒVqWdS(N(tlisttrangeRZR-tr_emptyRR.(trow_itertdelimtelemstrawtrowR!(t convertorstni(s5/tmp/pip-build-7oUkmx/scipy/scipy/io/arff/arffread.pyt generator[s  (RbRRhRtFalseRtTrueRRfRgRER\RRwR*R&RltNotImplementedErrorRRdtfromiter(RR{R|teR€thasstrRTRiRUtmetat acls2dtypet acls2convtdescrRtRDtnRjR›taR ((R™Ršs5/tmp/pip-build-7oUkmx/scipy/scipy/io/arff/arffread.pyRŽ$sL               cC`sP|jd|jd}tj|ƒtj|ƒtj|ƒtj|ƒ|fS(Ngð?i(tsizeRdtnanmintnanmaxtmeantstd(R tnbfac((s5/tmp/pip-build-7oUkmx/scipy/scipy/io/arff/arffread.pyt basic_stats}sc C`sØ|d}|dks.|dks.|dkrit|ƒ\}}}}td||||||fƒnk|d}x8tt|dƒdƒD]} ||d| d7}qŽW||dd 7}|d 7}t|ƒdS( NiRR Rs%s,%s,%f,%f,%f,%fs,{iRiÿÿÿÿt}(R®tprintR’R( RTR}R RUtminR'R«R¬R€R!((s5/tmp/pip-build-7oUkmx/scipy/scipy/io/arff/arffread.pytprint_attribute‚s $# ! cC`s_t|ƒ\}}tt|jƒƒt|jƒx&|D]}t|||||ƒq9WdS(N(RR°RtdtypeR¨R²(tfilenameR R¢R!((s5/tmp/pip-build-7oUkmx/scipy/scipy/io/arff/arffread.pyt test_wekas   t__main__i(7t __future__RRRR+RIRqt functoolsRtnumpyRdtscipy._lib.sixRt__all__R,tr_metaRZR“R[RFR]RLRMROtIOErrorRRRRR"R$R*R&RERKRWRNRPRbRgRlRwtobjectRRRŽR®R²RµRœt__test__R tsystargvR´(((s5/tmp/pip-build-7oUkmx/scipy/scipy/io/arff/arffread.pytsX         ( A   "  B Q Y