Xc@`sdZddlmZmZmZddlmZmZmZddl Z ddl Z ddl Z ddl Z ddl Z ddlZddlZddlZddlZddlZddlmZddlmZddlmZmZmZmZddlmZdd lmZm Z m!Z!m"Z"m#Z#dd l$m%Z%dd l&m'Z'd d l(m)Z*e j+dZ,dgZ-dZ.de/fdYZ0de0fdYZ1dZ2dZ3de0e4fdYZ5dfdYZ6de/fdYZ7de0fdYZ8e9dZ:dZ;d e/fd!YZ<d"e<fd#YZ=d$e<fd%YZ>e?fd&Z@d'e<fd(YZAd)eAfd*YZBd+e<fd,YZCd-e<fd.YZDd/e<fd0YZEd1fd2YZFd3e0fd4YZGd5eGfd6YZHd7ZId8ZJd9eGfd:YZKd;fd<YZLd=ZMd>eGfd?YZNd@ZOdAZPdBZQdCZRdDdEdFZSdS(Gsm Defines the base class for optimizations as well as a certain amount of useful generic optimization tools. i(tabsolute_importtprint_functiontdivision(tdequet defaultdictt OrderedDictN(tconfig(tizip(t string_typest iteritemst itervaluest integer_types(treduce(tgraphtoptutilstunifyttoolbox(tInconsistencyError(t OrderedSeti(tdestroyhandlerstheano.gof.optcC`sttj|j|jS(N(tlistR t io_toposorttinputstoutputs(tfgraph((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyt_list_of_nodes#st OptimizercB`steZdZdZdZdZdZdZdZdZ e j dd d Z e dd ZRS( s An L{Optimizer} can be applied to an L{FunctionGraph} to transform it. It can represent an optimization or in general any kind of transformation you could apply to an L{FunctionGraph}. cC`s6t|ds/td|_tdcd7s(tdhtDestroyHandlerRFR9(RMRGRN((RQs./tmp/pip-build-X4mzal/theano/theano/gof/opt.pytinplace_optimizers   t SeqOptimizercB`sweZdZedZdZdZdZdZe j dddZ edd Z ed Z RS( sU Takes a list of L{Optimizer} instances and applies them sequentially. cC`s|tjdt|tjdtjtjtjdkrO|n)tjdkrxtjt j dndS(s= Default failure_callback for SeqOptimizer. sSeqOptimizer apply %ss Traceback:traisetpdbiN( t_loggerterrorRKt tracebackt format_excRt on_opt_errorRXt post_mortemRCtexc_info(texcRRO((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pytwarns  cO`spt|dkr8t|dttfr8|d}n||(|jdd|_t|dksltdS(s Parameters ---------- *opts : The List of optimizers to be applied to a node failure_callback : callable or None Keyword only argument. A callback used when a failure happen during optimization. iitfailure_callbackN(tlent isinstanceRttupletpopR6RbtAssertionError(Rtoptstkw((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRIs + c C`sg}|jr6|jj}|g}|jj}n g}g}|j}t|j}g}g} x|D]} yt|j} tj} | j|} |j t tj| |j | | j | t|jf|jr|j |jjnWqmt k rqmt k rS}|j rM|j ||| qmqTqmXqmW|jr|jj|}i}xlt|jD]L\}}||kr|||}|dkr||| Applies each L{Optimizer} in self in turn. iN(tprofilet validate_timetexecute_callbacks_timestcopytexecute_callbacks_timeRct apply_nodesttimeR0tappendtfloatRgt ExceptionRbR R6(RRtltvalidate_beforetsub_validate_timetcallbacks_beforetcallback_beforetnb_node_beforet sub_profstnb_nodesROtnb_nodes_beforett0tsub_profteRktcallbacks_timetktvttt callback_time((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyR$sX                cC`sdtj|S(Ns SeqOpt(%s)(RRL(R((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRLscC`s tj|S(N(Rt__repr__(R((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRsiicC`st|dd}tdd||jj|t|fd||dkr|d8}x.|D]#}|j|d|dd |q_WndS( NR2s %s%s %s id=%iR3R4iiR;iR<(R5R6R7R8R9R R=(RR:R;R<R2topt((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyR=s*   c C`s|\ }}}}}}}} } } d|} t| dddd|t|drut| |jddd|n.t|drt| |jddd|ntdt|||fd|t| d |d|t| d |d||d krst| d d|x]tt| d dD]=} | d dkr/t| d| dd| d d|q/q/Wn|dkrt| dd|ng}xpt|| D]_\}}t|dr|j}n |j}|j|}|j ||j j|f|qWtt||d d}x|dddD]\}}|d} | r| | d | | }t| d|||fd|nt| d||fd||| r@|| j ||| d|d q@q@Wtd|dS(Ns RVtendR3R4R2R9s5 time %.3fs for %d/%d nodes before/after optimizations %.3fs for callbacks! %.3fs for fgraph.validate()is callbacks_timetkeycS`s |d S(Ni((ta((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRR3sis t,sM time - (name, class, index, nodes before, nodes after) - validate timecS`s|dS(Ni((R((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRRFsiis %.6fs - %s - %.3fss %.6fs - %sR;( R7RR2R9tsumtsortedR tziptindexRqR8R@(R:R?R;RhRkRRyt nb_node_afterRzRvR{RtblanctitllRtnb_nR2tidxtlllRtval_time((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyR@sR$  ".        c C`sg}g}g}xt|djt|dD]}|dj|}|dj|}|j|d||d||j|t|drt|d|t|d|kst|j|j|d||d|q6|jd q6Wddl m }xt|dj t|dD]}g|D]} | j ^qK} |j | kr| j|j } |} |} |j | || j | | j| jkr>||dkr|}n|}|| c|d|dj|7q>n||dkr|}n|}|j|d|dj||dj|} |j||j|d| q>Wt|}g}xPt|d|dD]7\}}|j|d|d|d|dfq<W|j|dt||j|dt|t|d|d}tg|dD]}|j ^qjtg|D]}|j ^qsttg|dD]}|j ^q#jtg|D]}|j ^qEsftt|t|kot|knst|||d|d|d |d d d |g||f S( sF Merge 2 profiles returned by this cass apply() fct. iit merge_profilei(tStringIOii iiiN(tsett intersectionRRqRRcRgRR6tsixRtsymmetric_differenceR2R=treadRVRtextendt merge_dicttissubset(tprof1tprof2tnew_ttnew_ltnew_sub_profileRttidx1tidx2Rtot new_l_namesRtio1tio2tptnew_optt new_nb_nodestp1tp2tnew_callbacks_times((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRVsv* .*    %,  "  $/&+&+4 (R9RARBRERaRIR$RLRRCRDR=R@R(((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRVs  :   6t _metadictcB`sVeZdZdZdZdZdZdZdZdZ dZ RS( s WRITEME cC`si|_g|_dS(N(tdRt(R((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRIs cC`s|j|dS(N(tgetR6(Rtitem((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyt __getitem__scC`sy||j|sRtnew_nodei(RR+RRcRRgtownerRdRR&R'RtAsserttTruetrangeRRqRRtallRRRtget_merged_assert_inputR2RR(RRRtnode_has_assertRRRtmerge_candidatest_tassert_clientsRtclienttreplacement_candidatest candidatetcand_has_asserttcand_inputs_assert_removedtnode_inputs_assert_removedt inputs_matchtpairst new_inputsRtpairt node_outputt cand_output((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRTs ,.                     c C`sg}xt|j|jD]\}}|jrt|jjtjjjr|jrt|jjtjjjr|jjd}|jjd}t t ||}|j tjjj |jjd|q|j |q|j |qW|S(Nii( RRRRdRR&R'RRRRRqt assert_op( RRRRtnode_itcand_it node_condt cand_condtnew_cond((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRs$"       ( R9RARBRRRRRRR(((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRs $    tMergeOptimizercB`sJeZdZdZdZdZeddZedZRS(s Merges parts of the graph that are identical and redundant. The basic principle is that if two Applies have ops that compare equal, and identical inputs, then they do not both need to be computed. The clients of one are transferred to the other and one of them is removed from the graph. This procedure is carried out in input->output order through the graph. The first step of merging is constant-merging, so that all clients of an int(1) for example, are transferred to a particular instance of int(1). cC`s&t|ds"|jtndS(NR(RRPR(RR((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyR%sc C`s|jj}d}tj}|jrN|jj}|j}|jj}nd}d} x|r|j} t } x| D]} | d\} }}|dkrt | drn&t | d s|t |d rq|ng| D]}|d ^q}| j r|j r| j }|j }g}x_|j D]T}|j rqt |j jtjjjrq|j|j j dq*|j|q*Wg}x_|j D]T}|j rt |j jtjjjr|j|j j dq|j|qW|dkrt }ntdt||D}|s*q|nt |ddjdr|ddj|ddj}tg|D]K\}}|dkrnt |jd rn|tj|jjjk^qndkrq|qqnt|dkrU|ddj|ddjkrU|ddjj|dd}|sU|dd|ddfg}qUny|j |d WnPt!k rt"} |d7}|jj#j|ddj |ddj fnX| r||t|7}t |ddt$j%r| d7} nPq|q|Wq]W|jr|jj|}|j|}i}xrt&|jD]L\}}||kr|||}|dkr|||Sstdestroy_handleritoutputt destroy_mapR((RRRpRjRkRnRlRmRfRRRRRdRR&R'RRRqRRRRRRtflattenRtvaluesRcttypetconvert_variabletreplace_all_validateRR+RR RR R6( RRtschedtnb_failR}RuRxRwt nb_mergedt nb_constantt pairs_listtsuccesstpairs_tvarRt merge_modeRRRRRRRRRtresRkRRRRR((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyR$s               B 4"%   '       cC`s |jjS(N(R8R9(R((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRLsic C`s|\}}}}}}} d|} t| dd|t| d||| fd|t| d|||fd||dkrt| dd|x]tt|dd D]=} | dd krt| d | d d | dd|qqWndS( Ns RR4s% nb fail=%5d merged=%5d constant=%5ds2 time replace=%2.2f validate=%2.2f callback=%2.2fis callbacks_timeRcS`s|dS(Ni((R((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRRsis R(R7RR ( R:R?R;R t replace_timeRkRRR R RR((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyR@s    "c C`sd}|d|d}|d|d}||d|d}||d|d}t|d|d}|d|d}|d|d} ||||||| fS( NcS`s(|dkr|S|dkr |S||S(N(R6(tv1tv2((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pytmerge_none_numbers   iiiiiii(R( RRRR RRkRRR R ((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRs  ( R9RARBR%R$RLRER@R(((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRs   ~ c C`s3|dkri}ntj|||g}|dd!}|d}tjjj|}tjjj||dt }x*t |D]\}}|j ||qWt j |g|D]} |j| | ^q} g| D]} | j^q\} } | dkr%| dkr%| d| dkS| | kSdS(s Merge-based implementation of `theano.gof.graph.is_same_graph`. See help on `theano.gof.graph.is_same_graph` for additional documentation. iitcloneiN(R6RmtdeepcopyR&tgofR Rtfgt FunctionGraphR+R treplaceRR0RR( tvar1tvar2tgivenstcopiedtvarsRRt to_replacet replace_byRt vars_replacedto1to2((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pytis_same_graph_with_merges    %"c`sUtit|tjr-|g}nfdtt|S(s Merge constants in the subgraph used to compute nodes in `vars`. `vars` is a list of nodes, and we want to merge together nodes that are constant inputs used to compute nodes in that list. Notes ----- This function will ignore nodes that are in an fgraph. It is used to pre-merge nodes generated inside an optimization, before it is inserted in the fgraph. It is useful if there are many such replacements to make, so that DebugMode will not check each of them. c`s|kr|St|ds#|S|jrBt|jdrB|Sj|t|tjr|j}y"|kr|S|| to use in place of `node`'s outputs in the greater graph. - dict(old variables -> new variables). A dictionary that map from old variables to new variables to replace. Parameters ---------- node : an Apply instance t transformN(RtMethodNotDefinedRR8R9(RR((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyR14s cC`sdS(s If this local optimization wants to add some requirements to the fgraph, this is the place to do it. N((RR((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyR%KsiicC`s1tdd||jjt|fd|dS(Ns %s%s id=%iR3R4(R7R8R9R (RR:R;R<((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyR=Us( R9RARBRR0R1R%RCRDR=(((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyR/s    tLocalMetaOptimizercB`sJeZdZdddZdZdZdZdZdZ RS( s Base class for meta-optimizers that try a set of LocalOptimizers to replace a node and choose the one that executes the fastest. cC`s+||_t||_tjj|_dS(N(t_tracksRt optimizersRtmetaopttverbose(RR0R5((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRIas cC`s|jj|dS(N(R5Rq(RRO((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pytregisterfscC`s|jS(N(R4(R((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyR0isc `sjdk r1t|jtjs1dSni}t}x|jD]z}t|tjj rhqJt |j drtj |j j|j j|jd|jdt||sis* %s: exceptions* %s: %.5g secs* %s: not applicables= %siii(&R4R6RdRReRRR&tcompiletSharedVariableRttagtsharedRtfilterR9R2R:RRtupdatetprovide_inputstdifference_updatetkeysR7R7R8R9RctninR5R1tfunctiontminRRsRqtsort( RRRtmissingtinputttimingsRRttimingR((RHRs./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyR1lsb    & # )      cC`s tdS(s If implemented, returns a dictionary mapping all symbolic variables in ``inputs`` to SharedVariable instances of suitable dummy values. The ``node`` can be inspected to infer required input shapes. N(R>(RRR((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyREscC`s!tj}|tj|S(N(Rp(RRHtstart((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyR>s N(( R9RARBR6RIR8R0R1RER>(((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyR3Zs   > tFromFunctionLocalOptimizercB`sMeZdZdd dZdZdZdZej dddZ RS( s WRITEME cC`s||_||_||_dS(N(R1R4RG(RRHR0RG((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRIs  cC`s"x|jD]}||q WdS(N(RG(RRRJ((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyR%scC`s|jS(N(R4(R((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyR0scC`st|ddS(NR9s%(R5(R((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRLs iicC`s4tdd|t|jt|fd|dS(Ns %s%s id=%iR3R4(R7RKR1R (RR:R;R<((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyR=s N(( R9RARBR6RIR%R0RLRCRDR=(((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRQs    c`sfd}|S(Nc`sdk rtdkr9td|j|jnxPD]E}t|tjpgt|tj s@td|j|jq@q@Wn}rt j t fdf}nt ||}|j|_|S(s WRITEME is8Use None instead of an empty list to apply to all nodes.s"Tracks are op classes or instancesc`s|jS(N(RP(R(RQ(s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRRsN(R6Rct ValueErrorRAR9RdRtOpt issubclasstPureOpRSRTReRQ(RMRRJRN(tinplaceRGR0(RQs./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyt decorators  $"   ((R0RVRGRW((RVRGR0s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pytlocal_optimizerst LocalOptGroupcB`skeZdZdZdZdZdZeddZdZ e j ddd Z d Z RS( sTakes a list of LocalOptimizer and applies them to the node. Parameters ---------- optimizers : The List of optimizers to be applied to a node reentrant : bool (Default True) Keyword only argument. Reentrant information. Some global optimizer like NavigatorOptimizer can use this value to determine if it ignore new nodes during a pass on the nodes. Sometimes, ignore_newtrees is not reentrant. apply_all_opts : bool (Default False) If False, it will return after the new node after the first optimizer applied. Otherwise, it will start again with the new node until no new optimization apply. cO`st|dkr8t|dtr8t|d}n||_t|jtsYttd|D|_td|D|_ |j dt |_ |j dt |_ td|_t|dkst|j ri|_i|_i|_i|_nx|jD]}|j rw|jj|d|jj|d|jj|d|jj|dnx(|jD]}|j|j|qWqWdS(Niics`s!|]}t|dtVqdS(t reentrantN(R5R(RR((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pys scs`s!|]}t|dtVqdS(tretains_inputsN(R5R+(RR((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pys stapply_all_optsRjcS`sgS(N((((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRRs(RcRdRReRhRgtanyRZRR[RfR+R\RjRt track_mapt time_optst process_countt applied_truet node_createdt setdefaultR0Rq(RR5R-RR((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRIs2%         c C`s9t|dddjg|jD]}t|^qS(NR9sLocalOptGroup(%s)R(R5tjoinRhRK(RR((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRL s cC`s@g}x3|jD](}|j}|r|j|qqW|S(N(RhR0R(RRRtttt((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyR0%s  c C`st|jdkrdS|j}d}xtr|jt|j|j|j|jd}d}x|D]}tj}|j |}tj}|j r|j |c||7<|j |cd7(RR((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRusicC`s{tdd||jjt|fd||dkrw|d8}x1|jD]#}|j|d|dd|qMWndS( Ns %s%s id=%iR3R4iiR;iR<(R7R8R9R RhR=(RR:R;R<tlopt((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyR=xs '  cC`s%x|jD]}|j|q WdS(N(RhR%(RRR((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyR%s(R9RARBRIRLR0R1RER@RRCRDR=R%(((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRYs   !& tGraphToGPULocalOptGroupcB`s eZdZdZdZRS(sThis is the equivalent of LocalOptGroup for GraphToGPU. The main different is the function signature of the local optimizer that use the GraphToGPU signature and not the normal LocalOptimizer signature. apply_all_opts=True is not supported cO`s2tt|j|||jtks.tdS(N(tsuperRvRIR\R+Rg(RR5R-((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRIsc C`st|jdkrdS|dj}|jt||j||jd}x|D]}tj}|j||||} tj} |jr|j |c|| 7<|j |cd7 add(div(x, y), add(y, x)) -> sub(div(x, y), sub(y, x)) cC`s||_||_||_dS(N(top1top2t transfer_tags(RRzR{R|((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRIs  cC`s|jS(N(Rz(R((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pytop_keyscC`s |jgS(N(Rz(R((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyR0scC`s|j|jkrtS|jj|j}|jrtj|j|_x;t |j |j D]!\}}tj|j|_q_Wn|j S(N( RRzR+R{t make_nodeRR|RmRARR(RRRhRt new_output((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyR1s "cC`sd|j|jfS(Ns%s -> %s(RzR{(R((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRLs( R9RARBR+RZRR[RIR}R0R1RL(((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRys    tOpRemovecB`sVeZdZeZdZdZdZdZdZ e j dddZ RS( sq Removes all applications of an op by transferring each of its outputs to the corresponding input. cC`s ||_dS(N(R(RR((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRIscC`s|jS(N(R(R((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyR}scC`s |jgS(N(R(R((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyR0scC`s|j|jkrtS|jS(N(RR+R(RR((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyR1scC`s d|jS(Ns %s(x) -> x(R(R((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRLsiicC`s=tdd||jjt|jt|fd|dS(Ns%s%s(%s) id=%iR3R4(R7R8R9RKRR (RR:R;R<((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyR=s   ( R9RARBR+RZRIR}R0R1RLRCRDR=(((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRs     t PatternSubcB`sqeZdZed d ed d d dZdZdZedZ dZ dZ e j ddd ZRS( s @todo update Replaces all occurrences of the input pattern by the output pattern: input_pattern ::= (op, , , ...) input_pattern ::= dict(pattern = , constraint = ) sub_pattern ::= input_pattern sub_pattern ::= string sub_pattern ::= a Constant instance sub_pattern ::= int sub_pattern ::= float constraint ::= lambda fgraph, expr: additional matching condition output_pattern ::= (op, , , ...) output_pattern ::= string output_pattern ::= int output_pattern ::= float Each string in the input pattern is a variable that will be set to whatever expression is found in its place. If the same string is used more than once, the same expression must be found in those places. If a string used in the input pattern is used in the output pattern, the matching expression will be inserted in its place. The input pattern cannot just be a string but the output pattern can. If you put a constant variable in the input pattern, there will be a match iff a constant variable with the same value and the same type is found in its place. You can add a constraint to the match by using the dict(...) form described above with a 'constraint' key. The constraint must be a function that takes the fgraph and the current Variable that we are trying to match and returns True or False according to an arbitrary criterion. The constructor creates a PatternSub that replaces occurrences of in_pattern by occurrences of out_pattern. Parameters ---------- in_pattern The input pattern that we want to replace. out_pattern The replacement pattern. allow_multiple_clients : bool If False, the pattern matching will fail if one of the subpatterns has more than one client. skip_identities_fn : TODO name Allows to override this optimizer name. pdb : bool If True, we invoke pdb when the first node in the pattern matches. tracks : optional The values that self.tracks() will return. Useful to speed up optimization sometimes. get_nodes : optional If you provide `tracks`, you must provide this parameter. It must be a function that takes the tracked node and returns a list of nodes on which we will try this optimizer. Notes ----- `tracks` and `get_nodes` can be used to make this optimizer track a less frequent Op, so this will make this optimizer tried less frequently. Examples -------- PatternSub((add, 'x', 'y'), (add, 'y', 'x')) PatternSub((multiply, 'x', 'x'), (square, 'x')) PatternSub((subtract, (add, 'x', 'y'), 'y'), 'x') PatternSub((power, 'x', Constant(double, 2.0)), (square, 'x')) PatternSub((boggle, {'pattern': 'x', 'constraint': lambda expr: expr.type == scrabble}), (scrabble, 'x')) c C`s||_||_| |_t|ttfrC|jd|_n2t|tri|jdd|_n td|j j dt |d|_ ||_ ||_ |r||_n||_||_||_|dkr|stndS(NitpatternsAThe pattern to search for must start with a specific Op instance.s This instance does: s ((t in_patternt out_patterntvalues_eq_approxRdRReRtdictRR8RBRKtallow_multiple_clientstskip_identities_fnR9RXR4t get_nodesRg( RRRRRR2RXR0RR((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRIMs$    !       cC`s|jS(N(R(R((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyR}iscC`s |jdkr|jS|jgS(N((R4R(R((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyR0lsc`so|rjdk rxj|D]}|dkr=q%nj|dt}|tk r%|dk r%t|jt|kstjrj|j_nt t |j|Sq%Wn|j j krtSttfdj |j tjtj}|rgfdj}||}jr`j|j_n|gStSdS(s Checks if the graph from node corresponds to in_pattern. If it does, constructs out_pattern and performs the replacement. RRc `sfd}tttfrjdkrFtSjjdk sy rtjdkr|Stdtjj kr|Sxt djj D].\}}||j stSqWntt ryd}Wn!t k r8t dnXjdd} | ry|jdS|Snttrtj}||k r|k r|Sj|ntttfr9ttjr9tjtjjjjkr/S|Sn>ttjrpttjrpjrpS|S|rddl}|jnS( Nc`sBjs tSj}|dkr,tS|dS(NR(RR+R6(t expr_equiv(RtexprtmatchRRtu(s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pytretry_with_equivs  iiRs.Malformed pattern: %s (expected key 'pattern')t constraintcS`stS(N(R(R((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRRsR( RdRReRR6R+RRcRRRRRRRRRtVarRR RrR RtnpRR&R'R)RRRXt set_trace( RRRRRXRRRt real_patternR(RR(RRRRs./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRsZ "&        '   c`st|ttfrIg|dD]}||^q }|d|St|tri|tj|St|ttfr|S|jSdS(Nii( RdRReRRRR RrR(RRRR,(tbuild(s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRs&N(RR6R1R+RcRRgRRARRRRtoutRt UnificationRRXR(RRRt real_nodeR/RR((RRRs./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyR1qs. ! >   c`sHt|ddr|jSfdd|j|jfS(NR9c`st|ttfrSdt|ddjg|dD]}|^q6fSt|trd|dt|jddfSt|SdS( Ns%s(%s)is, is%s subject to %sRRs no conditions(RdRReRKRdRR(RR(tpattern_to_str(s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRs . s%s -> %s(R5R6R9RR(R((Rs./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRLs   cC`s t|S(N(RK(R((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRsiic C`sjt|dt|dd}tdd||jj|t|jt|jt|fd|dS(NR9R2s%s%s %s(%s, %s) id=%iR3R4( R5R6R7R8R9RKRRR (RR:R;R<R2((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyR=s   N((R9RARBR+R6RIR}R0RR1RLRRCRDR=(((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRsO     h  tUpdatercB`sAeZddZdZdZdZdZdZRS(cC`s(||_||_||_||_dS(N(timportertprunertchinR2(RRRRR2((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRIs   cC`sdt|jS(Ns Updater{%s}(RKR2(R((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRLscC`s|jr|j|ndS(N(R(RRRR((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyR s cC`s|jr|j|ndS(N(R(RRRR((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRs cC`s)|jr%|j|||||ndS(N(R(RRRRRRR((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRs cC`sd|_d|_d|_dS(N(R6RRR(RR((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyt on_detachs  N( R9RAR6RIRLRRRR(((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRs      tNavigatorOptimizercB`seZdZedZedZedZdd dZd d dZ dZ d dZ d Z e jd d d ZRS(s Abstract class. Parameters ---------- local_opt A LocalOptimizer to apply over a FunctionGraph (or None is Ok too). ignore_newtrees - True: new subgraphs returned by an optimization is not a candidate for optimization. - False: new subgraphs returned by an optimization is a candidate for optimization. - 'auto': let the local_opt set this parameter via its 'reentrant' attribute. failure_callback A function that takes (exception, navigator, [(old, new), (old,new),...]) and we call it if there's an exception. If the trouble is from local_opt.transform(), the new variables will be 'None'. If the trouble is from validation (the new types don't match for example) then the new variables will be the ones created by transform(). If this parameter is None, then exceptions are not caught here (raised normally). cC`stjdkr`tjdt|tjdt|tjdtjtjntjdkrtjt j dn't |t stjdkr|ndS( sL Failure_callback for NavigatorOptimizer: print traceback. R=sOptimization failure due to: %ssnode: %ss TRACEBACK:RXiRWN( RR]RYRZRKR[R\RXR^RCR_RdRg(R`tnavt repl_pairst local_optR((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRa;s cC`s,t|trdStj|||||S(s Failure_callback for NavigatorOptimizer. Ignore InconsistencyErrors, print traceback. If error during replacement repl_pairs is set. Otherwise None. N(RdRRRa(R`RRRR((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyt warn_inplaceMs cC`sdS(sN Failure_callback for NavigatorOptimizer: ignore all errors. N((R`RRRR((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyt warn_ignore[stautocC`sD||_|dkr.t|dt |_n ||_||_dS(NRRZ(RR5Rtignore_newtreesRb(RRRRb((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRIcs    cC`sW|jrd}n|dkr.|dkr.dSt|||d|}|j||S(s Install some FunctionGraph listeners to help the navigator deal with the ignore_trees-related functionality. Parameters ---------- importer Function that will be called whenever optimizations add stuff to the graph. pruner Function to be called when optimizations remove stuff from the graph. chin "on change input" called whenever a node's inputs change. name name of the Updater to attach. Returns ------- object The FunctionGraph plugin that handles the three tasks. Keep this around so that you can detach later! R2N(RR6RRP(RRRRRR2R((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pytattach_updaterls   cC`s |dk r|j|ndS(s Undo the work of attach_updater. Parameters ---------- u A return-value of attach_updater. Returns ------- None N(R6tremove_feature(RRR((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pytdetach_updaters c C`sq|p |j}y|j|}Wn^tk r}|jdk r||j||g|jD]}|df^qY||tSnX|tks|dkrtS|j}g}t|trd|kr|j d}nt |j }t |j }n.t|t t fs0td||fnt|t|kr[td|nxMt||D]<\} } | dkrkt| jdkrktdqkqkWgt||D]0\} } | | k r| dk r| | f^q} t| dkrtSy!|j| d|d|tSWnBtk rl}|jdk rf|j||| ||tSnXdS(sH This function will use `lopt` to `transform` the `node`. The `transform` method will return either False or a list of Variables that are intended to replace `node.outputs`. If the fgraph accepts the replacement, then the optimization is successful, and this function returns True. If there are no replacement candidates or the fgraph rejects the replacements, this function returns False. Parameters ---------- fgraph A FunctionGraph. node An Apply instance in `fgraph` lopt A LocalOptimizer instance that may have a better idea for how to compute node's outputs. Returns ------- bool True iff the `node`'s outputs were replaced in the `fgraph`. tremovesKOptimizer %s gave wrong type of replacement. Expected list or tuple. Got %ss.Optimizer %s gave wrong number of replacementsis9A local optimizer tried to remove a Variable that is usedRN(RR1RsRbR6RR+RdRRfRRGRReRRcRRRRtreplace_all_validate_removeR( RRRRut replacementsRtxtold_varsRRtrnewR((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRsT     !* cC`s6tt|j||jr2|jj|ndS(N(RwRR%R(RR((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyR%s iicC`sdtdd||jjt|fd||dkr`|jj|d|dd|dndS( Ns %s%s (%i)R3R4iR;iR<i(R7R8R9R RR=(RR:R;R<((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyR=s ' N(R9RARBRERaRRR6RIRRRR%RCRDR=(((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRs #  S t TopoOptimizercB`sGeZdZdeddZddZeddZdZ RS(s TopoOptimizer has one local optimizer. It tries to apply to each node, in topological order (or reverse). Each time the local optimizer applies, the node gets replaced, and the topooptimizer moves on to the next one. t in_to_outcC`s>|dkrtdn||_tj||||dS(Nt out_to_inRs(order must be 'out_to_in' or 'in_to_out'(RR(RRtorderRRI(RRRRRb((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRI s   c `sz|dkr|j}n|j}t|j}tj}ttj|j |tj|}fd}fd}|j |||dt |dd} d} zstj}xPr|j dkrj } n j} | | |j|| 7} qWtj|} Wd|j|| X|j|} t|j}|| |||| | |jfS(Nc`s |k rj|ndS(N(Rq(R(t current_nodetq(s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRs c`s8|k r4yj|Wq4tk r0q4XndS(N(RRR(R(RR(s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRs   R2iR(R6RRnRcRoRpRR RRRR5RRftpopleftRRR(RRt start_fromRxtnb_nodes_startR}tio_tRRRtnbRtloop_tRt nb_nodes_end((RRs./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyR$s4         ic C`s4d|}|dkr-t|dd|dS|\}}}}}} } } t|dt|dt|ddd|t|d|||fd|t|d |d|t|d | d|t|d | d|t| tr0| jr0| j|| j| j| j | j | jfd |d q0ndS(Ns s+TopoOptimizer merge_profile not implementedR4sTopoOptimizer R2R9ts nb_node (start, end, changed)s init io_toposorts loop times callback_timeR;i( R6R7R5RdRYRjR@R_R`RaRb( R:R?R;RRRRRRRRRu((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyR@9s*    %   cC`st|ddS(NR9s(R5(R((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRLUs N( R9RARBR+R6RIR$RER@RL(((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRs  (cO`s|o|jdd}t|dkr9t|}n|\}|sT|j}nt|dddtj|}|r||_n|S(sS Uses the TopoOptimizer from the output nodes to input nodes of the graph. R2iRRRbN(RfR6RcRYR9RR(t local_optsR-R2R/((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pytout2inZs     cO`s|o|jdd}t|dkr9t|}n|\}|sT|j}nt|dddtj|}|r||_n|S(sS Uses the TopoOptimizer from the input nodes to output nodes of the graph. R2iRRRbN(RfR6RcRYR9RR(RR-R2R/((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pytin2outos     tOpKeyOptimizercB`s/eZdZeddZdZdZRS(s WRITEME cC`s8t|dstdntj||||dS(NR}s?LocalOptimizer for OpKeyOptimizer must have an 'op_key' method.(RRRRI(RRRRb((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRIsc `s|jjtttfrEttjt|jnt|jfd}fd}|j |||dt |dd}z3x,rj }||j ||qWWd|j||XdS(Nc`s2|k r.|jkr.j|q.ndS(N(RRq(R(RRR(s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRs c`sG|k rC|jkrCyj|WqCtk r?qCXndS(N(RRRR(R(RRR(s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRs  R2(RR}RdRReR t__iadd__R-RRR5R6RfRR(RRRRRR((RRRs./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyR$s!  cC`s-tt|j||jtjdS(sx Requires the following features: - NodeFinder - ReplaceValidate(Added by default) N(RwRR%RPRt NodeFinder(RR((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyR%sN(R9RARBR+R6RIR$R%(((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRs   t ChangeTrackercB`s>eZdZdZdZdZdZdZRS(cC`st|_d|_dS(Ni(R+tchangedt nb_imported(R((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRIs cC`s|jd7_t|_dS(Ni(RRR(RRRR((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRscC`s t|_dS(N(RR(RRRRRRR((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRscC`s t|_dS(N(R+R(R((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pytresetscC`s ||_dS(N(tchange_tracker(RR((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRscC`s |`dS(N(R(RR((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRs(R9RARIRRRRR(((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRs      cC`sV|j}xCt|D]5\}}||krD||c|7 0)cS`s|dt|dfS(Nii(RK(Rm((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyRRK ss s %.3fs - %ss%Global, final and clean up optimizerssIter %dismerge not implemented for (R7R5RRcRRRRRRtreversedRR RdRKRRcRqRKR@t__code__RRR>(!R:R?R;RRRRRRRR{R_RRbRRRRRRlRRuRRRRoRpRqR`RrRt n_createdtgf_opts((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyR@ s3     )&&&  5  #   &    ( # # # c `stdjjdj}tdjjdj}ttdjjdj}ttdjjdj}t|j|ddd|d|}d}|dd}td} g} g} g} xtt t | t dD]| } xKt dD]5\}}|| kr}| |c|7\}}|t|krM||c|7 s(RdR tApplyRRR RR(RR((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pyt check_chain sc`sYfd|jr0|jjj|}nd}||id\}}||S(s This function traverses the computation graph described by all ``node`` in the graph before the variable out but that are not in the fgraph. It applies each of the local_optimizations on the traversed graph. Its main use is to apply locally constant folding when generating the graph of the indices of a subtensor. We should not apply optimizations on node that are in fgraph. So we don't optimize node that have an attribute fgraph. Notes ----- This doesn't do an equilibrium... So if there is optimization like local_upcast_elemwise_constant_inputs in the list, that adds additional node to the inputs of the node, it can be needed to call this function multiple times. c`st|dds|g|fS|j}t|drD|j|fSxt|jD]\}}||kry||}n|jr||||d\}}x-t|jj|D]\} } | || } x5| jD]*} t| jd  s| jj rt SqWqWt!S(s  This function checks if the outputs of specific ops of a compiled graph have a stack. Parameters ---------- f_or_fgraph: theano.compile.function_module.Function or theano.gof.fg.FunctionGraph The compiled function or the function graph to be analysed. ops_to_check: it can be of four different types: - classes or instances inheriting from theano.gof.Op - tuple/list of classes or instances inheriting from theano.gof.Op - string - function returning a boolean and taking as input an instance of theano.gof.Op. - if ops_to_check is a string, it should be either 'last' or 'all'. 'last' will check only the last op of the graph while 'all' will check all the ops of the graph. - if ops_to_check is an op or a tuple/list of ops, the function will check that all the outputs of their occurrences in the graph have a stack trace. - if ops_to_check is a function, it should take as input a theano.gof.Op and return a boolean indicating if the input op should be checked or not. bug_print: string belonging to {'raise', 'warn', 'ignore'} You can specify the behaviour of the function when the specified ops_to_check are not in the graph of f_or_fgraph: it can either raise an exception, write a warning or simply ignore it. Returns ------- boolean True if the outputs of the specified ops have a stack, False otherwise. s(The type of f_or_fgraph is not supportedRRs)The string ops_to_check is not recognisedt scalar_opR1s)ops_to_check does not have the right typesHProvided op instances/classes are not in the graph or the graph is emptyRaRWR=s&The string bug_print is not recognisedR ("RdR&R?tfunction_moduletFunctiontmakerRRRRRRRStinspecttisclassRTRRRcRRRoReRRqRRRR(RaRsRAR R+R( t f_or_fgrapht ops_to_checkt bug_printRRtapply_nodes_to_checkt op_instancest op_classestobjRRR((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pytcheck_stack_traceu s^$    /    (%       (TRBt __future__RRRt collectionsRRRRmRtloggingRXRCRpR(R[tnumpyRR&Rt theano.compatRRRR R R t six.movesR t theano.gofR RRRRt theano.gof.fgRttheano.misc.ordered_setRRRRSt getLoggerRYRRtobjectRRFRORURRVRRRR6R&R.R/R3RQR+RXRYRvRyRRRRRRRRRRRRRR RR(((s./tmp/pip-build-X4mzal/theano/theano/gof/opt.pytsv          "(  V K ' 9>_#1#X  5  " D 1