Xc@`sdZddlmZmZmZdZdZdZdZddl Z ddl Z ddl Z ddl m Z ddlZddlZdd lmZdd lmZmZdd lmZdd lmZdd lmZmZddlmZmZddlmZe jdZ dddZ"de#fdYZ$ddZ%dZ&e#Z'de(e(e'dZ)gdZ*dZ+dZ,dZ-dZ.dddZ/dZ0de#fd YZ1d!Z2d"Z3d#Z4dd$Z5d%e#fd&YZ6d'Z7dS((s\ This module provides utility functions for the Scan Op. See scan.py for details on scan. i(tabsolute_importtprint_functiontdivisionsrestructedtext ensMRazvan Pascanu Frederic Bastien James Bergstra Pascal Lamblin Arnaud Bergerons (c) 2010, Universite de Montreals Razvan Pascanu N(t OrderedDict(tizip(t string_typest iteritems(txrange(trebuild_collect_shared(tgoftcompat(ttensortscalar(tget_scalar_constant_valuestheano.scan_utilstcC`st|dr.|jdk r.|j|}nd}t|tjr|r|j|kr|j|}|j|j |j |j}t |j |_ |S|j Snt|tjrK|rtjd|}n |j }||_tjjdkrGytjj|}Wntk r+qGX|j j||j _n|Sytj|}Wntk rqnX|r|j|kr|j|}n|j }||_tjjdkry%t jtjj||j _Wqtk rqXn|S(s] Internal function that constructs a new variable from x with the same type, but with a different name (old name + tag). This function is used by gradient, or the R-op to construct new variables for the inputs of the inner graph such that there is no interference between the original graph and the newly constructed graph. tnametdtypetoffN(thasattrRtNonet isinstancettheanotConstantRtastypet __class__ttypetdatatcopyttagtcloneR tScalarVariabletget_scalar_typetconfigtcompute_test_valueR toptget_test_valuetAttributeErrortfiltert test_valueR tas_tensor_variablet TypeErrortdeepcopy(txRRtnw_nametcasted_xtnwxtnw_xt x_test_value((s=/tmp/pip-build-X4mzal/theano/theano/scan_module/scan_utils.pytsafe_new)sH        % tuntilcB`seZdZdZRS(s Class used to encode the different things the inner function of scan can (or needs) to return. This class has to be used when scan needs to halt when a condition is met, otherwise the list of outputs and dictionary can directly be return as a tuple. The reason is that otherwise scan has no way to distinguish between the condition and the list of outputs ( unless we enforce and order, but since this was not impose up to know it can make quite a bit of code to fail). cC`s.tj||_|jjdks*tdS(Ni(R R't conditiontndimtAssertionError(tselfR2((s=/tmp/pip-build-X4mzal/theano/theano/scan_module/scan_utils.pyt__init__~s(t__name__t __module__t__doc__R6(((s=/tmp/pip-build-X4mzal/theano/theano/scan_module/scan_utils.pyR1ps c C`s|dkrt}n||kr(|S|j|ddlm}ddlm}m}ddlm }ddl m } ||krt |j |jr|j|||RRtCudaNdarrayTypeR4t context_nametownertcuda_availableR"tinputsR R'ttraverse( toutR*tx_copytdtvisitedR:R;R<R=R>tinp((s=/tmp/pip-build-X4mzal/theano/theano/scan_module/scan_utils.pyRJs<      cC`sd}t|trUxt|D],\}}|t|N}|t|N}q"WnIt|ttfrx1|D]}|t|N}qqWn|t|N}|S(Ni(RtdictRthash_listsDictsTuplestlistttuplethash(R*t hash_valuetktv((s=/tmp/pip-build-X4mzal/theano/theano/scan_module/scan_utils.pyRQs c C`s^|tk r.tjd|s%t|}nt|trRt|j}n[t|ttfrp|}n=|dkrg}n(t dt |t t |fg|D]\}}||j f^q}gt ||D]$\\} }\} }||f^q} t|g|g||\} } } t| g| g||\} } } | S(s Function that allows replacing subgraphs of a computational graph. It returns a copy of the initial subgraph with the corresponding substitutions. Parameters ---------- output : Theano Variables (or Theano expressions) Theano expression that represents the computational graph. replace : dict Dictionary describing which subgraphs should be replaced by what. share_inputs : bool If True, use the same inputs (and shared variables) as the original graph. If False, clone them. Note that cloned shared variables still use the same underlying storage, so they will always have the same value. copy_inputs Deprecated, use share_inputs. seIn `clone()` function, the argument `copy_inputs` has been deprecated and renamed into `share_inputs`sZreplace is neither a dictionary, list, tuple or None ! The value provided is %s,of type %sN(tDEPRECATED_ARGtwarningstwarnR4RRPRRtitemsRSRt ValueErrortstrRtzipR( toutputtreplacetstrictt share_inputst copy_inputsR[R*tyt tmp_replacet_t new_replacet_outstouts((s=/tmp/pip-build-X4mzal/theano/theano/scan_module/scan_utils.pyRs8       "+ 1c`stfdt|}tttjj|t|}tt|}gt||D]$\}}||k rt||f^qt}t|dtd|}tttjj|t|}g|D]}t |dt r|^q} t| dt } tt| | }tt|t| t| }t|dtd|}tj j ||dt } ttj jdfd} tj j| d} | j| | j}| j|S( sdConstruct new graphs based on 'graphs' with some variables replaced according to 'replacer'. :param replacer: function that takes a variable and returns its replacement. :param graphs: an iterable of graphs in which to replace variables :param additional_inputs: an iterable of graph inputs not used in any of 'graphs' but possibly used in the graphs returned by `replacer` :return: the new graphs, in the same order as 'graphs' Example: .. code-block:: python tag = "replaceme" a = tensor.scalar("a") b = tensor.scalar("b") c = tensor.scalar("c") ab = a + b ab.tag.replacement = a * b u = ab + c v, = map_variables(lambda graph: return getattr(graph.tag, "replacement", graph), [u]) # v is now equal to a * b + c c`s1|kr|S|}j||SdS(N(R@(tgrapht new_graph(t graphs_seentreplacer(s=/tmp/pip-build-X4mzal/theano/theano/scan_module/scan_utils.pytwrapped_replacer)s    RbR`tcachedRc `s1|krtSddlm}ddlm}t|j||fr td|jjd|jd|jj d|j\}}}t|j|r||||jj dd}n-t|j|r||||jj }n|j |}j||j Sj|tt|j SdS( Ni(tScan(t OpFromGrapht inner_inputst outer_inputst inner_outputst containing_opttypeConstructor(tFalsettheano.scan_module.scan_opRpttheano.compileRqRR"t_map_variables_innerRItoutputstinfoRtkwargst make_nodeR@RRtmap(tnodeRpRqtnew_inner_inputstnew_outer_inputstnew_inner_outputstnew_optnew_node(t nodes_seenRn(s=/tmp/pip-build-X4mzal/theano/theano/scan_module/scan_utils.pytlocal_transformNs2        t out_to_inN(R?RRR RjRIRR^RtTruetgetattrRwtfgt FunctionGraphtopttlocal_optimizerRt TopoOptimizertoptimizeR{tdisown(Rmtgraphstadditional_inputstinputs_t new_inputstinput_t new_inputt replacementsR*tcached_constantstcopied_constantsRRttopo_transformt new_graphs((RlRRmRns=/tmp/pip-build-X4mzal/theano/theano/scan_module/scan_utils.pyt map_variabless.!  ((+& $&   c `stt||ggddlmddlm}ddlmfd}t||}t ||}t ||} || |fS(Ni(t scan_utils(tchain(R c `s|}g}g}xajj|gD]J}t|jr.t|jrh|j|qx|j|q.q.Wtt|tj}| r| r|Sg}x=|D]5}|j }d|j |_ |j||fqWx|D]}t |dt r.t d|fn|jkrj|dd} | |<j| j||jjj| qqW|jjtj |gdtd|\}|S(Ns %s_copiedtupdatesReplacement introduces shared variable %s which has an update associated with it into the inner graph of %s. This is not currently supported.Rt_copyRbR`(RjRIRtVariableRtappendRRR?tvaluesRRRRwtNotImplementedErrortkeysR0RGtfgrapht add_inputtextendR[RR( RjRkt other_inputst constantsRtforeign_inputsRRt outer_inputt inner_input(Rutextra_inner_inputstextra_outer_inputsR touter_to_innerRmR(s=/tmp/pip-build-X4mzal/theano/theano/scan_module/scan_utils.pytinner_replacers@ "        ( RPR^ttheano.scan_moduleRt itertoolsRRR RRR( RmRrRsRtRuRRRRR((RuRRR RRmRs=/tmp/pip-build-X4mzal/theano/theano/scan_module/scan_utils.pyRz|s !@c`sd}d}d}d}fd|sNtdn||rpd ||tfS||rd gt|fSd|f}t|ttfst|nt|}d}t|d kr||d re||d r%d ||d t|d fS||d rV|d j||d tfSt|q||d r||d rt|q||d r|d jgt|d fSt|qt|nt|d krw||d rh||d rY||d rJ|d j||d t|d fSt|qtt|qt|n t|d S(s~ This function tries to recognize the updates OrderedDict, the list of outputs and the stopping condition returned by the lambda expression and arrange them in a predefined order. WRITEME: what is the type of ls? how is it formatted? if it's not in the predefined order already, how does this function know how to put it in that order? cS`s^t|ttfrDtg|D]}t|tj^qrDtSt|tjrZtStS(N(RRRRStallRRRRw(telemR*((s=/tmp/pip-build-X4mzal/theano/theano/scan_module/scan_utils.pyt is_outputss +cS`st|tr\t|tj rXt|dkrXtjdtt|dnt St|t t frt g|D]-}t|t t fot|dk^q{rt St S(Nis,Expected OrderedDict or OrderedUpdates, got s.. This can make your script non-deterministic.i(RRPR RtlenRYRZR]RRRRRSRRw(RR*((s=/tmp/pip-build-X4mzal/theano/theano/scan_module/scan_utils.pyt is_updatess:cS`st|tjjS(N(RRt scan_moduleR1(R((s=/tmp/pip-build-X4mzal/theano/theano/scan_module/scan_utils.pyt is_conditionscS`s*t|ttfrt|S|gSdS(N(RRRRS(R*((s=/tmp/pip-build-X4mzal/theano/theano/scan_module/scan_utils.pyt_lists c`sd}t|ts$t|tr-|}nt|trKt|}n|dk rqtfd|DSt|tjpt|tj j SdS(s Ensure `x` is made only of allowed data types. Return True iff `x` is made only of lists, tuples, dictionaries, Theano variables or `theano.scan_module.until` objects. c3`s|]}|VqdS(N((t.0Rd(t_filter(s=/tmp/pip-build-X4mzal/theano/theano/scan_module/scan_utils.pys sN( RRRRRSRPRRRRRR1(R*titer_on(R(s=/tmp/pip-build-X4mzal/theano/theano/scan_module/scan_utils.pyRs   s*The return value of your scan lambda expression may only be made of lists, tuples, or dictionaries containing Theano variables (or `theano.scan_module.until` objects for conditions). In particular if you need to use constant values, you can use `tensor.constant` to turn them into Theano variables.sJScan cannot parse the return value of your lambda expression, which is: %ssThe return value of the lambda function has been restricted. you have to always return first the outputs (if any), afterwards the updates (if any) and at the end the conclusioniiiiN(R\RRRRRRSRR2(tlsRRRRt error_msgtdeprecation_msg((Rs=/tmp/pip-build-X4mzal/theano/theano/scan_module/scan_utils.pytget_updates_and_outputssR          !!(cC`s |dk}y1tj|}tj|}t|t}Wn#tk rbt}t}t}nX| r| ry.t|}tj|}tj|}Wqtk rt}t}qXnt|t j rt|j trt }nt}|p |p |p |S(N( RtnumpytisnantisinfRRt ExceptionRwR R RRR(R*tisNonetisNaNtisInftisStrtval((s=/tmp/pip-build-X4mzal/theano/theano/scan_module/scan_utils.pytisNaN_or_Inf_or_NoneMs*      $ cC`s|dkr|Sgt|jD]}|j|^q }||dg|d}tj|j|}tj||d |}t|j_ |S(s Transforms the shape of a tensor from (d1, d2 ... ) to ( d1+size, d2, ..) by adding uninitialized memory at the end of the tensor. ii( RR3tshapeR t AllocEmptyRt set_subtensorRwRtnan_guard_mode_check(t tensor_vartsizeR*tshapest new_shapetemptytret((s=/tmp/pip-build-X4mzal/theano/theano/scan_module/scan_utils.pyt expand_emptyfs ) c`sWt|t|kst|dkr3g}n|dkrHg}nxt||D]\}}|jr{|j r{tS|jr|j rtS|jr|jjj||jjj|krtSn||krX|j|jkrXtSqXWt|t|krtSx3t||D]"\}}|j|jkrtSqWt t ||}t } xt||D]t\} } | jsrt | t j rt | t j r| j| stSq| | f|kr| | krtSqrqrWfdxWtt|D]C} || jr || j|| j|| } | sOtSq q WtS(sChecks if Theano graphs represent the same computations. The two lists `xs`, `ys` should have the same number of entries. The function checks if for any corresponding pair `(x,y)` from `zip(xs,ys)` `x` and `y` represent the same computations on the same variables (unless equivalences are provided using `in_xs`, `in_ys`). If `in_xs` and `in_ys` are provided, then when comparing a node `x` with a node `y` they are automatically considered as equal if there is some index `i` such that `x == in_xs[i]` and `y == in_ys[i]`(and they both have the same type). Note that `x` and `y` can be in the list `xs` and `ys`, but also represent subgraphs of a computational graph in `xs` or `ys`. c`s&|j|jkrtSt|jt|jkr8tSt|jt|jkrZtSt}xTt|j|jD]=\}}||f|krtS||f|krvt}qvqvW|rtSx!t|j|jD] \}}||f|kr|jrl|jrl|jjj||jjj|krl|j|j||}|s|j ||ftSq|jdkr|jdkr||krt |t j rt |t j r|j|stSqtSqqtSqqWx6t|j|jD]\}}|j ||fqWtSdS(sh Compare two nodes to determine if they perform equal computation. This is done by comparing the ops, the number of inputs, outputs and by ensuring that the inputs themselves are the result of equal computation. NOTE : This function relies on the variable common to cache results to be more efficient. N(R"RwRRIR{RRRGtindexR@RRR Rtequals(tnd_xtnd_ytcommont differentt all_in_commontdxtdyt nodes_equal(t compare_nodes(s=/tmp/pip-build-X4mzal/theano/theano/scan_module/scan_utils.pyRsD " "   "N(RR4RRRGRwR{RRR?R^RR RRtrangeR(txstystin_xstin_ysR*Rdt_xt_yRRRRtitis_equal((Rs=/tmp/pip-build-X4mzal/theano/theano/scan_module/scan_utils.pytequal_computationsxsL     *   D # c`sx\t||D]K\}}|dk rt||jkrt||jks[tqqWtjjjt j j ggx-t||D]\}}j ||qWfdg}x,|D]$}||j j|qW|S(s Compute the shape of the outputs given the shape of the inputs of a theano graph. We do it this way to avoid compiling the inner function just to get the shape. Changes to ShapeFeature could require changes in this function. c`s|jkrdS|jdkr2j|nRx0|jjD]"}|jkr?|q?q?Wd}j||jdddS(sX Go back in the graph, from out, adding computable shapes to shape_of. Ntreasontdummy(tshape_ofRGRtinit_rRIt on_import(RKROt dummy_fgraph(tlocal_traverset shape_feature(s=/tmp/pip-build-X4mzal/theano/theano/scan_module/scan_utils.pyRsN(RRRR3R4R Rt ShapeFeaturet on_attachRR Rt set_shapeRR(RiRIt input_shapesROtinp_shpRto((RRs=/tmp/pip-build-X4mzal/theano/theano/scan_module/scan_utils.pyt infer_shapes !"  t ValidatorcB`s)eZdZddddZdZRS(s Check if variables can be expressed without using variables in invalid. Parameters ---------- valid_equivalent Provides a dictionary mapping some invalid variables to valid ones that can be used instead. cC`s|dkrg}n|dkr*g}n|dkrBt}nt||_t||_|j|_|jjt|j |jjt|j dS(N( RRR?tvalidtinvalidRtvalid_equivalentRRRRR(R5RRR((s=/tmp/pip-build-X4mzal/theano/theano/scan_module/scan_utils.pyR6>s      c C`s||jkr|tfS||jkr9|j|tfS||jkrLdS|jdkrt|tj r|j }|jj ||jj |||j|<|tfS|tfSng|jj D]}|j |^q}d|kr|jj |dSg|D]\}}|^q }g|D]\}}|s+|^q+}|r|jj|} | j|j}|jj ||jj |||j|<|tfS|tfS(s Go backwards in the graph, from out, and check if out is valid. If out is a valid node, (out, True) is returned. If out is not valid, but has an equivalent e, (e, False) is returned. If out is not valid and has no equivalent, None is returned. N(RRRRwRRRGRR tTensorConstantRR@RItchecktclone_with_new_inputsR{R( R5RKt cloned_outRRIROtis_validt all_inputst equiv_inputst cloned_node((s=/tmp/pip-build-X4mzal/theano/theano/scan_module/scan_utils.pyRQs8      ( %  N(R7R8R9RR6R(((s=/tmp/pip-build-X4mzal/theano/theano/scan_module/scan_utils.pyR2s cC`sEgt|jD]\}}||kr|^q}tjj|}g}|j}|j|j|j}xPt |D]B} t |j d| } ||j||| !g7}|| 7}qyW|gt |j D] } g^q7}|gt |jD]} |j|| g^q7}t} g|D] } d^q&} x| rt} xt|D]u\}} | |rTtjg|| D]}||k^q{rTd| |<|tjj|j| g7}t} qTqTWq;Wgt|D]"\}}| |dkr|^q}gt|D]"\}}| |dkr|^q}||fS(s Looks at all outputs defined by indices ``out_idxs`` and see whom can be removed from the scan op without affecting the rest. Return two lists, the first one with the indices of outs that can be removed, the second with the outputs that can not be removed. t tap_arrayii(t enumerateR{R RjRItn_seqst n_mit_mott n_mit_sott n_sit_sotRRR|Rt n_nit_sott n_shared_outsRRwRtany(R"tout_idxsRRt non_removabletrequired_inputstout_instoffsettlimtidxtn_insRVtaddedt out_idxs_masktposR*t required_outst not_required((s=/tmp/pip-build-X4mzal/theano/theano/scan_module/scan_utils.pytscan_can_remove_outss4 &4  ,  5cC`st}g|d<|jd|dscs`s|]}t|VqdS(N(R(RRC((s=/tmp/pip-build-X4mzal/theano/theano/scan_module/scan_utils.pys sRRRR!R R"RR%t destroy_mapR#R$R'R(( struncate_gradientsnamesmodeRDsgpusgpuasas_whilesprofilesallow_gc(#tn_stepsRAtcondt outer_in_seqst inner_in_seqstmit_mot_in_slicestmit_sot_in_slicestsumtinner_in_mit_motRRtinner_in_mit_sottouter_in_mit_mottouter_in_mit_sottouter_in_sit_sottinner_in_sit_sottouter_in_sharedtinner_in_sharedtouter_in_nit_sottouter_in_non_seqstinner_in_non_seqsR!touter_out_mit_mottinner_out_mit_mottouter_out_mit_sottinner_out_mit_sottouter_out_sit_sottinner_out_sit_sottouter_out_nit_sottinner_out_nit_sottouter_out_sharedtinner_out_sharedR4Rt other_info(R5Rst outer_outputst _inner_inputst_inner_outputsR|trvalRtRrtptqR R R t n_mit_mot_inst n_mit_sot_instiimmtqqtsltiimsRRRR tiommRV((s=/tmp/pip-build-X4mzal/theano/theano/scan_module/scan_utils.pyR6ls           !  !                !             cC`s<|jt|jgt|jg|j|j|jS(N(RHRKRLRMRQRSRV(R5((s=/tmp/pip-build-X4mzal/theano/theano/scan_module/scan_utils.pyts4cC`s;|jg|j|j|j|j|j|j|jS(N(RERGRNRORPRRRTRU(R5((s=/tmp/pip-build-X4mzal/theano/theano/scan_module/scan_utils.pyRos3cC`s3t|jg|j|j|j|j|jS(N(RKRXRZR\R^R`RF(R5((s=/tmp/pip-build-X4mzal/theano/theano/scan_module/scan_utils.pyRos+cC`s#|j|j|j|j|jS(N(RWRYR[R]R_(R5((s=/tmp/pip-build-X4mzal/theano/theano/scan_module/scan_utils.pyRoscC`stdt|jdt|jdt|jd|j|jdggt|jdt|jdt|j dt|j d t d |j Dd |j |j S( NR R R R iRRRR cs`s|]}t|VqdS(N(R(RRC((s=/tmp/pip-build-X4mzal/theano/theano/scan_module/scan_utils.pys sR!(RRRGRNRORIRJRQRPRTRRRKR!Ra(R5((s=/tmp/pip-build-X4mzal/theano/theano/scan_module/scan_utils.pyRos cC`stjt|}|jj|jx~|jD]s}|jds|jds|jds|jds|d kr2t||tjt||q2q2W|S( Ntinner_int inner_outtouter_int outer_outR!RIRJRa(smit_mot_out_slicessmit_mot_in_slicessmit_sot_in_slicess other_info( tobjectt__new__Rt__dict__Rt startswithtsetattrRR(R5trestattr((s=/tmp/pip-build-X4mzal/theano/theano/scan_module/scan_utils.pyt__copy__ s )cC`stj|}x{|jD]p}|jdsg|jdsg|jdsg|jdsg|dkrt||jt||qqW|S( NRpRqRrRsR!RIRJ(smit_mot_out_slicessmit_mot_in_slicessmit_sot_in_slices(RRvRwRR(R5totherRyRz((s=/tmp/pip-build-X4mzal/theano/theano/scan_module/scan_utils.pytmerges &( R7R8R9R6tpropertyRrRsRtRbR|R{R}(((s=/tmp/pip-build-X4mzal/theano/theano/scan_module/scan_utils.pyRBfs t  cC`s|dkrdSt}ddlm}|}|j|g}x|r|j}||krlqHn|j|t|g|gr|j||fqH|jrH|j |jj qHqHWt |dkr|St |d|S(s Check all internal values of the graph that compute the variable ``out`` for occurrences of values identical with ``x``. If such occurrences are encountered then they are replaced with variable ``y``. Parameters ---------- out : Theano Variable x : Theano Variable y : Theano Variable Examples -------- out := sigmoid(wu)*(1-sigmoid(wu)) x := sigmoid(wu) forced_replace(out, x, y) := y*(1-y) Note ---- When it find a match, it don't continue on the corresponding inputs. i(tdequeR`N( RR?t collectionsRRtpopleftR@RRGt extendleftRIRR(RKR*RdRNRRgt to_replaceRj((s=/tmp/pip-build-X4mzal/theano/theano/scan_module/scan_utils.pytforced_replace"s&         (8R9t __future__RRRt __docformat__t __authors__t __copyright__t __contact__RtloggingRYRRRRt theano.compatRtsixRRt six.movesRttheano.compile.pfuncRR R R R ttheano.tensor.basicR t getLoggert_loggerRR0RtR1RJRQRXRRRRzRRRRRRRR4R<RARBR(((s=/tmp/pip-build-X4mzal/theano/theano/scan_module/scan_utils.pytsT     G 5   : u [ v   5V (