ó àÆ÷Xc@`sôddlmZmZmZddlZddlZddlZddlmZddlmZ ddl m Z ddl m ZejjjejƒZdejfd„ƒYZd efd „ƒYZd efd „ƒYZd dd„ƒYZdS(i(tabsolute_importtprint_functiontdivisionN(tconfig(ttensor(tScan(tunittest_toolstTestGaussNewtoncB`s2eZdZd„Zd„Zd„Zd„ZRS(s‡ Regression test for code exhibiting various optimization errors. This test case is based on code by Sigurd Spieckermann. cC`stjjtjƒƒ|_dS(N(tnumpytrandomt RandomStatetuttt fetch_seedtrng(tself((sF/tmp/pip-build-X4mzal/theano/theano/scan_module/tests/test_scan_opt.pytsetUpsc `s—|dkr'||f}|df}n|||f}||df}tj|jjd|ƒjtjƒdtƒ}tj|jjd|ƒjtjƒdtƒ}|dkrÒtj dƒ} tj dƒ} ntj dƒ} tj dƒ} |j dtƒ| j _ |j dtƒ| j _ tjd|jjd|dfƒjtjƒdtƒ‰tjd|jjddƒjtjƒdtƒ‰tjd|jjddƒjtjƒdtƒ} tjtjdƒjtjƒdtƒ‰tjtjdƒjtjƒdtƒ} ˆˆ| ˆ| g} ‡‡‡fd†}|dkrmtjd dƒjtjƒ}n!tjd |dƒjtjƒ}tj|d | gd |gƒ\}}tj|| ƒ| }t|ƒ}| |d jd dƒjƒ}tj|dtg| D]}|d |f^q ƒƒ}|d| d|d| dtjdƒƒ}tjg|g|di|| 6|| 6d|ƒ}|ƒdS(Nitsizetborrowtinputsttargetsg{®Gáz„?i c`s3tjtj|ˆƒtj|ˆƒˆƒ}|S(N(tTttanhtdot(tx_tth_tm1th(tW_hhtW_xhtb_h(sF/tmp/pip-build-X4mzal/theano/theano/scan_module/tests/test_scan_opt.pytstepCs/gt sequencest outputs_infoitaxistreplacetvtcostt parameterstdampgð?tgivenstmode(i i (i i(ttheanotsharedR tuniformtastypeRtfloatXtTrueRtmatrixttensor3t get_valuettagt test_valueRtzerostalloctscanRtGaussNewtonMatrixtsumtmeantclonetdicttconstanttfunction(Rt num_featurest num_timestepst batch_sizeR't inputs_sizet targets_sizeRRtxtttW_hytb_ytparamsRth_0RtupdatestytgnR#tpitcost_tGvtf((RRRsF/tmp/pip-build-X4mzal/theano/theano/scan_module/tests/test_scan_opt.pyt_runs^  ! !   % % ** !!     ,** cC`s |jdddddtƒdS(Nidi R?iR'(ROR'(R((sF/tmp/pip-build-X4mzal/theano/theano/scan_module/tests/test_scan_opt.pyt test_batchlscC`s/tjdƒ}|jddddd|ƒdS(Nt'remove_constants_and_unused_inputs_scanidi R?iR'(R't excludingRO(Rtmode_exc((sF/tmp/pip-build-X4mzal/theano/theano/scan_module/tests/test_scan_opt.pyt test_nobatchqs(t__name__t __module__t__doc__RRORPRT(((sF/tmp/pip-build-X4mzal/theano/theano/scan_module/tests/test_scan_opt.pyRs   U R6cB`seZd„Zd„ZRS(cC`s ||_dS(N(t_s(Rts((sF/tmp/pip-build-X4mzal/theano/theano/scan_module/tests/test_scan_opt.pyt__init__{sc C`s´tj|j||ƒ}tjtjtj||jƒ|ƒ|jd|gƒ}tjtj||jƒ|d||gƒ}gt||ƒD]\}} ||| ^q}|S(Ntconsider_constant(RtRopRXtgradR7tzip( RR"R#R$R%tJvtHJvtJHJvtJHJvitvi((sF/tmp/pip-build-X4mzal/theano/theano/scan_module/tests/test_scan_opt.pyt__call__€s. 0(RURVRZRd(((sF/tmp/pip-build-X4mzal/theano/theano/scan_module/tests/test_scan_opt.pyR6zs tTestPushOutScanOutputDotcB`s)eZdZd„Zd„Zd„ZRS(s Test class for the PushOutScanOutput optimizer in the case where the inner function of a scan op has an output which is the result of a Dot product on a non-sequence matrix input to scan and a vector that is the result of computation in the inner function. cC`s†tjƒ}tjƒ}tj||ƒ}tjdƒ}tj||gtj||ƒd|ƒ}tj dƒ}tj||gtj||ƒd|ƒ}g|j j j ƒD]}t |jtƒr¯|^q¯d} t| jjƒdksõt‚t | jjdtjƒ st‚tjjdƒjtjƒ} tjjd ƒjtjƒ} || | ƒ} || | ƒ} tj| | ƒdS( sy Test the case where the vector input to the dot is not already an output of the inner function. R5R'tscanOp_pushout_outputiiiiN(ii(RtvectorR.RR't includingR(R<tjacobianRRtmakertfgraphttoposortt isinstancetopRtlentoutputstAssertionErrortDotRR R+RR,R tassert_allclose(RR"tmtoutputtopt_modetf_optt no_opt_modetf_no_opttnodet scan_nodetv_valuetm_valuet output_optt output_no_opt((sF/tmp/pip-build-X4mzal/theano/theano/scan_module/tests/test_scan_opt.pyttest_dot_not_output•s   **"#c C`s¶tjƒ}tjƒ}d„}tjd|ddgdd|d|ƒ\}}tjdƒ}tj||g|d|ƒ}tjd ƒ}tj||g|d|ƒ} g|j j j ƒD]} t | j tƒr¿| ^q¿d } t| j jƒdkst‚t | j jd tjƒ s(t‚tjjdƒjtjƒ} tjjdƒjtjƒ} || | ƒ}| | | ƒ}tj|d |d ƒtj|d|dƒdS(s{ Test the case where the vector input to the dot is already a nitsot output of the inner function. cS`s |d}tj||ƒ|fS(Ni(RR(tvecttmatt vect_squared((sF/tmp/pip-build-X4mzal/theano/theano/scan_module/tests/test_scan_opt.pyt inner_fctÂs tfnRiRt non_sequencesR5R'RfiiiiiN(ii(ii(RR.R(R5tNoneR'RhR<RRRjRkRlRmRnRRoRpRqRrRR R+RR,R Rs(RtatbR„RpRHRvRwRxRyRzR{ta_valuetb_valueR~R((sF/tmp/pip-build-X4mzal/theano/theano/scan_module/tests/test_scan_opt.pyttest_dot_nitsot_output¹s*    "#c C`s¹tjƒ}tjƒ}d„}tjd|d|ddgd|d|ƒ\}}tjdƒ}tj||g|d|ƒ}tjd ƒ}tj||g|d|ƒ} g|j j j ƒD]} t | j tƒrÂ| ^qÂd} t| j jƒd kst‚t | j jdtjƒ s+t‚tjjdƒjtjƒ} tjjdƒjtjƒ} || | ƒ}| | | ƒ}tj|d|dƒtj|d|dƒdS(s› Test the case where the vector input to the dot is not already a non-nitsot (in this case a sitsot) output of the inner function. cS`s&||}tj||ƒ}||fS(N(RR(tseq1tprevious_output1tnonseq1toutput1toutput2((sF/tmp/pip-build-X4mzal/theano/theano/scan_module/tests/test_scan_opt.pyR„ñs R…RiRR†R5R'RfiiiiiN(ii(ii(RR.R(R5R‡R'RhR<RRRjRkRlRmRnRRoRpRqRrRR R+RR,R Rs(RRˆR‰R„RpRHRvRwRxRyRzR{RŠR‹R~R((sF/tmp/pip-build-X4mzal/theano/theano/scan_module/tests/test_scan_opt.pyttest_dot_sitsot_outputès*   "#(RURVRWR€RŒR’(((sF/tmp/pip-build-X4mzal/theano/theano/scan_module/tests/test_scan_opt.pyRes $ /tTestPushOutSumOfDotcB`s eZdZd„Zd„ZRS(sÇ Test case for the PushOutScanOutput optimizer in the case where the scan is used to compute the sum over the dot products between the corresponding elements of two list of matrices. c `sMd}d}d}d}tjtjjd||fddƒjtjƒƒ‰dˆ_tjˆj ƒƒ‰dˆ_tjˆj ƒƒ‰d ˆ_t j d ƒ}tjjd|||fddƒjtjƒ}t j d ƒ}|}t j d ƒ} |} t j tj tjd ƒ||ƒ} ‡‡‡fd†} tjdƒ} tj| d||| gd|d| ddd| ƒ\}}|djƒ}t j|ˆˆˆgƒ}tjd||| gd|d| ƒ}tjdƒ}tj| d||| gd|d| ddd|ƒ\}}|djƒ}t j|ˆˆˆgƒ}tjd||| gd|d|ƒ}g|jjjƒD]}t|jtƒrƒ|^qƒd}xg|jjD]Y}t|jjt jjƒotg|jj D]}t|t j!ƒ^qéƒ s¸t"‚q¸W|||| ƒ}|||| ƒ}t#j$||ƒdS(s  This test case comes from https://github.com/rizar/scan-grad-speed and is an example of actual computation done with scan in the context of machine translation 'dim' has been reduced from 1000 to 5 to make the test run faster iPi2iRtscaleg-Cëâ6?tUtVtWRBtritziic `s||jˆƒ}||jˆƒ}tjj|ƒ}tjj|ƒ}||}||jˆƒ} tj| ƒ} || d||} | S(Ni(RRtnnettsigmoidR( RBR˜R™Rtpre_rtpre_ztrtztafter_rtpre_htnew_htres_h(R•R–R—(sF/tmp/pip-build-X4mzal/theano/theano/scan_module/tests/test_scan_opt.pyt rnn_step1Bs R5Rtn_stepsRtnametfpass1R'iÿÿÿÿRRpRfiNi (%R(R)RR tnormalR+RR,R¦R0RR/R4tcastR'RhR5R7R]R<RRRjRkRlRmRnRRptownertelemwisetElemwisetanyRRrRqR Rs(RR?tseq_lentn_wordstdimRBtx_valueR˜tri_valueR™tzi_valuetinitR¤RvRt_R#tgrad1RwRxRyRztscan_node_gradRutit f_opt_outputtf_no_opt_output((R•R–R—sF/tmp/pip-build-X4mzal/theano/theano/scan_module/tests/test_scan_opt.pyttest_machine_translationsV    %! ! "6c`stjƒ}tjƒ}tjƒ}tjtjjddƒƒjtj ƒ‰tjtjjddƒƒjtj ƒ‰‡‡fd†}tj tjjddƒƒ}t j dƒ}tj |d |||gd |d |ƒ\}}|d } tj|||g| d |ƒ} t jd ƒ} tj |d |||gd |d | ƒ\}}|d } tj|||g| d | ƒ} tjjdƒjtj ƒ} tjjdƒjtj ƒ}tjjdƒjtj ƒ}| | ||ƒ}| | ||ƒ}tj||ƒdS(sa Test the case where the initial value for the nitsot output is non-zero Riiiic`sBtj|ˆƒ|}tj|ˆƒ}tj||ƒ}||S(N(RR(Rtseq2tseq3tprevious_outputttemp1ttemp2t dot_output(R•R—(sF/tmp/pip-build-X4mzal/theano/theano/scan_module/tests/test_scan_opt.pyR„€siR5RRR'iÿÿÿÿRfiN(ii(ii(ii(iii(iii(iii(RR/R(R)RR R¨R+RR,tas_tensor_variableR'RhR5R<RRR Rs(Rtinput1tinput2tinput3R„R´RvRRµRuRwRxRyt input1_valuet input2_valuet input3_valueR~R((R•R—sF/tmp/pip-build-X4mzal/theano/theano/scan_module/tests/test_scan_opt.pyttest_non_zero_initss:   **      (RURVRWR»RÉ(((sF/tmp/pip-build-X4mzal/theano/theano/scan_module/tests/test_scan_opt.pyR“s U((t __future__RRRRtunittestR(RRRttheano.scan_module.scan_opRt theano.testsRR tcompileR'tget_modetTestCaseRtobjectR6ReR“(((sF/tmp/pip-build-X4mzal/theano/theano/scan_module/tests/test_scan_opt.pyts   lŠ