ó ÉÈ÷Xc@`s´dZddlmZddlmZddlmZddlZddlZddlm Z ddl Z ddl m Z ddl mZdd lmZd efd „ƒYZdS( s7Sequence-to-sequence model with an attention mechanism.i(tabsolute_import(tdivision(tprint_functionN(txrange(trnn_cell(tseq2seq(t data_utilst Seq2SeqModelcB`s2eZdZeded„Zd„Zd„ZRS(sßSequence-to-sequence model with attention and for multiple buckets. This class implements a multi-layer recurrent neural network as encoder, and an attention-based decoder. This is the same as the model described in this paper: http://arxiv.org/abs/1412.7449 - please look there for details, or into the seq2seq library for complete model implementation. This class also allows to use GRU cells in addition to LSTM cells, and sampled softmax to handle large output vocabulary size. A single-layer version of this model, but with bi-directional encoder, was presented in http://arxiv.org/abs/1409.0473 and sampled softmax is described in Section 3 of the following paper. http://arxiv.org/pdf/1412.2007v2.pdf ic  `s¦ˆˆ_ˆˆ_|ˆ_|ˆ_tjt|ƒdtƒˆ_ˆjj ˆj| ƒˆ_ tjddtƒˆ_ d‰d} ˆdkr"ˆˆjkr"tj dƒGtjd|ˆjgƒ}tj|ƒ‰tjdˆjgƒ‰WdQX|ˆf‰‡‡‡‡fd†}|} ntj|ƒ}| rItj|ƒ}n|‰|dkrttj|g|ƒ‰n‡‡‡‡fd †‰gˆ_gˆ_gˆ_xPt|d dƒD]:}ˆjjtjtjd dgd d j|ƒƒƒq¼Wxˆt|d ddƒD]n}ˆjjtjtjd dgd dj|ƒƒƒˆjjtjtjd dgd dj|ƒƒƒqWgttˆjƒdƒD]}ˆj|d^qŸ}| rztjˆjˆj|ˆj|ˆj‡fd†d| ƒ\ˆ_ ˆ_!ˆdk rÂx^tt|ƒƒD]G‰gˆj ˆD]&}tj"j#|ˆdˆdƒ^q=ˆj ˆ‹stsoftmax_loss_functionc`sˆ||tƒS(N(tFalse(R$R%(R (so/tmp/pip-build-UG86a1/tensorflow/tensorflow-0.6.0.data/purelib/tensorflow/models/rnn/translate/seq2seq_model.pyR&—st global_step(0RRtbucketst batch_sizeR tVariabletfloatR(t learning_ratetassigntlearning_rate_decay_opR)tNoneR t get_variablet transposeRtGRUCellt BasicLSTMCellt MultiRNNCellRRttarget_weightsRtappendt placeholdertint32tformattfloat32tlenRtmodel_with_bucketstoutputstlossesRt xw_plus_bttrainable_variablestgradient_normstupdatesttraintGradientDescentOptimizert gradientstclip_by_global_normtapply_gradientstziptSavert all_variablestsaver(RRRR*tsizet num_layerstmax_gradient_normR+R.tlearning_rate_decay_factortuse_lstmRt forward_onlyR'twRt single_celltittargetstoutputtparamstoptRGtclipped_gradientstnorm(( RRRRRR RRRso/tmp/pip-build-UG86a1/tensorflow/tensorflow-0.6.0.data/purelib/tensorflow/models/rnn/translate/seq2seq_model.pyt__init__1s„           !!!4 H    #cC`s|j|\}}t|ƒ|krDtdt|ƒ|fƒ‚nt|ƒ|krutdt|ƒ|fƒ‚nt|ƒ|kr¦tdt|ƒ|fƒ‚ni} x,t|ƒD]} || | |j| jW| | |fS(s½Get a random batch of data from the specified bucket, prepare for step. To feed data in step(..) it must be a list of batch-major vectors, while data here contains single length-major cases. So the main logic of this function is to re-index data cases to be in the proper format for feeding. Args: data: a tuple of size len(self.buckets) in which each element contains lists of pairs of input and output data that we use to create a batch. bucket_id: integer, which bucket to get the batch for. Returns: The triple (encoder_inputs, decoder_inputs, target_weights) for the constructed batch that has the proper format to call step(...) later. iR^g(R*RR+trandomtchoiceRtPAD_IDR=R8tlisttreversedtGO_IDR`tarrayR:tonesR<(RtdataRdReRfRRt_t encoder_inputt decoder_inputt encoder_padtdecoder_pad_sizetbatch_encoder_inputstbatch_decoder_inputst batch_weightst length_idxt batch_idxt batch_weightttarget((so/tmp/pip-build-UG86a1/tensorflow/tensorflow-0.6.0.data/purelib/tensorflow/models/rnn/translate/seq2seq_model.pyt get_batchæs4  8 4(t__name__t __module__t__doc__R(R]RkR(((so/tmp/pip-build-UG86a1/tensorflow/tensorflow-0.6.0.data/purelib/tensorflow/models/rnn/translate/seq2seq_model.pyR"s  v <(R„t __future__RRRRltnumpyR`t six.movesRt tensorflowR ttensorflow.models.rnnRRttensorflow.models.rnn.translateRtobjectR(((so/tmp/pip-build-UG86a1/tensorflow/tensorflow-0.6.0.data/purelib/tensorflow/models/rnn/translate/seq2seq_model.pyts