ó ¾÷Xc@@sÜddlmZddlmZddlmZddlmZddlmZddlmZddlm Z dd l m Z d efd „ƒYZ d efd „ƒYZdefd„ƒYZdefd„ƒYZdS(i(tabsolute_importi(t initializers(t regularizers(t constraints(tLayer(t InputSpec(tbackend(t interfacest LeakyReLUcB@s,eZdZdd„Zd„Zd„ZRS(s’Leaky version of a Rectified Linear Unit. It allows a small gradient when the unit is not active: `f(x) = alpha * x for x < 0`, `f(x) = x for x >= 0`. # Input shape Arbitrary. Use the keyword argument `input_shape` (tuple of integers, does not include the samples axis) when using this layer as the first layer in a model. # Output shape Same shape as the input. # Arguments alpha: float >= 0. Negative slope coefficient. # References - [Rectifier Nonlinearities Improve Neural Network Acoustic Models](https://web.stanford.edu/~awni/papers/relu_hybrid_icml2013_final.pdf) g333333Ó?cK@s5tt|ƒj|t|_tj|ƒ|_dS(N(tsuperRt__init__tTruetsupports_maskingtKtcast_to_floatxtalpha(tselfRtkwargs((s@/tmp/pip-build-isqEY4/keras/keras/layers/advanced_activations.pyR #s cC@stj|d|jƒS(NR(R treluR(Rtinputs((s@/tmp/pip-build-isqEY4/keras/keras/layers/advanced_activations.pytcall(scC@sKi|jd6}tt|ƒjƒ}tt|jƒƒt|jƒƒƒS(NR(RR Rt get_configtdicttlisttitems(Rtconfigt base_config((s@/tmp/pip-build-isqEY4/keras/keras/layers/advanced_activations.pyR+s(t__name__t __module__t__doc__R RR(((s@/tmp/pip-build-isqEY4/keras/keras/layers/advanced_activations.pyR s  tPReLUcB@sJeZdZejddddd„ƒZd„Zdd„Zd„Z RS(s¼Parametric Rectified Linear Unit. It follows: `f(x) = alpha * x for x < 0`, `f(x) = x for x >= 0`, where `alpha` is a learned array with the same shape as x. # Input shape Arbitrary. Use the keyword argument `input_shape` (tuple of integers, does not include the samples axis) when using this layer as the first layer in a model. # Output shape Same shape as the input. # Arguments alpha_initializer: initializer function for the weights. alpha_regularizer: regularizer for the weights. alpha_constraint: constraint for the weights. shared_axes: the axes along which to share learnable parameters for the activation function. For example, if the incoming feature maps are from a 2D convolution with output shape `(batch, height, width, channels)`, and you wish to share parameters across space so that each filter only has one set of parameters, set `shared_axes=[1, 2]`. # References - [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://arxiv.org/abs/1502.01852) tzeroscK@s¤tt|ƒj|t|_tj|ƒ|_tj|ƒ|_ t j|ƒ|_ |dkrmd|_ n3t|ttfƒs‘|g|_ nt|ƒ|_ dS(N(R RR R R Rtgettalpha_initializerRtalpha_regularizerRtalpha_constrainttNonet shared_axest isinstanceRttuple(RR!R"R#R%R((s@/tmp/pip-build-isqEY4/keras/keras/layers/advanced_activations.pyR Rs   c C@st|dƒ}tgt|ƒ|_|jdk rkx3|jD]%}d||d= 0`. # Input shape Arbitrary. Use the keyword argument `input_shape` (tuple of integers, does not include the samples axis) when using this layer as the first layer in a model. # Output shape Same shape as the input. # Arguments alpha: scale for the negative factor. # References - [Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)](https://arxiv.org/abs/1511.07289v1) gð?cK@s5tt|ƒj|t|_tj|ƒ|_dS(N(R RAR R R R RR(RRR((s@/tmp/pip-build-isqEY4/keras/keras/layers/advanced_activations.pyR £s cC@stj||jƒS(N(R teluR(RR((s@/tmp/pip-build-isqEY4/keras/keras/layers/advanced_activations.pyR¨scC@sQit|jƒd6}tt|ƒjƒ}tt|jƒƒt|jƒƒƒS(NR(tfloatRR RARRRR(RRR((s@/tmp/pip-build-isqEY4/keras/keras/layers/advanced_activations.pyR«s(RRRR RR(((s@/tmp/pip-build-isqEY4/keras/keras/layers/advanced_activations.pyRAs  tThresholdedReLUcB@s/eZdZdd„Zdd„Zd„ZRS(s:Thresholded Rectified Linear Unit. It follows: `f(x) = x for x > theta`, `f(x) = 0 otherwise`. # Input shape Arbitrary. Use the keyword argument `input_shape` (tuple of integers, does not include the samples axis) when using this layer as the first layer in a model. # Output shape Same shape as the input. # Arguments theta: float >= 0. Threshold location of activation. # References - [Zero-Bias Autoencoders and the Benefits of Co-Adapting Features](http://arxiv.org/abs/1402.3337) gð?cK@s5tt|ƒj|t|_tj|ƒ|_dS(N(R RDR R R R Rttheta(RRER((s@/tmp/pip-build-isqEY4/keras/keras/layers/advanced_activations.pyR Çs cC@s#|tj||jktjƒƒS(N(R tcastREtfloatx(RRR<((s@/tmp/pip-build-isqEY4/keras/keras/layers/advanced_activations.pyRÌscC@sQit|jƒd6}tt|ƒjƒ}tt|jƒƒt|jƒƒƒS(NRE(RCRER RDRRRR(RRR((s@/tmp/pip-build-isqEY4/keras/keras/layers/advanced_activations.pyRÏsN(RRRR R$RR(((s@/tmp/pip-build-isqEY4/keras/keras/layers/advanced_activations.pyRD±s  N(t __future__RtRRRtengineRRRR tlegacyRRRRARD(((s@/tmp/pip-build-isqEY4/keras/keras/layers/advanced_activations.pyts$\$