ʽXc@`sdZddlmZmZmZddlmZddlZddl m Z ddl m Z m Z mZddlmZddlmZmZejejZd Zdd d d Zd ZdZdddZddZddZdZ dZ!ddZ"ddZ#dZ$dZ%dZ&dZ'dZ(dZ)dZ*ddZ+dZ,d Z-d!Z.e/d"Z0e/d#Z1d$Z2d%Z3dS(&s+Functions used by least-squares algorithms.i(tdivisiontprint_functiontabsolute_import(tcopysignN(tnorm(t cho_factort cho_solvet LinAlgError(tissparse(tLinearOperatortaslinearoperatorc C`stj||}|dkr-tdntj||}tj|||d}|dkrttdntj||||}|t|| }||}||} || kr|| fS| |fSdS(s}Find the intersection of a line with the boundary of a trust region. This function solves the quadratic equation with respect to t ||(x + s*t)||**2 = Delta**2. Returns ------- t_neg, t_pos : tuple of float Negative and positive roots. Raises ------ ValueError If `s` is zero or `x` is not within the trust region. is `s` is zero.is#`x` is not within the trust region.N(tnptdott ValueErrortsqrtR( txtstDeltatatbtctdtqtt1tt2((s9/tmp/pip-build-7oUkmx/scipy/scipy/optimize/_lsq/common.pytintersect_trust_regions      g{Gz?i c  C`sd} ||} ||krDt||d} |d| k} nt} | r|j|| } t| |kr| ddfSnt| |}| r| d| ||\}}| |}nd}|d ks| r|dkrtd|||d}n|}xt|D]}||ks9||krWtd|||d}n| || ||\}}|dkr|}n||}t|||}|||||8}tj|||krPqqW|j| |d| } | |t| 9} | ||dfS( sSolve a trust-region problem arising in least-squares minimization. This function implements a method described by J. J. More [1]_ and used in MINPACK, but it relies on a single SVD of Jacobian instead of series of Cholesky decompositions. Before running this function, compute: ``U, s, VT = svd(J, full_matrices=False)``. Parameters ---------- n : int Number of variables. m : int Number of residuals. uf : ndarray Computed as U.T.dot(f). s : ndarray Singular values of J. V : ndarray Transpose of VT. Delta : float Radius of a trust region. initial_alpha : float, optional Initial guess for alpha, which might be available from a previous iteration. If None, determined automatically. rtol : float, optional Stopping tolerance for the root-finding procedure. Namely, the solution ``p`` will satisfy ``abs(norm(p) - Delta) < rtol * Delta``. max_iter : int, optional Maximum allowed number of iterations for the root-finding procedure. Returns ------- p : ndarray, shape (n,) Found solution of a trust-region problem. alpha : float Positive value such that (J.T*J + alpha*I)*p = -J.T*f. Sometimes called Levenberg-Marquardt parameter. n_iter : int Number of iterations made by root-finding procedure. Zero means that Gauss-Newton step was selected as the solution. References ---------- .. [1] More, J. J., "The Levenberg-Marquardt Algorithm: Implementation and Theory," Numerical Analysis, ed. G. A. Watson, Lecture Notes in Mathematics 630, Springer Verlag, pp. 105-116, 1977. cS`sR|d|}t||}||}tj|d|d |}||fS(sFunction of which to find zero. It is defined as "norm of regularized (by alpha) least-squares solution minus `Delta`". Refer to [1]_. ii(RR tsum(talphatsufRRtdenomtp_normtphit phi_prime((s9/tmp/pip-build-7oUkmx/scipy/scipy/optimize/_lsq/common.pytphi_and_derivativels   iiggMbP?g?iiN( tEPStFalseR RtNonetmaxtrangeR tabs(tntmtufRtVRt initial_alphatrtoltmax_iterR!Rt thresholdt full_ranktpt alpha_upperRR t alpha_lowerRtittratio((s9/tmp/pip-build-7oUkmx/scipy/scipy/optimize/_lsq/common.pytsolve_lsq_trust_region;s@1     cC`syRt|\}}t||f| }tj|||dkrQ|tfSWntk renX|d|d}|d |d}|d |d}|d|} |d|} tj| | d||| d|d| || | | g} tj| } tj| tj | } |tj d| d| dd| dd| df}dtj ||j|ddtj||} tj | }|dd|f}|t fS( sSolve a general trust-region problem in 2 dimensions. The problem is reformulated as a 4-th order algebraic equation, the solution of which is found by numpy.roots. Parameters ---------- B : ndarray, shape (2, 2) Symmetric matrix, defines a quadratic term of the function. g : ndarray, shape (2,) Defines a linear term of the function. Delta : float Radius of a trust region. Returns ------- p : ndarray, shape (2,) Found solution. newton_step : bool Whether the returned solution is the Newton step which lies within the trust region. iiiig?taxisN(ii(ii(ii(RRR R tTrueRtarraytrootstrealtisrealtvstackRtargminR#(tBtgRtRtlowerR1RRRRtftcoeffstttvalueti((s9/tmp/pip-build-7oUkmx/scipy/scipy/optimize/_lsq/common.pytsolve_trust_region_2ds* ?=6cC`sa|dkr||}nd}|dkr8d|}n|dkrW|rW|d9}n||fS(sUpdate the radius of a trust region based on the cost reduction. Returns ------- Delta : float New radius. ratio : float Ratio between actual and predicted reductions. Zero if predicted reduction is zero. ig?g?g@((Rtactual_reductiontpredicted_reductiont step_normt bound_hitR5((s9/tmp/pip-build-7oUkmx/scipy/scipy/optimize/_lsq/common.pytupdate_tr_radiuss     c C`s|j|}tj||}|dk rJ|tj|||7}n|d9}tj||}|dk r|j|}|tj||7}dtj||tj||} |dk r|tj|||7}| dtj|||7} n||| fS||fSdS(sParameterize a multivariate quadratic function along a line. The resulting univariate quadratic function is given as follows: :: f(t) = 0.5 * (s0 + s*t).T * (J.T*J + diag) * (s0 + s*t) + g.T * (s0 + s*t) Parameters ---------- J : ndarray, sparse matrix or LinearOperator shape (m, n) Jacobian matrix, affects the quadratic term. g : ndarray, shape (n,) Gradient, defines the linear term. s : ndarray, shape (n,) Direction vector of a line. diag : None or ndarray with shape (n,), optional Addition diagonal part, affects the quadratic term. If None, assumed to be 0. s0 : None or ndarray with shape (n,), optional Initial point. If None, assumed to be 0. Returns ------- a : float Coefficient for t**2. b : float Coefficient for t. c : float Free term. Returned only if `s0` is provided. g?N(R R R$( tJR@Rtdiagts0tvRRtuR((s9/tmp/pip-build-7oUkmx/scipy/scipy/optimize/_lsq/common.pytbuild_quadratic_1ds   & ! c C`s||g}|dkrUd||}||ko=|knrU|j|qUntj|}||d|||}tj|}||||fS(sMinimize a 1-d quadratic function subject to bounds. The free term `c` is 0 by default. Bounds must be finite. Returns ------- t : float Minimum point. y : float Minimum value. igi(tappendR tasarrayR>( RRtlbtubRREtextremumtyt min_index((s9/tmp/pip-build-7oUkmx/scipy/scipy/optimize/_lsq/common.pytminimize_quadratic_1d/s  cC`s|jdkr\|j|}tj||}|dk r|tj|||7}qn[|j|j}tj|ddd}|dk r|tj||ddd7}ntj||}d||S(sCompute values of a quadratic function arising in least squares. The function is 0.5 * s.T * (J.T * J + diag) * s + g.T * s. Parameters ---------- J : ndarray, sparse matrix or LinearOperator, shape (m, n) Jacobian matrix, affects the quadratic term. g : ndarray, shape (n,) Gradient, defines the linear term. s : ndarray, shape (k, n) or (n,) Array containing steps as rows. diag : ndarray, shape (n,), optional Addition diagonal part, affects the quadratic term. If None, assumed to be 0. Returns ------- values : ndarray with shape (k,) or float Values of the function. If `s` was 2-dimensional then ndarray is returned, otherwise float is returned. iiR7ig?N(tndimR R R$tTR(RNR@RROtJsRtl((s9/tmp/pip-build-7oUkmx/scipy/scipy/optimize/_lsq/common.pytevaluate_quadraticFs   $cC`stj||k||k@S(s$Check if a point lies within bounds.(R tall(RRVRW((s9/tmp/pip-build-7oUkmx/scipy/scipy/optimize/_lsq/common.pyt in_boundspscC`stj|}||}tj|}|jtjtjdd3tj|||||||||| 0 and lb[i] > -np.inf | 1, otherwise According to this definition v[i] >= 0 for all i. It differs from the definition in paper [1]_ (eq. (2.2)), where the absolute value of v is used. Both definitions are equivalent down the line. Derivatives of v with respect to x take value 1, -1 or 0 depending on a case. Returns ------- v : ndarray with shape of x Scaling vector. dv : ndarray with shape of x Derivatives of v[i] with respect to x[i], diagonal elements of v's Jacobian. References ---------- .. [1] M.A. Branch, T.F. Coleman, and Y. Li, "A Subspace, Interior, and Conjugate Gradient Method for Large-Scale Bound-Constrained Minimization Problems," SIAM Journal on Scientific Computing, Vol. 21, Number 1, pp 1-23, 1999. iii(R t ones_likeRvRw(RR@RVRWRQtdvtmask((s9/tmp/pip-build-7oUkmx/scipy/scipy/optimize/_lsq/common.pytCL_scaling_vectors  c C`st|||r%|tj|fStj|}tj|}|j}tj|dt}||@}tj||d||||||<||||k||<||@}tj||d||||||<||||k||<||@}||}tj ||||d||} ||tj| d||| ||<| ||k||t|tr|t ||}n||ddtj f9}|S(slCompute diag(d) J. If `copy` is False, `J` is modified in place (unless being LinearOperator). N( RR RRRR trepeattdifftindptrRR(RNRR((s9/tmp/pip-build-7oUkmx/scipy/scipy/optimize/_lsq/common.pyt left_multiplys *c C`s\|||ko|dk}||||k}|r@|r@dS|rJdS|rTdSdSdS(s8Check termination condition for nonlinear least squares.g?iiiN(R$( tdFtFtdx_normtx_normR5tftoltxtoltftol_satisfiedtxtol_satisfied((s9/tmp/pip-build-7oUkmx/scipy/scipy/optimize/_lsq/common.pytcheck_terminations cC`sc|dd|d|d}t||tk<|dC}||d|9}t||dt|fS(sdScale Jacobian and residuals for a robust loss function. Arrays are modified in place. iig?R(R"RR#(RNRCtrhotJ_scale((s9/tmp/pip-build-7oUkmx/scipy/scipy/optimize/_lsq/common.pytscale_for_robust_loss_functions  (4t__doc__t __future__RRRtmathRtnumpyR t numpy.linalgRt scipy.linalgRRRt scipy.sparseRtscipy.sparse.linalgR R tfinfotfloattepsR"RR$R6RHRMRSR[R`RbRtRRRRRRRRRRRRRR8RRRR(((s9/tmp/pip-build-7oUkmx/scipy/scipy/optimize/_lsq/common.pytsF  'q 3 3  *   '  , "