U Kv fh;@s dZddlZddlmZmZeejZ dZ ddZ ddidfd d Z ddidfd d Z ddifd dZddifddZddifddZeddddddee ddidfddZeddddddee ddidfddZed d!d!d"d#dee ddifd$d%ZeZejd&7_dS)'aynumerical differentiation function, gradient, Jacobian, and Hessian Author : josef-pkt License : BSD Notes ----- These are simple forward differentiation, so that we have them available without dependencies. * Jacobian should be faster than numdifftools because it does not use loop over observations. * numerical precision will vary and depend on the choice of stepsizes N)Appender Substitutiona Calculate Hessian with finite difference derivative approximation Parameters ---------- x : array_like value at which function derivative is evaluated f : function function of one array f(x, `*args`, `**kwargs`) epsilon : float or array_like, optional Stepsize used, if None, then stepsize is automatically chosen according to EPS**(1/%(scale)s)*x. args : tuple Arguments for function `f`. kwargs : dict Keyword arguments for function `f`. %(extra_params)s Returns ------- hess : ndarray array of partial second derivatives, Hessian %(extra_returns)s Notes ----- Equation (%(equation_number)s) in Ridout. Computes the Hessian as:: %(equation)s where e[j] is a vector with element j == 1 and the rest are zero and d[i] is epsilon[i]. References ----------: Ridout, M.S. (2009) Statistical applications of the complex-step method of numerical differentiation. The American Statistician, 63, 66-74 cCsv|dkr.td|ttt|d}n>t|rNt|}||nt|}|j|jkrlt dt|S)Ng?g?z6If h is not a scalar it must have the same shape as x.) EPSnpmaximumabsasarrayZisscalaremptyfillshape ValueError)xsepsilonnhr=/tmp/pip-unpacked-wheel-2v6byqio/statsmodels/tools/numdiff.py _get_epsilon^s&     rrFc CsBt|}||f||}t|j}t|f|tt|j} t|ft} |st|d||}t |D]D} || | | <||| f||||| | | ddf<d| | <qhntt|d||d}t |D]X} || | | <||| f||||| f||d|| | | ddf<d| | <q|dkr4| j S| j SdS)aR Gradient of function, or Jacobian if function f returns 1d array Parameters ---------- x : ndarray parameters at which the derivative is evaluated f : function `f(*((x,)+args), **kwargs)` returning either one value or 1d array epsilon : float, optional Stepsize, if None, optimal stepsize is used. This is EPS**(1/2)*x for `centered` == False and EPS**(1/3)*x for `centered` == True. args : tuple Tuple of additional arguments for function `f`. kwargs : dict Dictionary of additional keyword arguments for function `f`. centered : bool Whether central difference should be returned. If not, does forward differencing. Returns ------- grad : ndarray gradient or Jacobian Notes ----- If f returns a 1d array, it returns a Jacobian. If a 2d array is returned by f (e.g., with a value for each observation), it returns a 3d array with the Jacobian of each observation with shape xk x nobs x xk. I.e., the Jacobian of the first observation would be [:, 0, :] Ng@) lenrZ atleast_1dr zerosZ promote_typesfloatZdtyperrangeTsqueeze) r frargskwargscenteredrf0Zdimgradeikrrr approx_fprimems.!   ,      r'c Cst|}d}||f||}|sNt|d||}|||f||||} nBt|d||d}|||f|||||f||d|} | S)a' Gradient of function vectorized for scalar parameter. This assumes that the function ``f`` is vectorized for a scalar parameter. The function value ``f(x)`` has then the same shape as the input ``x``. The derivative returned by this function also has the same shape as ``x``. Parameters ---------- x : ndarray Parameters at which the derivative is evaluated. f : function `f(*((x,)+args), **kwargs)` returning either one value or 1d array epsilon : float, optional Stepsize, if None, optimal stepsize is used. This is EPS**(1/2)*x for `centered` == False and EPS**(1/3)*x for `centered` == True. args : tuple Tuple of additional arguments for function `f`. kwargs : dict Dictionary of additional keyword arguments for function `f`. centered : bool Whether central difference should be returned. If not, does forward differencing. Returns ------- grad : ndarray Array of derivatives, gradient evaluated at parameters ``x``. rrrr)rrr) r rrr r!r"rr#epsr$rrr_approx_fprime_scalars r)csRt}td|t|d}fddt|D}t|jS)a Calculate gradient or Jacobian with complex step derivative approximation Parameters ---------- x : ndarray parameters at which the derivative is evaluated f : function `f(*((x,)+args), **kwargs)` returning either one value or 1d array epsilon : float, optional Stepsize, if None, optimal stepsize is used. Optimal step-size is EPS*x. See note. args : tuple Tuple of additional arguments for function `f`. kwargs : dict Dictionary of additional keyword arguments for function `f`. Returns ------- partials : ndarray array of partial derivatives, Gradient or Jacobian Notes ----- The complex-step derivative has truncation error O(epsilon**2), so truncation error can be eliminated by choosing epsilon to be very small. The complex-step derivative avoids the problem of round-off error with small epsilon because there is no subtraction. r?cs.g|]&\}}|fj|qSr)imag).0iZihr rrr!r rr sz$approx_fprime_cs..)rrridentity enumeratearrayr)r rrr r!rZ incrementspartialsrr.rapprox_fprime_css!r4cCsNt|}|jd}t|d||}d|}|||f||j|}t|S)a Calculate gradient for scalar parameter with complex step derivatives. This assumes that the function ``f`` is vectorized for a scalar parameter. The function value ``f(x)`` has then the same shape as the input ``x``. The derivative returned by this function also has the same shape as ``x``. Parameters ---------- x : ndarray Parameters at which the derivative is evaluated. f : function `f(*((x,)+args), **kwargs)` returning either one value or 1d array. epsilon : float, optional Stepsize, if None, optimal stepsize is used. Optimal step-size is EPS*x. See note. args : tuple Tuple of additional arguments for function `f`. kwargs : dict Dictionary of additional keyword arguments for function `f`. Returns ------- partials : ndarray Array of derivatives, gradient evaluated for parameters ``x``. Notes ----- The complex-step derivative has truncation error O(epsilon**2), so truncation error can be eliminated by choosing epsilon to be very small. The complex-step derivative avoids the problem of round-off error with small epsilon because there is no subtraction. rr*)rrr rr+r2)r rrr r!rr(r3rrr_approx_fprime_cs_scalars %  r6c Cst|}t|d||}t|}t||}t|}t|D]} t| |D]} t||d|| ddf|| ddff||||d|| ddf|| ddff||jd|| | f|| | f<|| | f|| | f<qJq<|S)aCalculate Hessian with complex-step derivative approximation Parameters ---------- x : array_like value at which function derivative is evaluated f : function function of one array f(x) epsilon : float stepsize, if None, then stepsize is automatically chosen Returns ------- hess : ndarray array of partial second derivatives, Hessian Notes ----- based on equation 10 in M. S. RIDOUT: Statistical Applications of the Complex-step Method of Numerical Differentiation, University of Kent, Canterbury, Kent, U.K. The stepsize is the same for the complex and the finite difference part. rr*Nr)rrrdiagouterrrr+ r rrr r!rreehessr-jrrrapprox_hess_cs0s(   2.  r=3zFreturn_grad : bool Whether or not to also return the gradient z7grad : nparray Gradient if return_grad == True 7zB1/(d_j*d_k) * ((f(x + d[j]*e[j] + d[k]*e[k]) - f(x + d[j]*e[j]))) )ZscaleZ extra_paramsZ extra_returnsZequation_numberZequationcCs$t|}t|d||}t|}||f||} t|} t|D](} |||| ddff||| | <qBt||} t|D]} t| |D]p} |||| ddf|| ddff||| | | | | | | | f| | | f<| | | f| | | f<qq|r| | |}| |fS| SdS)Nrrrrr7rrr8)r rrr r! return_gradrrr:r#gr-r;r<r$rrr approx_hess1]s0   &  .  rCz7grad : ndarray Gradient if return_grad == True 8z1/(2*d_j*d_k) * ((f(x + d[j]*e[j] + d[k]*e[k]) - f(x + d[j]*e[j])) - (f(x + d[k]*e[k]) - f(x)) + (f(x - d[j]*e[j] - d[k]*e[k]) - f(x + d[j]*e[j])) - (f(x - d[k]*e[k]) - f(x))) c Cst|}t|d||}t|}||f||} t|} t|} t|D]L} |||| ddff||| | <|||| ddff||| | <qLt||} t|D]} t| |D]}|||| ddf||ddff||| | | || |||| ddf||ddff||| | | || d| | |f| | |f<| | |f| || f<qq|r| | |}| |fS| SdS)Nrrr@)r rrr r!rArrr:r#rBZggr-r;r<r$rrr approx_hess2sD    $&  ..  rE49a 1/(4*d_j*d_k) * ((f(x + d[j]*e[j] + d[k]*e[k]) - f(x + d[j]*e[j] - d[k]*e[k])) - (f(x - d[j]*e[j] + d[k]*e[k]) - f(x - d[j]*e[j] - d[k]*e[k]))c CsBt|}t|d||}t|}t||}t|D]} t| |D]} t|||| ddf|| ddff|||||| ddf|| ddff|||||| ddf|| ddff|||||| ddf|| ddff||d|| | f|| | f<|| | f|| | f<qDq4|S)Ng@)rrrr7r8rrr9rrr approx_hess3s&   .... rJz& This is an alias for approx_hess3)__doc__ZnumpyrZstatsmodels.compat.pandasrrZfinforr(rZ _hessian_docsrr'r)r4r6r=rCrErJZ approx_hessrrrrsR- ): .,/-