U Kv f!2 @sddlZddlmZddlmmZddlm Z ddZ dddZ dddZ GdddeZ GdddejZeee dS)N)Results)cache_readonlyc s@fdd}fdd}fdd}|||fS)ao Negative penalized log-likelihood functions. Returns the negative penalized log-likelihood, its derivative, and its Hessian. The penalty only includes the smooth (L2) term. All three functions have argument signature (x, model), where ``x`` is a point in the parameter space and ``model`` is an arbitrary statsmodels regression model. csJ|j}dt|dd}|jtj|f}| ||S)N)nobsnpsumZlogliker_)paramsmodelrZpen_llfZllf)L1_wtalphak loglike_kwds@/tmp/pip-unpacked-wheel-2v6byqio/statsmodels/base/elastic_net.py nploglike)s"z_gen_npfuncs..nploglikecs@|j}d|}|jtj|fd |}||S)Nrr)rZscorerr )r r rZpen_gradZgr)r r r score_kwdsrrnpscore/sz_gen_npfuncs..npscorecs<|j}d}|jtj|fd ||}|S)Nr)rr)rZhessianrr )r r rZpen_hessh)r r hess_kwdsrrrnphess5s"z_gen_npfuncs..nphessr) rr r rrrrrrr)r r rrrrr _gen_npfuncss r coord_descentd?Hz>:0yE>FTc ( sjjd} dkrindkr(indkr8intrTt| |dkrht| }n|}d}tjt|td} }d|d<| dd}d|kr|ddk r|dkrt | d}n|t | d7}fd d t | D}d}t |D]}|}t | D]}||r6q$|}d ||<t j|}|dk rf||7}jjjdd|ffd|i|}||\}}}t||||||||| d ||<|d kr$t|||kr$d ||<d||<q$tt||}||krd }q"qd |t||k<|sRt|}||_t|St|} t| | f}!tfdd jD}t| d krֈjjjdd| ff|}"|"}#|#j|| <|#j|!t| | <n,jjjddd ff|}"|"jd d}#t|#jtjr|#j j}$n|#j}$t!|#dr6|#j"}%nd}%j#j$}&}'t| _#j%j#_$|$||!|%d}d |_&||_||_'d|di|_(|&|'_#_$|S)a Return an elastic net regularized fit to a regression model. Parameters ---------- model : model object A statsmodels object implementing ``loglike``, ``score``, and ``hessian``. method : {'coord_descent'} Only the coordinate descent algorithm is implemented. maxiter : int The maximum number of iteration cycles (an iteration cycle involves running coordinate descent on all variables). alpha : scalar or array_like The penalty weight. If a scalar, the same penalty weight applies to all variables in the model. If a vector, it must have the same length as `params`, and contains a penalty weight for each coefficient. L1_wt : scalar The fraction of the penalty given to the L1 penalty term. Must be between 0 and 1 (inclusive). If 0, the fit is a ridge fit, if 1 it is a lasso fit. start_params : array_like Starting values for `params`. cnvrg_tol : scalar If `params` changes by less than this amount (in sup-norm) in one iteration cycle, the algorithm terminates with convergence. zero_tol : scalar Any estimated coefficient smaller than this value is replaced with zero. refit : bool If True, the model is refit using only the variables that have non-zero coefficients in the regularized fit. The refitted model is not regularized. check_step : bool If True, confirm that the first step is an improvement and search further if it is not. loglike_kwds : dict-like or None Keyword arguments for the log-likelihood function. score_kwds : dict-like or None Keyword arguments for the score function. hess_kwds : dict-like or None Keyword arguments for the Hessian function. Returns ------- Results A results object. Notes ----- The ``elastic net`` penalty is a combination of L1 and L2 penalties. The function that is minimized is: -loglike/n + alpha*((1-L1_wt)*|params|_2^2/2 + L1_wt*|params|_1) where |*|_1 and |*|_2 are the L1 and L2 norms. The computational approach used here is to obtain a quadratic approximation to the smooth part of the target function: -loglike/n + alpha*(1-L1_wt)*|params|_2^2/2 then repeatedly optimize the L1 penalized version of this function along coordinate axes. rNg-C6?)ZdtypeFhasconstoffsetZexposurec sg|]}t|qSr)r.0r)r r rrrrr sz"fit_elasticnet..r)tol check_stepTrcsg|]}|t|dfqSN)getattrr!)r rrr#s)maxiterscaler)r) iteration))ZexogshaperZisscalarZoneszeroscopylenboolZ_get_init_kwdspoplograngedot __class__Zendog_opt_1dabsmaxRegularizedResults convergedRegularizedResultsWrapperZ flatnonzerodictZ _init_keysZfitr Znormalized_cov_paramsZix_ issubclasswrapResultsWrapperZ_resultshasattrr)Zdf_modelZdf_residrZ regularizedmethodZ fit_history)(r r@r(r r Z start_paramsZ cnvrg_tolZzero_tolZrefitr%rrrZk_exogr ZbtolZ params_zeroZ init_argsZ model_offsetZfgh_listr9itrZ params_saverZparams0r Z model_1varfuncgradhessZpchangeresultsiiZcovZmodel1Zrsltklassr)pqr)r r rrr rrfit_elasticnet>sJ                    rJcCs|}|||} |||} |||} | | |} |t| kr@dS| dkrV|| | } n| dkrn||  | } ntjS|s|| S||| ||t|| }|| |t|dkr|| Sddlm}|||f|d|df|d}|S)az One-dimensional helper for elastic net. Parameters ---------- func : function A smooth function of a single variable to be optimized with L1 penaty. grad : function The gradient of `func`. hess : function The Hessian of `func`. model : statsmodels model The model being fit. start : real A starting value for the function argument L1_wt : non-negative real The weight for the L1 penalty function. tol : non-negative real A convergence threshold. check_step : bool If True, check that the first step is an improvement and use bisection if it is not. If False, return after the first step regardless. Notes ----- ``func``, ``grad``, and ``hess`` have argument signature (x, model), where ``x`` is a point in the parameter space and ``model`` is the model being fit. If the log-likelihood for the model is exactly quadratic, the global minimum is returned in one step. Otherwise numerical bisection is used. Returns ------- The argmin of the objective function. rrg|=)brentr)argsZbrackr$)rr6nanZscipy.optimizerK)rBrCrDr startr r$r%xfbcdrf1rKZx_optrrrr5s(5      r5cs,eZdZdZfddZeddZZS)r8z Results for models estimated using regularization Parameters ---------- model : Model The model instance used to estimate the parameters. params : ndarray The estimated (regularized) parameters. cstt|||dSr&)superr8__init__)selfr r r4rrrVqszRegularizedResults.__init__cCs|j|jS)zR The predicted values from the model at the estimated parameters. )r Zpredictr )rWrrr fittedvaluestszRegularizedResults.fittedvalues)__name__ __module__ __qualname____doc__rVrrY __classcell__rrrXrr8fs r8c@seZdZddddZeZdS)r:columnsZrows)r ZresidrYN)rZr[r\_attrsZ _wrap_attrsrrrrr:|s r:) rrrrNrrFTNNN)T)ZnumpyrZstatsmodels.base.modelrZstatsmodels.base.wrapperbasewrapperr=Zstatsmodels.tools.decoratorsrrrJr5r8r>r:Zpopulate_wrapperrrrrs0  ! T V