U Kv fwOã@sàddlZddlZddlZddlmZddlmm Z ddl m Z Gdd„dej ƒZGdd„deƒZGdd „d eƒZGd d „d eƒZGd d „d ejƒZGdd„de jƒZe  ee¡dd„ZGdd„deƒZeZeZeZeZdS)éN)Úmodel)ÚConvergenceWarningcs(eZdZdZ‡fdd„Zdd„Z‡ZS)Ú_DimReductionRegressionzB A base class for dimension reduction regression methods. c stt|ƒj||f|ŽdS©N)ÚsuperrÚ__init__©ÚselfÚendogÚexogÚkwargs©Ú __class__©úA/tmp/pip-unpacked-wheel-2v6byqio/statsmodels/regression/dimred.pyrsz _DimReductionRegression.__init__cCs€t |j¡}|j|dd…f}|| d¡8}t |j|¡|jd}tj  |¡}tj  ||j¡j}||_ ||_ t  ||¡|_dS©Nr)ÚnpÚargsortr r ÚmeanÚdotÚTÚshapeÚlinalgZcholeskyÚsolveÚwexogÚ_covxrÚ array_splitÚ _split_wexog)r Ún_sliceÚiiÚxÚcovxZcovxrrrrÚ_preps  z_DimReductionRegression._prep)Ú__name__Ú __module__Ú __qualname__Ú__doc__rr"Ú __classcell__rrr rr s rc@s4eZdZdZddd„Zdd„Zdd„Zdd d„Zd S)ÚSlicedInverseRega0 Sliced Inverse Regression (SIR) Parameters ---------- endog : array_like (1d) The dependent variable exog : array_like (2d) The covariates References ---------- KC Li (1991). Sliced inverse regression for dimension reduction. JASA 86, 316-342. éc Ksêt|ƒdkrd}t |¡|jjd|}| |¡dd„|jDƒ}dd„|jDƒ}t |¡}t |¡}t  |j |dd…df|¡|  ¡}tj   |¡\}} t | ¡} || }| dd…| f} tj  |jj | ¡} t|| |d} t| ƒS)zÄ Estimate the EDR space using Sliced Inverse Regression. Parameters ---------- slice_n : int, optional Target number of observations per slice rz1SIR.fit does not take any extra keyword argumentscSsg|]}| d¡‘qS©r©r©Ú.0ÚzrrrÚ Jsz(SlicedInverseReg.fit..cSsg|]}|jd‘qSr*©rr,rrrr/KsN©Úeigs)ÚlenÚwarningsÚwarnr rr"rrÚasarrayrrÚsumrÚeighrrrÚDimReductionResultsÚDimReductionResultsWrapper) r Úslice_nr ÚmsgrÚmnÚnZmncÚaÚbÚjjÚparamsÚresultsrrrÚfit6s"     & zSlicedInverseReg.fitcCsÆ|j}|j}|j}|j}d}t |||jf¡}t|jƒD]0}t |j |dd…|f¡}|t  ||¡7}q8t ||¡} tj   | ¡\} } t | t | j |j ¡¡} |j | } |t || |   d¡¡7}|Sr)Úk_varsÚ_covxÚ _slice_meansÚ _slice_propsrÚreshapeÚndimÚrangerÚpen_matr7rZqrr)r ÚAÚpr!r=ÚphÚvÚkÚuÚcovxaÚqÚ_ZqdZqurrrÚ_regularized_objective[s  z'SlicedInverseReg._regularized_objectivec Cs@|j}|j}|j}|j}|j}|j}| ||f¡}dt |j j t |j |¡¡}| ||f¡}t ||¡} t || ¡} t | j | ¡} tj   | ¡} t  ||f¡} tj  | | j ¡}dg||}t|ƒD]¶}t|ƒD]¨}| d9} d| ||f<t | j | ¡}||j 7}t | t || ¡¡ }t t || ¡|¡}|t | t || j ¡¡7}|t | tj  | t | j |¡¡¡7}|||||<qÐqÄtj  | t | j |j ¡¡}|t | |¡j }t|ƒD]†}||dd…f}||dd…f}t|ƒD]V}t|ƒD]F}t |t |||||¡¡}|||fd|||8<qèqÜq°| ¡S)Néré)rErJrFrrGrHrIrrrLrrÚinvÚzerosrrKÚravel)r rMrNrJr!rr=rOZgrrSZcovx2aÚQZQiZjmZqcvÚftrTÚrZumatZfmatÚchZcuÚirRrPÚfrrrÚ_regularized_gradssJ       $    ,z"SlicedInverseReg._regularized_gradrXNédçü©ñÒMbP?cKsÀt|ƒdkrd}t |¡|dkr*tdƒ‚| dd¡}| dd¡}|jjd|} t |j ¡} |j| dd…f} | |   d¡8} t  | j ¡} t  | | ¡} dd „| Dƒ}d d „| Dƒ}t |¡}t |¡}|| ¡|_||_| jd|_||_| |_| |_||_|dkr8t |j|f¡}t |¡|d|…d|…f<|}n |jd |krTd }t|ƒ‚|}t||j|j||ƒ\}}}|sª| | ¡¡}t t ||¡¡}d |}t |¡t||dd}t |ƒS)a© Estimate the EDR space using regularized SIR. Parameters ---------- ndim : int The number of EDR directions to estimate pen_mat : array_like A 2d array such that the squared Frobenius norm of `dot(pen_mat, dirs)`` is added to the objective function, where `dirs` is an orthogonal array whose columns span the estimated EDR space. slice_n : int, optional Target number of observations per slice maxiter :int The maximum number of iterations for estimating the EDR space. gtol : float If the norm of the gradient of the objective function falls below this value, the algorithm has converged. Returns ------- A results class instance. Notes ----- If each row of `exog` can be viewed as containing the values of a function evaluated at equally-spaced locations, then setting the rows of `pen_mat` to [[1, -2, 1, ...], [0, 1, -2, 1, ..], ...] will give smooth EDR coefficients. This is a form of "functional SIR" using the squared second derivative as a penalty. References ---------- L. Ferre, A.F. Yao (2003). Functional sliced inverse regression analysis. Statistics: a journal of theoretical and applied statistics 37(6) 475-488. rz3SIR.fit_regularized does not take keyword argumentsNzpen_mat is a required argumentÚ start_paramsr;r)cSsg|]}| d¡‘qSr*r+r,rrrr/åsz4SlicedInverseReg.fit_regularized..cSsg|]}|jd‘qSr*r0r,rrrr/æsrXz1Shape of start_params is not compatible with ndimz,SIR.fit_regularized did not converge, |g|=%fr1)!r3r4r5Ú ValueErrorÚgetr rrrr rÚcovrrr6r7rHrJrErLrFrrGrZÚeyeÚ _grass_optrVrbr[Úsqrtrr9r:)r rJrLr;ÚmaxiterÚgtolr r<rerrr r!Z split_exogr=r>rBrUÚcnvrgÚgÚgnrCrrrÚfit_regularized¢sX*           ÿ  z SlicedInverseReg.fit_regularized)r))rXNr)rcrd)r#r$r%r&rDrVrbrqrrrrr(%s %/ÿr(c@seZdZdZdd„ZdS)ÚPrincipalHessianDirectionsa Principal Hessian Directions (PHD) Parameters ---------- endog : array_like (1d) The dependent variable exog : array_like (2d) The covariates Returns ------- A model instance. Call `fit` to obtain a results instance, from which the estimated parameters can be obtained. References ---------- KC Li (1992). On Principal Hessian Directions for Data Visualization and Dimension Reduction: Another application of Stein's lemma. JASA 87:420. cKsØ| dd¡}|j|j ¡}|j|j d¡}|rRddlm}|||ƒ ¡}|j}t  d|||¡}|t |ƒ}t  |j ¡}tj  ||¡} tj  | ¡\} } t t | ¡ ¡} | | } | dd…| f} t|| | d}t|ƒS)aŸ Estimate the EDR space using PHD. Parameters ---------- resid : bool, optional If True, use least squares regression to remove the linear relationship between each covariate and the response, before conducting PHD. Returns ------- A results instance which can be used to access the estimated parameters. ÚresidFr)ÚOLSz i,ij,ik->jkNr1)rgr rr Z#statsmodels.regression.linear_modelrtrDrsrZeinsumr3rhrrrZeigrÚabsr9r:)r r rsÚyr rtr^ÚcmZcxÚcbr?r@rArBrCrrrrDs"    zPrincipalHessianDirections.fitN)r#r$r%r&rDrrrrrrsrrcs(eZdZdZ‡fdd„Zdd„Z‡ZS)ÚSlicedAverageVarianceEstimationa] Sliced Average Variance Estimation (SAVE) Parameters ---------- endog : array_like (1d) The dependent variable exog : array_like (2d) The covariates bc : bool, optional If True, use the bias-corrected CSAVE method of Li and Zhu. References ---------- RD Cook. SAVE: A method for dimension reduction and graphics in regression. http://www.stat.umn.edu/RegGraph/RecentDev/save.pdf Y Li, L-X Zhu (2007). Asymptotics for sliced average variance estimation. The Annals of Statistics. https://arxiv.org/pdf/0708.0462.pdf c s:tt|ƒj||f|Žd|_d|kr6|ddkr6d|_dS)NFÚbcT)rÚSAVErrzrr rrrasz(SlicedAverageVarianceEstimation.__init__cKs| dd¡}|jjd|}| |¡dd„|jDƒ}dd„|jDƒ}|jjd}|js¢d}t||ƒD]*\}} t  |¡| } ||t  | | ¡7}qf|t |ƒ}nd} |D]} | t  | | ¡7} qª| t |ƒ} d} |jD]R}||  d¡}t |jdƒD]0}||dd…f}t ||¡}| t  ||¡7} qöqÖ| |jjd} t  |¡} | | d| dd d}| d| dd d}|| || }t  |¡d t|ƒt |ƒ|}tj |¡\}}t | ¡}||}|dd…|f}tj |jj|¡}t|||d }t|ƒS) z“ Estimate the EDR space. Parameters ---------- slice_n : int Number of observations per slice r;é2rcSsg|]}t |j¡‘qSr)rrhrr,rrrr/zsz7SlicedAverageVarianceEstimation.fit..cSsg|]}|jd‘qSr*r0r,rrrr/{srXNrWr1)rgr rr"rrrzÚziprrirr3rrKÚouterr7rr8rrrrr9r:)r r r;rZcvÚnsrNZvmÚwZcvxZicvÚavÚcZvnr r^r`rRÚmZk1Zk2Zav2r?r@rArBrCrrrrDhsH       " z#SlicedAverageVarianceEstimation.fit)r#r$r%r&rrDr'rrr rryIs rycs eZdZdZ‡fdd„Z‡ZS)r9aV Results class for a dimension reduction regression. Notes ----- The `params` attribute is a matrix whose columns span the effective dimension reduction (EDR) space. Some methods produce a corresponding set of eigenvalues (`eigs`) that indicate how much information is contained in each basis direction. cstt|ƒ ||¡||_dSr)rr9rr2)r rrBr2r rrr·s  ÿzDimReductionResults.__init__)r#r$r%r&rr'rrr rr9ªs r9c@seZdZddiZeZdS)r:rBÚcolumnsN)r#r$r%Ú_attrsZ _wrap_attrsrrrrr:½sÿr:cs|j\}}| ¡}||ƒ}d}t|ƒD]Ö} ||ƒ} | t | |¡|t ||¡8} t t | | ¡¡|krrd}qþ|  ||f¡} tj  | d¡\‰‰‰| ||f¡} t | ˆj ¡‰‡‡‡‡fdd„} d}|dkr&| | ƒ}||ƒ}||krò|}|}q&|d}qÆq&| ||f¡}|||fS) a Minimize a function on a Grassmann manifold. Parameters ---------- params : array_like Starting value for the optimization. fun : function The function to be minimized. grad : function The gradient of fun. maxiter : int The maximum number of iterations. gtol : float Convergence occurs when the gradient norm falls below this value. Returns ------- params : array_like The minimizing value for the objective function. fval : float The smallest achieved value of the objective function. cnvrg : bool True if the algorithm converged to a limit point. Notes ----- `params` is 2-d, but `fun` and `grad` should take 1-d arrays `params.ravel()` as arguments. Reference --------- A Edelman, TA Arias, ST Smith (1998). The geometry of algorithms with orthogonality constraints. SIAM J Matrix Anal Appl. http://math.mit.edu/~edelman/publications/geometry_of_algorithms.pdf FTrcs4ˆt ˆ|¡ˆt ˆ|¡}t |ˆ¡ ¡Sr)rÚcosÚsinrr[)ÚtÚpa©Zpa0ÚsrRZvtrrÚgeos$z_grass_opt..geog@g»½×Ùß|Û=rW) rr[rKrrrkr7rIrZsvdr)rBZfunZgradrlrmrNÚdZf0rnrUroZgmZparamsmrŒÚstepr‰Úf1rrŠrrjÇs4&     rjcs:eZdZdZ‡fdd„Zdd„Zdd„Zd d d „Z‡ZS)ÚCovarianceReductiona/ Dimension reduction for covariance matrices (CORE). Parameters ---------- endog : array_like The dependent variable, treated as group labels exog : array_like The independent variables. dim : int The dimension of the subspace onto which the covariance matrices are projected. Returns ------- A model instance. Call `fit` on the model instance to obtain a results instance, which contains the fitted model parameters. Notes ----- This is a likelihood-based dimension reduction procedure based on Wishart models for sample covariance matrices. The goal is to find a projection matrix P so that C_i | P'C_iP and C_j | P'C_jP are equal in distribution for all i, j, where the C_i are the within-group covariance matrices. The model and methodology are as described in Cook and Forzani. The optimization method follows Edelman et. al. References ---------- DR Cook, L Forzani (2008). Covariance reducing models: an alternative to spectral modeling of covariance matrices. Biometrika 95:4. A Edelman, TA Arias, ST Smith (1998). The geometry of algorithms with orthogonality constraints. SIAM J Matrix Anal Appl. http://math.mit.edu/~edelman/publications/geometry_of_algorithms.pdf c s¾tt|ƒ ||¡gg}}tj|j|jd}| |j¡D](\}}|  |  ¡j ¡|  |j d¡q:t |ƒ|_d} t|ƒD]\} }| || || 7} qz| |j} | |_||_||_||_dS)N)Úindexr)rrrÚpdZ DataFramer r Úgroupbyr‘ÚappendrhÚvaluesrr3ÚnobsÚ enumerateÚcovmÚcovsrÚdim) r r r ršr™rZdfrUrPr˜r`r rrr@s   zCovarianceReduction.__init__c Cs¦|jjd}| ||jf¡}t |jt |j|¡¡}tj |¡\}}|j |d}t |j ƒD]D\}}t |jt ||¡¡}tj |¡\}}||j ||d8}q\|S)zô Evaluate the log-likelihood Parameters ---------- params : array_like The projection matrix used to reduce the covariances, flattened to 1d. Returns the log-likelihood. rrW) r˜rrIršrrrrZslogdetr–r—r™r) r rBrNÚprojr‚rUZldetraÚjrrrÚloglikeWs zCovarianceReduction.loglikec Cs¸|jjd}| ||jf¡}t |jt |j|¡¡}t |j|¡}|jtj  ||j¡j}t |j ƒD]J\}}t |jt ||¡¡}t ||¡}||j |tj  ||j¡j8}qd|  ¡S)a  Evaluate the score function. Parameters ---------- params : array_like The projection matrix used to reduce the covariances, flattened to 1d. Returns the score function evaluated at 'params'. r)r˜rrIršrrrr–rrr—r™rr[) r rBrNr›Zc0ZcProrœr‚rrrÚscorers  "zCovarianceReduction.scoreNéÈç-Cëâ6?c sЈjjd}ˆj}|dkrHt ||f¡}t |¡|d|…d|…f<|}n|}t|‡fdd„‡fdd„||ƒ\}}}|d9}|s´ˆ | ¡¡} t  t  | | ¡¡} d| } t   | t ¡tˆ|dd} || _t| ƒS) aö Fit the covariance reduction model. Parameters ---------- start_params : array_like Starting value for the projection matrix. May be rectangular, or flattened. maxiter : int The maximum number of gradient steps to take. gtol : float Convergence criterion for the gradient norm. Returns ------- A results instance that can be used to access the fitted parameters. rNcs ˆ |¡ Sr)r©r ©r rrÚ®óz)CovarianceReduction.fit..cs ˆ |¡ Sr)ržr¡r¢rrr£¯r¤éÿÿÿÿz/CovReduce optimization did not converge, |g|=%fr1)r˜rršrrZrirjržr[rkr7r4r5rr9Úllfr:) r rerlrmrNrrBr¦rnrorpr<rCrr¢rrDs*  þ  zCovarianceReduction.fit)NrŸr ) r#r$r%r&rrržrDr'rrr rrs ' r)r4ZnumpyrZpandasr’Zstatsmodels.baserZstatsmodels.base.wrapperÚbaseÚwrapperÚwrapZstatsmodels.tools.sm_exceptionsrZModelrr(rrryZResultsr9ZResultsWrapperr:Zpopulate_wrapperrjrZSIRZPHDr{ZCORErrrrÚs,  dAaÿQ'