U Ov fe@s*dZddlZddlmZmZddlZddlmZddl m Z m Z m Z m Z ddlmZddlmZmZmZdd lmZmZmZmZdd lmZd d gZd dZddZddZddZd*ddZ ddZ!ddZ"edgdgdgdgdddd+d d!d"dd#d$dd%ddd&dd' d(d Z#Gd)d d e e e Z$dS),z Python implementation of the fast ICA algorithms. Reference: Tables 8.3 and 8.4 page 196 in the book: Independent Component Analysis, by Hyvarinen et al. N)IntegralReal)linalg) BaseEstimatorClassNamePrefixFeaturesOutMixinTransformerMixin _fit_context)ConvergenceWarning)as_float_array check_arraycheck_random_state)IntervalOptions StrOptionsvalidate_params)check_is_fittedfasticaFastICAcCs,|tj||d|j|d|g8}|S)a Orthonormalize w wrt the first j rows of W. Parameters ---------- w : ndarray of shape (n,) Array to be orthogonalized W : ndarray of shape (p, n) Null space definition j : int < p The no of (from the first) rows of Null space W wrt which w is orthogonalized. Notes ----- Assumes that W is orthogonal w changed in place N)npr multi_dotT)wWjrB/tmp/pip-unpacked-wheel-qu3nn_q2/sklearn/decomposition/_fastica.py_gs_decorrelation s(rcCsTtt||j\}}tj|t|jjdd}tj |dt ||j|gS)z@Symmetric decorrelation i.e. W <- (W * W.T) ^{-1/2} * W N)Za_minZa_max?) reighrdotrZclipfinfodtypeZtinyrsqrt)rsurrr_sym_decorrelation9sr&cCs|jd}tj||f|jd}g}t|D]} || ddf} | t| d} t|D]} |t| j ||\} } || j dd| | }t ||| |t|d}t t || d}|} ||kr`qq`| | d| || ddf<q*|t|fS)zcDeflationary FastICA using fun approx to neg-entropy function Used internally by FastICA. rr"Nraxis)shaperzerosr"rangecopyr#sumr rmeanrabsappendmax)Xtolgfun_argsmax_iterw_init n_componentsrn_iterrrigwtxg_wtxZw1limrrr_ica_defHs$    r@c Cst|}~t|jd}t|D]x}|t|||\} } tt| |j|| ddtjf|} ~ ~ tt t t d| |d} | }| |kr qq t dt ||dfS)zCParallel FastICA. Used internally by FastICA --main loop r(Nzij,ij->iz\FastICA did not converge. Consider increasing tolerance or the maximum number of iterations.)r&floatr+r-rr rnewaxisr3r1Zeinsumwarningswarnr ) r4r5r6r7r8r9rZp_iir=r>ZW1r?rrr_ica_parks  ,rFcCsh|dd}||9}t||}tj|jd|jd}t|D] \}}|d|d||<q>||fS)Nalpharrr'r(r)getrtanhemptyr+r" enumerater0)xr7rGgxg_xr<Zgx_irrr_logcoshs  rOcCs<t|d d}||}d|d|}||jddfS)Nrr(r))rexpr0)rLr7rQrMrNrrr_expsrRcCs|dd|djddfS)NrrPr))r0rLr7rrr_cubesrU array-likeboolean)r4 return_X_meancompute_sources return_n_iterFZprefer_skip_nested_validationparallel unit-variancelogcosh-C6?svdT) algorithmwhitenfunr7r8r5r9 whiten_solver random_staterXrYrZc  Cst||||||||| | d }||j|| d}|jdkrJ|j}|j}nd}d}||j|g}| rl||| r|||j|S)aPerform Fast Independent Component Analysis. The implementation is based on [1]_. Read more in the :ref:`User Guide `. Parameters ---------- X : array-like of shape (n_samples, n_features) Training vector, where `n_samples` is the number of samples and `n_features` is the number of features. n_components : int, default=None Number of components to use. If None is passed, all are used. algorithm : {'parallel', 'deflation'}, default='parallel' Specify which algorithm to use for FastICA. whiten : str or bool, default='unit-variance' Specify the whitening strategy to use. - If 'arbitrary-variance', a whitening with variance arbitrary is used. - If 'unit-variance', the whitening matrix is rescaled to ensure that each recovered source has unit variance. - If False, the data is already considered to be whitened, and no whitening is performed. .. versionchanged:: 1.3 The default value of `whiten` changed to 'unit-variance' in 1.3. fun : {'logcosh', 'exp', 'cube'} or callable, default='logcosh' The functional form of the G function used in the approximation to neg-entropy. Could be either 'logcosh', 'exp', or 'cube'. You can also provide your own function. It should return a tuple containing the value of the function, and of its derivative, in the point. The derivative should be averaged along its last dimension. Example:: def my_g(x): return x ** 3, (3 * x ** 2).mean(axis=-1) fun_args : dict, default=None Arguments to send to the functional form. If empty or None and if fun='logcosh', fun_args will take value {'alpha' : 1.0}. max_iter : int, default=200 Maximum number of iterations to perform. tol : float, default=1e-4 A positive scalar giving the tolerance at which the un-mixing matrix is considered to have converged. w_init : ndarray of shape (n_components, n_components), default=None Initial un-mixing array. If `w_init=None`, then an array of values drawn from a normal distribution is used. whiten_solver : {"eigh", "svd"}, default="svd" The solver to use for whitening. - "svd" is more stable numerically if the problem is degenerate, and often faster when `n_samples <= n_features`. - "eigh" is generally more memory efficient when `n_samples >= n_features`, and can be faster when `n_samples >= 50 * n_features`. .. versionadded:: 1.2 random_state : int, RandomState instance or None, default=None Used to initialize ``w_init`` when not specified, with a normal distribution. Pass an int, for reproducible results across multiple function calls. See :term:`Glossary `. return_X_mean : bool, default=False If True, X_mean is returned too. compute_sources : bool, default=True If False, sources are not computed, but only the rotation matrix. This can save memory when working with big data. Defaults to True. return_n_iter : bool, default=False Whether or not to return the number of iterations. Returns ------- K : ndarray of shape (n_components, n_features) or None If whiten is 'True', K is the pre-whitening matrix that projects data onto the first n_components principal components. If whiten is 'False', K is 'None'. W : ndarray of shape (n_components, n_components) The square matrix that unmixes the data after whitening. The mixing matrix is the pseudo-inverse of matrix ``W K`` if K is not None, else it is the inverse of W. S : ndarray of shape (n_samples, n_components) or None Estimated source matrix. X_mean : ndarray of shape (n_features,) The mean over features. Returned only if return_X_mean is True. n_iter : int If the algorithm is "deflation", n_iter is the maximum number of iterations run across all components. Else they are just the number of iterations taken to converge. This is returned only when return_n_iter is set to `True`. Notes ----- The data matrix X is considered to be a linear combination of non-Gaussian (independent) components i.e. X = AS where columns of S contain the independent components and A is a linear mixing matrix. In short ICA attempts to `un-mix' the data by estimating an un-mixing matrix W where ``S = W K X.`` While FastICA was proposed to estimate as many sources as features, it is possible to estimate less by setting n_components < n_features. It this case K is not a square matrix and the estimated A is the pseudo-inverse of ``W K``. This implementation was originally made for data of shape [n_features, n_samples]. Now the input is transposed before the algorithm is applied. This makes it slightly faster for Fortran-ordered input. References ---------- .. [1] A. Hyvarinen and E. Oja, "Fast Independent Component Analysis", Algorithms and Applications, Neural Networks, 13(4-5), 2000, pp. 411-430. r:rbrcrdr7r8r5r9rerfrY)r]arbitrary-varianceN) rZ_validate_params_fit_transformrc whitening_mean_ _unmixingr2n_iter_)r4r:rbrcrdr7r8r5r9rerfrXrYrZZestSKX_meanZreturned_valuesrrrrs4!     c s"eZdZUdZeedddddgeddhgedd heed hged d d he ge dgeeddddgee ddddgddgeddhgdgd Z e e d<d*dd d ddddddd fdd Zd+ddZeddd,ddZeddd-d d!Zd.d"d#Zd/d$d%Zed&d'Zd(d)ZZS)0raFastICA: a fast algorithm for Independent Component Analysis. The implementation is based on [1]_. Read more in the :ref:`User Guide `. Parameters ---------- n_components : int, default=None Number of components to use. If None is passed, all are used. algorithm : {'parallel', 'deflation'}, default='parallel' Specify which algorithm to use for FastICA. whiten : str or bool, default='unit-variance' Specify the whitening strategy to use. - If 'arbitrary-variance', a whitening with variance arbitrary is used. - If 'unit-variance', the whitening matrix is rescaled to ensure that each recovered source has unit variance. - If False, the data is already considered to be whitened, and no whitening is performed. .. versionchanged:: 1.3 The default value of `whiten` changed to 'unit-variance' in 1.3. fun : {'logcosh', 'exp', 'cube'} or callable, default='logcosh' The functional form of the G function used in the approximation to neg-entropy. Could be either 'logcosh', 'exp', or 'cube'. You can also provide your own function. It should return a tuple containing the value of the function, and of its derivative, in the point. The derivative should be averaged along its last dimension. Example:: def my_g(x): return x ** 3, (3 * x ** 2).mean(axis=-1) fun_args : dict, default=None Arguments to send to the functional form. If empty or None and if fun='logcosh', fun_args will take value {'alpha' : 1.0}. max_iter : int, default=200 Maximum number of iterations during fit. tol : float, default=1e-4 A positive scalar giving the tolerance at which the un-mixing matrix is considered to have converged. w_init : array-like of shape (n_components, n_components), default=None Initial un-mixing array. If `w_init=None`, then an array of values drawn from a normal distribution is used. whiten_solver : {"eigh", "svd"}, default="svd" The solver to use for whitening. - "svd" is more stable numerically if the problem is degenerate, and often faster when `n_samples <= n_features`. - "eigh" is generally more memory efficient when `n_samples >= n_features`, and can be faster when `n_samples >= 50 * n_features`. .. versionadded:: 1.2 random_state : int, RandomState instance or None, default=None Used to initialize ``w_init`` when not specified, with a normal distribution. Pass an int, for reproducible results across multiple function calls. See :term:`Glossary `. Attributes ---------- components_ : ndarray of shape (n_components, n_features) The linear operator to apply to the data to get the independent sources. This is equal to the unmixing matrix when ``whiten`` is False, and equal to ``np.dot(unmixing_matrix, self.whitening_)`` when ``whiten`` is True. mixing_ : ndarray of shape (n_features, n_components) The pseudo-inverse of ``components_``. It is the linear operator that maps independent sources to the data. mean_ : ndarray of shape(n_features,) The mean over features. Only set if `self.whiten` is True. n_features_in_ : int Number of features seen during :term:`fit`. .. versionadded:: 0.24 feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0 n_iter_ : int If the algorithm is "deflation", n_iter is the maximum number of iterations run across all components. Else they are just the number of iterations taken to converge. whitening_ : ndarray of shape (n_components, n_features) Only set if whiten is 'True'. This is the pre-whitening matrix that projects data onto the first `n_components` principal components. See Also -------- PCA : Principal component analysis (PCA). IncrementalPCA : Incremental principal components analysis (IPCA). KernelPCA : Kernel Principal component analysis (KPCA). MiniBatchSparsePCA : Mini-batch Sparse Principal Components Analysis. SparsePCA : Sparse Principal Components Analysis (SparsePCA). References ---------- .. [1] A. Hyvarinen and E. Oja, Independent Component Analysis: Algorithms and Applications, Neural Networks, 13(4-5), 2000, pp. 411-430. Examples -------- >>> from sklearn.datasets import load_digits >>> from sklearn.decomposition import FastICA >>> X, _ = load_digits(return_X_y=True) >>> transformer = FastICA(n_components=7, ... random_state=0, ... whiten='unit-variance') >>> X_transformed = transformer.fit_transform(X) >>> X_transformed.shape (1797, 7) r(Nleft)closedr\ deflationrir]Fr^rQcubegrVrrarfrg_parameter_constraintsr_r`) rbrcrdr7r8r5r9rerfc sJt||_||_||_||_||_||_||_||_ | |_ | |_ dSN) super__init__r:rbrcrdr7r8r5r9rerf) selfr:rbrcrdr7r8r5r9rerf __class__rrrys zFastICA.__init__csj|jtjtjgddj}jdkr,inj}tj}| dd}d|kr\dksfnt dj dkrvt }n6j d krt }n&j d krt}ntj rfd d }|j\}} j} js| dk rd} td | dkrt| |} | t| |krt| |} td| jr:|jdd} || ddtjf8}jdkrt||\} } t| ddd}t| jj}| |k}t|rtd|| |<tj | | d| || dd|f} } n(jdkrtj!|ddddd\} } | t"| d9} | | jd| }~ ~ t||}|t | 9}n t#|dd}j$}|dkrttj%|j&| | fd|jd}n.t%|}|j| | fkrt dd| | fij'||j(|d}j)dkrt*|f|\}}nj)dkrt+|f|\}}~|_,|r,jrtj-|||gj}nt||j}nd}jrjd kr|s^tj-|||gj}tj.|dd!d"}||}||j}t||_/| _0|_1n|_/tj2j/dd#_3|_4|S)$adFit the model. Parameters ---------- X : array-like of shape (n_samples, n_features) Training data, where `n_samples` is the number of samples and `n_features` is the number of features. compute_sources : bool, default=False If False, sources are not computes but only the rotation matrix. This can save memory when working with big data. Defaults to False. Returns ------- S : ndarray of shape (n_samples, n_components) or None Sources matrix. `None` if `compute_sources` is `False`. r)r.r"Zensure_min_samplesNrGrr(zalpha must be in [1,2]r^rQrucsj|f|Srw)rdrTrzrrr69sz!FastICA._fit_transform..gz(Ignoring n_components with whiten=False.z/n_components is too large: it will be set to %srPr)rzfThere are some small singular values, using whiten_solver = 'svd' might lead to more accurate results.)outraF)Z full_matrices check_finiter)r.)sizer'z/w_init has invalid shape -- should be %(shape)sr+)r5r6r7r8r9r\rtr]T)r*Zkeepdims)r)5_validate_datarcrfloat64float32rr7r rfrH ValueErrorrdrOrRrUcallabler+r:rCrDminr0rBrerrr Zargsortr!r"epsanyr#rasignr r9Zasarraynormalr5r8rbrFr@rnrZstd components_rlrkZpinvmixing_rm)rzr4rYZXTr7rfrGr6Z n_featuresZ n_samplesr:rqdr%Z sort_indicesrZdegenerate_idxrpZX1r9kwargsrr;roZS_stdrr}rrjs                         zFastICA._fit_transformTr[cCs|j|ddS)a5Fit the model and recover the sources from X. Parameters ---------- X : array-like of shape (n_samples, n_features) Training data, where `n_samples` is the number of samples and `n_features` is the number of features. y : Ignored Not used, present for API consistency by convention. Returns ------- X_new : ndarray of shape (n_samples, n_components) Estimated sources obtained by transforming the data with the estimated unmixing matrix. Trhrjrzr4yrrr fit_transformszFastICA.fit_transformcCs|j|dd|S)aFit the model to X. Parameters ---------- X : array-like of shape (n_samples, n_features) Training data, where `n_samples` is the number of samples and `n_features` is the number of features. y : Ignored Not used, present for API consistency by convention. Returns ------- self : object Returns the instance itself. Frhrrrrrfitsz FastICA.fitcCsHt||j||o|jtjtjgdd}|jr8||j8}t||jj S)a_Recover the sources from X (apply the unmixing matrix). Parameters ---------- X : array-like of shape (n_samples, n_features) Data to transform, where `n_samples` is the number of samples and `n_features` is the number of features. copy : bool, default=True If False, data passed to fit can be overwritten. Defaults to True. Returns ------- X_new : ndarray of shape (n_samples, n_components) Estimated sources obtained by transforming the data with the estimated unmixing matrix. F)r.r"reset) rrrcrrrrlr rrrzr4r.rrr transforms  zFastICA.transformcCsHt|t||o|jtjtjgd}t||jj}|jrD||j 7}|S)a1Transform the sources back to the mixed data (apply mixing matrix). Parameters ---------- X : array-like of shape (n_samples, n_components) Sources, where `n_samples` is the number of samples and `n_components` is the number of components. copy : bool, default=True If False, data passed to fit are overwritten. Defaults to True. Returns ------- X_new : ndarray of shape (n_samples, n_features) Reconstructed data obtained with the mixing matrix. )r.r") rr rcrrrr rrrlrrrrinverse_transforms  zFastICA.inverse_transformcCs |jjdS)z&Number of transformed output features.r)rr+r}rrr_n_features_outszFastICA._n_features_outcCsdtjtjgiS)NZpreserves_dtype)rrrr}rrr _more_tags szFastICA._more_tags)N)F)N)N)T)T)__name__ __module__ __qualname____doc__rrrrboolrdictrrv__annotations__ryrjr rrrrpropertyrr __classcell__rrr{rrcsL            )N)N)%rrCZnumbersrrZnumpyrZscipyrbaserrrr exceptionsr utilsr r r Zutils._param_validationrrrrZutils.validationr__all__rr&r@rFrOrRrUrrrrrrsT    #"  7