U Ov fČPć@sÜddlZddlZddlZddlmZddlZddlm Z m Z m Z ddl m Z mZddlmZddlmZddlmZddlmZddlmZdd lmZdd lmZmZmZdd lm Z d d „Z!dd„Z"dd„Z#dd„Z$dd„Z%ej& 'dddddg”ej& 'dddg”dd„ƒƒZ(d d!„Z)d"d#„Z*d$d%„Z+d&d'„Z,ej& 'dddddg”d(d)„ƒZ-d*d+„Z.d,d-„Z/d.d/„Z0d0d1„Z1ej& 'dddddg”d2d3„ƒZ2ej& 'd4ddd5g”d6d7„ƒZ3d8d9„Z4d:d;„Z5dd?„Z7d@dA„Z8dBdC„Z9dS)DéN)Ś load_irisŚ make_blobsŚ make_circles)ŚPCAŚ KernelPCA)ŚNotFittedError)Ś Perceptron)Ś rbf_kernel)Ś GridSearchCV)ŚPipeline)ŚStandardScaler)Śassert_allcloseŚassert_array_almost_equalŚassert_array_equal)Ś_check_psd_eigenvaluesc Csątj d”}| d”}| d”}dd„}dD]®}ddd |fD]œ}t|ƒ }td |||d }| |”}| |” |”} t t  |”t  | ”ƒ|j dks˜t ‚| |”} | j d |j d ksŗt ‚|r<| | ”} | j |j ks.histogram)ŚautoŚdenseŚarpackŚ randomizedŚlinearŚrbfŚpolyr)ŚkernelŚ eigen_solverŚfit_inverse_transforméN)rŚrandomŚ RandomStateŚ random_sampleŚcallablerŚ fit_transformŚfitŚ transformrŚabsŚsizerŚshapeŚinverse_transform) ŚrngŚX_fitŚX_predrr'r&ŚinvŚkpcaŚX_fit_transformedŚX_fit_transformed2ŚX_pred_transformedZX_pred2rrrŚtest_kernel_pcas2    ’ ’  r=c CsDtdddd}d}tjt|d| tj dd””W5QRXdS)zšCheck that kPCA raises an error if the parameters are invalid Tests fitting inverse transform with a precomputed kernel raises a ValueError. é TŚ precomputed)Ś n_componentsr(r&z6Cannot fit_inverse_transform with a precomputed kernel©ŚmatchN)rŚpytestŚraisesŚ ValueErrorr/rr*Śrandn)Z estimatorZerr_msrrrŚ"test_kernel_pca_invalid_parametersFs’rGcCsbtj d”}| dd”}t|d |”}| |”}| ”}d|dd…df<| |”}t||ƒdS)zćCheck robustness to mutations in the original training array Test that after fitting a kPCA model, it stays independent of any mutation of the values of the original data object by relying on an internal copy. rr>)Ś random_stateišN) rr*r+Śrandrr/r0Ścopyr)ŚstateŚXr9Z transformed1ZX_copyZ transformed2rrrŚ$test_kernel_pca_consistent_transformTs    rMc Cstj d”}| dd”}d}|D]j}t d”}tdƒD],}td||d}| |”d||dd…f<q6t|t  |ddd…fd”  dd”ƒq dS) zTest that Kernel PCA produces deterministic output Tests that the same inputs and random state produce the same output. rr>)r!r )érrNr)r@r'rHN) rr*r+rIŚzerosŚrangerr.r ZtileŚreshape)r5rLr'ŚsolverZ transformed_XŚir9rrrŚ$test_kernel_pca_deterministic_outputgs    rTc CsČtj d”}t | d””}t | d””}dD]’}dD]ˆ}td||ddd}| |”}| |”  |”}t t  |”t  |”ƒ|  |”}|j d |j d ks t ‚t t”| |”W5QRXq8q0d S) z¤Test that kPCA works on a sparse data input. Same test as ``test_kernel_pca except inverse_transform`` since it's not implemented for sparse matrices. rrr)rr!r")r#r$r%rF)r&r'r(rHr)N)rr*r+ŚspZ csr_matrixr,rr.r/r0rr1r3rrCrDrr4) r5r6r7r'r&r9r:r;r<rrrŚtest_kernel_pca_sparsexs, ū ’  rVrRrr r!r"Ś n_featuresrr>cCs†tj d”}| d|f”}| d|f”}|dkr4dnd}tt t||d |” |””t t ||dkrj|nd d  |” |””ƒd S) z•Test that kPCA with linear kernel is equivalent to PCA for all solvers. KernelPCA with linear kernel should produce the same output as PCA. rrrr!ér)r'r Śfull)Z svd_solverN) rr*r+r,rr1rr/r0r)rRrWr5r6r7Zn_compsrrrŚtest_kernel_pca_linear_kernels ’ž’žrZcCsftj d”}| d”}| d”}dD]<}dD]2}t||d}| |” |”j}|d|fks,t‚q,q$dS) zĒTest that `n_components` is correctly taken into account for projections For all solvers this tests that the output has the correct shape depending on the selected number of components. rrr©r r!r")r)rr©r@r'rN) rr*r+r,rr/r0r3r)r5r6r7r'Ścr9r3rrrŚtest_kernel_pca_n_components·s    r^cCs„t ddgddgddgg”}tƒ}| |”}|jdks:t‚tdd}| |”}|jdks\t‚tddd}| |”}|jdks€t‚d S) zĄCheck that the ``remove_zero_eig`` parameter works correctly. Tests that the null-space (Zero) eigenvalues are removed when remove_zero_eig=True, whereas they are not by default. ēš?r))rXrr©r@)rXrT)r@Śremove_zero_eigN)rŚarrayrr.r3r)rLr9ZXtrrrŚtest_remove_zero_eigÉs     rcc Cst ddgddgg”}t ”ht dt”tjddDtdddd }| |”  |”}|  |”}t t  |”t  |”ƒW5QRXW5QRXd S) z¹Non-regression test for issue #12141 (PR #12143) This test checks that fit().transform() returns the same result as fit_transform() in case of non-removed zero eigenvalue. r)rŚerrorŚwarn)ŚallrFr )r@rar'N) rrbŚwarningsŚcatch_warningsŚ simplefilterŚRuntimeWarningZerrstaterr/r0r.rr1)r6ŚkŚAŚBrrrŚtest_leave_zero_eigßs   rncCsōtj d”}| d”}| d”}dD]Ź}td|dd |” |”}td|ddd t ||j”” t ||j””}td|ddd  t ||j””}td|ddd t ||j”” t ||j””}t t  |”t  |”ƒt t  |”t  |”ƒq$d S) z?Test that kPCA works with a precomputed kernel, for all solversrrrr[r©r'rHr?)r'r&rHN) rr*r+r,rr/r0ŚdotŚTr.rr1)r5r6r7r'ŚX_kpcaZX_kpca2Z X_kpca_trainZ X_kpca_train2rrrŚtest_kernel_pca_precomputedųsT   ’ž’’ ż ü’’ ž’ ż ü’rscCstddgddgg}td|ddd}| |”dd gd dgg}td|ddd}| |”t|j|jƒt|j|jƒd S) zŖCheck that the kernel centerer works. Tests that a non symmetric precomputed kernel is actually accepted because the kernel centerer does its job correctly. r)rrXé(r?r)r&r'r@rHé i÷’’’N)rr/rZ eigenvectors_Ś eigenvalues_)rRŚKr9ZKcZkpca_crrrŚ)test_kernel_pca_precomputed_non_symmetrics$ ’ ’ rxcCs|tddddd\}}tddd}td |fd td d fgƒ}td t dd”d}t|d|d}| ||”|j dksxt ‚dS)z©Check that kPCA works as expected in a grid search pipeline Test if we can do a grid-search to find parameters to separate circles with a perceptron model. éē333333Ó?ēš™™™™™©?r©Ś n_samplesZfactorZnoiserHr$r©r&r@Ś kernel_pcarr©Zmax_iterē@éž’’’)Zkernel_pca__gammarX©ZcvŚ param_gridr)N) rrr rŚdictrŚaranger r/Ś best_score_r)rLrr9Śpipeliner„Ś grid_searchrrrŚtest_gridsearch_pipeline9s  rŠcCs„tddddd\}}tddd}td |fd td d fgƒ}tt d d ”d}t|d|d}t|dd}|  ||”|j d ks€t ‚dS)zŅCheck that kPCA works as expected in a grid search pipeline (2) Test if we can do a grid-search to find parameters to separate circles with a perceptron model. This test uses a precomputed kernel. ryrzr{rr|r?rr~rrrr€r))ZPerceptron__max_iterrXrƒr)ŚgammaN) rrr rr…rr†r r r/r‡r)rLrr9rˆr„r‰ZX_kernelrrrŚ$test_gridsearch_pipeline_precomputedHs   rŒcCs~tddddd\}}tdd ||” ||”}|dks:t‚td d d d d }| |”}tdd ||” ||”}|dkszt‚dS)aCheck that kPCA projects in a space where nested circles are separable Tests that 2D nested circles become separable with a perceptron when projected in the first 2 kPCA using an RBF kernel, while raw samples are not directly separable in the original space. ryrzr{rr|rr€gš™™™™™é?r$rTr)r&r@r(r‹r_N)rrr/Zscorerrr.)rLrZ train_scorer9rrrrrŚtest_nested_circlesXs ’ rcCs^ddgddgddgg}tdddd }| |”|j ”dks@t‚t |jt|jƒk”sZt‚d S) z}Check that ``_check_psd_eigenvalues`` is correctly called in kPCA Non-regression test for issue #12140 (PR #12145). rr)gwĢ«@g:Œ0āŽyE>rr#rT)r&r@r(N)rr/rvŚminrrrfr©rLr9rrrŚtest_kernel_conditioningts  rcCsddddddddgdddd d d d d gddd d d d ddgdd d dddddgdd d dddddgdd d dddddgdd ddddddgdd ddddddgg}td|dd}tjtdd| |”W5QRXtd|dd}|dkrtjtdd| |”W5QRXn | |”dS)aŅCheck how KernelPCA works with non-PSD kernels depending on n_components Tests for all methods what happens with a non PSD gram matrix (this can happen in an isomap scenario, or with custom kernel functions, or maybe with ill-posed datasets). When ``n_component`` is large enough to capture a negative eigenvalue, an error should be raised. Otherwise, KernelPCA should run without error since the negative eigenvalues are not selected. gģQø…ė@gšæg¤p= ×# @g¤p= ×£@g ×£p= ĄgĀõ(\)ĄgģQø…ėĄg@g×£p= ×óægģQø…ėéægö(\Āõ@gö(\Āõ.@gø…ėQø@g333333&Ąg{®Gį:7Ąg@g333333 ĄgRø…ėQš?gĶĢĢĢĢĢģægq= ×£p@g€#@gö(\Āu5@r?é)r&r'r@z*There are significant negative eigenvaluesrArr"N)rrCrDrEr/)rRrwr9rrrŚtest_precomputed_kernel_not_psd„s(ų ’r’r@rNc CsČd\}}t||dddd\}}|d|…dd…f||d…dd…f}}t|ddd |” |”}t|d dd |” |”}tt |”t |”ƒt|d dd |” |”} tt | ”t |”ƒdS) zGCheck that 'dense' 'arpack' & 'randomized' solvers give similar results)ičédrzr{rr|Nr ror!r")rrr/r0rrr1) r@Zn_trainZn_testrLŚ_r6r7Zref_predZa_predZr_predrrrŚ#test_kernel_pca_solvers_equivalenceøs8’ *’ž’’ž’’ž’r•cCs^tdddd^}}tddddd }| |”}| |”}tj ||”tj |”d ksZt‚d S) zōTest if the reconstruction is a good approximation. Note that in general it is not possible to get an arbitrarily good reconstruction because of kernel centering that does not preserve all the information of the original data. r“rr©r}rWrHrNr$Tgü©ńŅMbP?)r@r&r(Śalphaēš™™™™™¹?N)rrr.r4rZlinalgZnormr)rLr”r9ZX_transZ X_reconstrrrŚ0test_kernel_pca_inverse_transform_reconstructionŻs’  r™c CsHtj d” dd”}tƒ}| |”t t”|  |”W5QRXdS)NérrX) rr*rFrQrr/rCrDrr4rrrrŚ&test_kernel_pca_raise_not_fitted_errorīs   r›cCsjtddddgdddggddd\}}tƒ |”}|| ”8}tƒ}| |”j| | tj””jksft ‚dS)zžTest that the decomposition is similar for 32 and 64 bits data Non regression test for https://github.com/scikit-learn/scikit-learn/issues/18146 érr)r˜)r}ZcentersrHZ cluster_stdN) rr r.rŽrr3ZastyperZfloat32r)rLrr9rrrŚtest_32_64_decomposition_shapeös’   rcCsFtdddd^}}tdd |”}| ”}tdd„tdƒDƒ|ƒd S) z&Check feature names out for KernelPCA.r“rrr–rr`cSsg|]}d|›‘qS)Z kernelpcar)Ś.0rSrrrŚ sz5test_kernel_pca_feature_names_out..N)rrr/Zget_feature_names_outrrP)rLr”r9ŚnamesrrrŚ!test_kernel_pca_feature_names_outsr”cCs¬tj d”}| d”}d|dddœ}d|jd}tfdd i|—Ž |”}tfd|i|—Ž |”}|j|kspt‚|j|ks~t‚|  |  |””}|  |  |””}t ||ƒd S) z]Check that gamma is set correctly when not provided. Non-regression test for #26280 rrrTr$)r@rHr(r&r)r‹N) rr*r+r,r3rr/Zgamma_rr4r0r )r5rLrZexpected_gammaZkpca1Zkpca2ZX1_reconZX2_reconrrrŚ%test_kernel_pca_inverse_correct_gammas  ür¢c CsHt d”tddd\}}tjddtddd |”W5QRXdS) z©Check that KernelPCA works with pandas output when the solver is arpack. Non-regression test for: https://github.com/scikit-learn/scikit-learn/issues/27579 ZpandasT)Zas_frameZ return_X_y)Ztransform_outputrr!r\N)rCZ importorskiprŚsklearnZconfig_contextrr.)rLr”rrrŚtest_kernel_pca_pandas_output,s r¤):rgZnumpyrrCZ scipy.sparseŚsparserUr£Zsklearn.datasetsrrrZsklearn.decompositionrrZsklearn.exceptionsrZsklearn.linear_modelrZsklearn.metrics.pairwiser Zsklearn.model_selectionr Zsklearn.pipeliner Zsklearn.preprocessingr Zsklearn.utils._testingr rrZsklearn.utils.validationrr=rGrMrTrVŚmarkZ parametrizerZr^rcrnrsrxrŠrŒrrr’r•r™r›rr”r¢r¤rrrrŚsT        .% &  3 $