iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
PCA ranks directions in your data by how much variance they explain. Keep fewer components and the reduced data no longer represents variation along the omitted directions. Those directions are “quiet” in the sense that they account for less variance—not necessarily useless for every prediction or scientific question.
What PCA keeps and discards
Principal component analysis (PCA) finds orthogonal directions through multivariate data. The first direction captures as much variance as possible; each later direction captures as much as possible subject to being orthogonal to the earlier ones. The components are ordered by explained variance, from greatest to least. The scikit-learn PCA API documentation and decomposition guide describe this behavior.
In scikit-learn, fitting PCA learns the directions, and transforming data projects each observation onto them. Setting n_components determines how many directions to retain. With fewer components, the projection represents the data in a lower-dimensional subspace; variation represented only by the omitted directions is absent from that reduced representation.
Recommended Free Tools
Does PCA remove the least useful information?
Not necessarily. PCA orders components by variance, not by their relevance to a particular target or scientific question. A low-variance direction is lower priority for a variance-preserving compression, but that ranking alone does not establish whether the direction matters for a downstream task. Treat a component cutoff as a choice about how much variance to represent—not as proof that everything excluded is noise or universally unimportant.
#1 Best Overall
Choose and assess the cutoff in light of the job the reduced data must do. If the goal is to represent variation compactly, explained variance is directly relevant. If the goal is a prediction or a scientific conclusion, evaluate the result against that specific objective; the variance ranking by itself cannot answer it.
How to choose the number of components
Set a fixed count
A fixed n_components retains the number of directions you specify. This directly controls the reduced dimensionality, while omitting all remaining component directions. The API also documents a variance-threshold option: with the full solver, PCA selects enough components to exceed the requested explained-variance percentage. That threshold is a variance-retention rule, not a guarantee about downstream usefulness.
Rank #2
- This guide is a perfect overview for the topics covered in introductory statistics courses.
Check what the cutoff means
- Decide whether you need a particular reduced dimension or a target level of variance representation.
- Inspect the explained variance associated with the retained components to understand what the cutoff preserves.
- For a task beyond variance compression, judge the transformed data using that task’s own evidence rather than assuming that a high retained-variance share guarantees success.
Scaling changes what PCA sees
Scikit-learn’s PCA centers each input feature before applying singular value decomposition, but does not scale each feature to unit variance. As a result, features with larger numerical scales can contribute more strongly to the variation PCA ranks. If features use different units or scales, decide whether to standardize them before PCA rather than assuming the transformer does it for you. The scikit-learn preprocessing guide explains standardization and related preprocessing choices.
Quick wins for a faster PC:
Scan for outdated or missing drivers - takes under a minuteDriver Scan →Repair Windows errors before they cause bigger problemsFix Now →Whitening is different from dropping components
Whitening changes the scale of retained component outputs so that each has unit variance. Scikit-learn’s PCA API explains that this removes the components’ relative variance scales; it is not the same operation as truncating the component list. The documentation notes that whitening may sometimes help downstream estimators with particular assumptions. Whether it suits a given task depends on that task and estimator.
Quick Recap
Best Value
Rank #3
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
