Sample Distance and Dimensionality Reduction

Sample distance is a single number that summarizes how similar two observations with multiple numerical features are. Dimensionality reduction converts observations with too many features to plot directly into two or three new coordinates so that you can inspect their overall structure.

PCA (principal component analysis) and UMAP (uniform manifold approximation and projection) are common dimensionality-reduction methods. Both produce low-dimensional plots, but they preserve different relationships, so you should not interpret them as the same kind of map.

One row is a vector with multiple features

Suppose server-run records accumulate in a DataFrame like this:

run	latency_ms	cpu_pct	memory_mb	error_rate
`run_A`	120	45	820	0.01
`run_B`	125	47	800	0.01
`run_C`	850	92	1,900	0.18

Each row is a vector with four numbers. run_A and run_B are similar in most features and will probably be close together, while run_C differs across several features and will probably be far away.

Four features make a four-dimensional space; 10,000 features make a 10,000-dimensional space. A screen can show only two or three axes, but you can still calculate distances and summarize structure in a high-dimensional space.

Check the scale before the distance formula

Euclidean distance squares and adds the differences between corresponding features. Features with large units and numerical ranges can dominate that distance.

In the table above, memory_mb is measured in the hundreds, while error_rate falls between 0 and 1. Calculated as-is, differences in memory carry far more weight than differences in error rate. First decide whether that reflects the intended importance of the features or merely their different units.

The solution depends on the question.

If only the units differ and the features should have equal weight, consider standardization.
If gaps between large values are too wide, consider a logarithm or a transformation such as VST.
If there are many nearly constant or noisy features, select only the features you need.
If direction or proportion matters, consider a metric such as cosine distance instead of Euclidean distance.

Changing the preprocessing or distance formula also changes what “close” means. Do not look only at the distance value and assume the metric matched the analytical goal.

PCA preserves directions with large overall variation

PCA chooses the linear direction along which observations spread out the most as PC1, then chooses the perpendicular direction with the next-largest spread as PC2. It can calculate what percentage of the original data’s overall variation each axis explains.

Points close together in PCA are similar along the displayed principal-component axes. If PC1 and PC2 capture only part of the total variation, two points that overlap on the screen may still differ along PC3 and later axes.

UMAP preserves nearby-neighbour relationships

UMAP is a nonlinear method that places high-dimensional neighbours close together in low dimensions as well. Curved structures and local clusters may appear more clearly than they do in PCA.

The overall orientation of a UMAP plot, empty space between clusters, and absolute distances have no direct units. Changing the number of neighbours, minimum distance, random seed, or preprocessing can rotate the map or alter the spacing between clusters.

What you can safely say from the map

Nearby points are similar under the chosen features, preprocessing, and distance criterion.
Far-apart points differ in their overall pattern, but the cause must be investigated separately.
Even if colour-coded groups separate, dimensionality reduction has not proved the cause.
A point that looks like an outlier may be an error or a genuinely rare observation.
Do not compare coordinates directly between maps made with different preprocessing or settings.

Example: applying this to an expression matrix

In an expression matrix, each row is a tissue sample and each column is a gene’s expression value. The distance between samples depends on which genes are used and how the values are transformed. If public and new data form separate groups, check batch effects such as experimental equipment or processing date before attributing the split to disease.

Relationship to machine learning

PCA and UMAP are classified as unsupervised-learning tools because they find structure without answer labels. They are not a step for calculating predictive-model accuracy. They are closer to tools for exploring data structure and quality problems and for generating follow-up analytical questions.