Choosing a measure

Choosing is the hard part. Calling is one line.

There is no “overall quality” of a projection, and any library that offers one is averaging over questions you did not ask. Start from what you are going to claim about the picture.

Start from the claim you want to make

“These points are close together, so they’re similar.” You are claiming the drawn neighbourhoods are real. Use trustworthiness — it penalises exactly the error that would make this claim false, points drawn together that were not.

“There’s no group here, so there wasn’t one.” The mirror claim, and a different measure: continuity penalises structure the projection hides rather than invents.

“This gap is twice as big as that one.” You are reading distances, not neighbourhoods. Use stress or, if you only mean the ordering of the gaps, spearmanRho — the difference matters more than it sounds, and there is a case where they disagree completely.

“This cluster is denser than that one.” Neighbourhood measures will not catch a projection that flattens the contrast. Use densityPreservation.

“These classes are separable.” Careful: most separability measures see only the projection, so a layout that invents clean groups scores well on all of them. Use them, but read the case where they all agree and are all wrong. gabrielClassificationError is the only one here that also looks at the data.

“This is one connected structure / this loop is real.” Topology. topologicalH0 for merge structure, topologicalH1 for loops — the only measure here that notices a circle has been cut open.

“I want one number to compare twenty projections.” aucLogRnx. It integrates R_NX over every neighbourhood size on a log scale, so it does not depend on a k you chose arbitrarily. It is still only the rank-based view.

One number is never enough

Each of the first five rows of the disagreement gallery is a case where a measure scores above 0.9 while the projection is badly wrong in a way another measure catches immediately. The last row is the mirror image: a projection that is fine for the question being asked, scored badly by a measure asking a different one. Pick two or three from different families.

How many do you need?

Two, usually three, from different families — they are cheap once the pass has run, and they fail in different directions:

1. One rank-based: trustworthiness and continuity, or aucLogRnx alone.
2. One distance-based: scaleNormalizedStress, or spearmanRho if only ordering matters.
3. One that matches your specific claim: density, topology, or separability.

What you have determines what you can use

you have	you can use
the projection only	scagnostics — and nothing else here, because with no original data there is nothing to be faithful to
the data and the projection	everything except the label measures
data, projection and class labels	everything

Nine of the ten label measures never look at the data. That is not a flaw — they answer “does this picture look separable”, which is a real question — but it is not the same question as “is this picture honest”, and they are routinely used as though it were.

Running them on the data instead

There is a way to get the second question out of them: compute the same measure on the data with the same labels, and compare. Every separability measure except gabrielClassificationError takes points of any dimension, so nothing stops you pointing one at the high-dimensional side:

import { clusters, silhouette } from "@saehrimnir/sickle";

const inData = silhouette(hd, clusters(hd, labels)).value;
const drawn  = silhouette(ld, clusters(ld, labels)).value;

On the false separation case, where the labels are noise in the data and the layout draws them 5 units apart, that comparison is decisive — the data-side numbers sit exactly at chance while the drawn ones are perfect:

measure		on the data	on the drawing
silhouette	▲	0.000	0.902
distanceConsistency	▲	0.507	1.000
neighborhoodHit	▲	0.488	1.000
classificationError	▼	0.479	0.000

Two things to be careful about. A gap is normal. On an honest PCA projection of well-separated blobs the same comparison gives silhouette 0.755 in 8-D against 0.884 in 2-D — projecting away noise dimensions genuinely tightens clusters, and distances concentrate in high dimensions, so the drawn figure starts with an advantage. It is the size of the gap that carries information, not its sign.

Only the bounded measures survive the trip. calinskiHarabasz, dunnIndex, averageBetweenWithin and hypothesisMargin have no upper limit and their scale depends on the dimensionality and spread of whatever space they were computed in — Calinski–Harabasz reads 1 144 in the data and 4 687 in the drawing for the good projection above. Differences between spaces are not interpretable for those four. Stick to the four in the table, which are bounded and have a known chance level.

This is a useful check, not a replacement for gabrielClassificationError. It compares two summaries and so catches wholesale invented separation; it is much weaker on a few individual points put in the wrong place, where the summaries barely move.

The full matrix

Generated from the library’s own TSDoc, so it cannot drift from the code.

Search 33 of 33

measure	answers	data	labels	range	cost
Neighbourhood
aucLogRnx	One number summarising neighbourhood preservation at every scale.	✓	·	[0, 1] ▲ better	O(N)
continuity	Are the data's neighbours still together in the projection?	✓	·	[0, 1] ▲ better	O(1)
lcmc	Neighbourhood overlap with the chance level subtracted.	✓	·	(-1, 1] ▲ better	O(1)
mrreFalse	Rank error among the neighbours the projection shows.	✓	·	[0, 1] ▲ better	O(k)
mrreMissing	Rank error among the neighbours the data actually has.	✓	·	[0, 1] ▲ better	O(k)
qnx	Fraction of each point's k nearest neighbours that survive the projection.	✓	·	[0, 1] ▲ better	O(1)
rnx	Neighbourhood preservation rescaled so chance is 0 and perfect is 1.	✓	·	[0, 1] in practice ▲ better	O(1)
trustworthiness	Are the neighbours you see in the projection real?	✓	·	[0, 1] ▲ better	O(1)
Distance
nonMetricStress	Stress measured against the best monotone fit rather than the raw distances.	✓	·	[0, 1] ▼ better	O(N² log N)
pearsonR	Linear correlation between the original and projected distances.	✓	·	[-1, 1] ▲ better	O(1)
residualVariance	The share of distance variance the projection fails to explain.	✓	·	[0, 1] ▼ better	O(1)
scaleNormalizedStress	Stress after rescaling the projection to fit as well as it can.	✓	·	[0, 1] ▼ better	O(1)
spearmanRho	Rank correlation between original and projected distances — Shepard goodness.	✓	·	[-1, 1] ▲ better	O(N² log N)
stress	How far the projected distances are from the originals.	✓	·	[0, ∞) ▽ better	O(1)
Embedding cost
curvilinearStress	Stress that forgives tearing but not folding.	✓	·	[0, ∞) ▽ better	O(1)
nerv	Neighbourhood preservation framed as an information-retrieval trade-off.	✓	·	[0, ∞) ▽ better	O(N²·D)
sammonStress	Stress that weights short distances most heavily.	✓	·	[0, ∞) ▽ better	O(1)
Class separability
averageBetweenWithin	Mean distance between classes divided by mean distance within them.	·	✓	[0, ∞) △ better	O(N²·D)
calinskiHarabasz	Ratio of between-class to within-class scatter.	·	✓	[0, ∞) △ better	O(N·D)
classificationError	Share of points whose label is not the majority among their visual neighbours.	·	✓	[0, 1] ▼ better	O(N²·D)
daviesBouldin	Average worst-case overlap between each class and its closest rival.	·	✓	[0, ∞) ▽ better	O(N·D + k²)
distanceConsistency	Fraction of points lying nearest their own class centroid.	·	✓	[0, 1] ▲ better	O(N·k·D)
dunnIndex	Smallest centroid-to-centroid gap divided by the widest class.	·	✓	[0, ∞) △ better	O(N²·D)
gabrielClassificationError	Class disagreements between visually adjacent points, weighted by how close they really are.	✓	✓	[0, ∞) ▽ better	O(N log N)
hypothesisMargin	Average gap between each point's nearest same-class and nearest other-class neighbour.	·	✓	(-∞, ∞) △ better	O(N²·D)
neighborhoodHit	Share of each point's visual neighbours that carry its own label.	·	✓	[0, 1] ▲ better	O(N²·D)
silhouette	How much closer each point is to its own class than to the nearest other class.	·	✓	[-1, 1] ▲ better	O(N²·D)
Structure
densityPreservation	Whether dense regions stayed dense and sparse ones sparse.	✓	·	[-1, 1] ▲ better	O(N)
tripletAccuracy	Share of point triples whose relative ordering survives the projection.	✓	·	[0, 1] ▲ better	O(1)
Topology
topologicalH0	Do the data and the projection merge into connected pieces at the same scales?	✓	·	[0, ∞) ▽ better	O(N²·D)
topologicalH1	Do the projection's loops match the data's?	✓	·	[0, ∞) ▽ better	O(N³)
Cluster reliability
snc	Does the projection show the right groups?	✓	·	both [0, 1] ▲ better	O(N²)
Scagnostics
scagnostics	Nine measures of what the scatterplot looks like.	·	·	each in [0, 1]	O(N log N)

▲ higher is better · ▼ lower is better · △▽ hollow: unbounded, so the number is comparable only between projections of the same data