dimred

Dimensionality reduction for wlearn: PCA, t-SNE, UMAP and TriMap for Python, Node and browsers. A standalone C backend is the default; optional Polygrad execution supports CPU, CUDA, WebAssembly and WebGPU. Python and JavaScript share the C algorithms.

Package	Install	Documentation
npm @wlearn/dimred	npm install @wlearn/dimred	js/README.md
PyPI wlearn-dimred	pip install wlearn-dimred	py/README.md

Models saved in one language load in the other.

Changes in 0.2.0

• Standalone C is now the default; Polygrad is optional. To keep the previous execution path, select backend: 'polygrad' (Python: backend='polygrad').
• C PCA uses a direct double-precision eigensolver; nonlinear methods no longer depend on an iterative Polygrad PCA initialization converging.
• C t-SNE defaults to Barnes-Hut; C UMAP uses sequential scheduled updates and learning rate 1. Layouts can differ from 0.1.0 and the Polygrad backend.
• Native and Wasm C builds share portable math routines, with long-fit byte comparisons, numerical reference tests and layout-quality checks.
• Fitted-model files remain cross-language. Optional Polygrad PCA also gains convergence fixes for close-variance inputs.

How it works

Choose backend: 'c' | 'polygrad'. The default c backend runs the standalone code in src/base/: a native library in Python, packaged WebAssembly in Node and browsers. It neither imports nor requires Polygrad. All JS fits stay async.

The optional src/polygrad/ implementations build tensor graphs through the public C API. They use the caller's runtime, not a second embedded Polygrad. Passing polygrad options or a runtime selects this backend unless backend is explicitly set. Install Polygrad separately for acceleration. Backend choice can change numerical algorithms; it is not just a device switch.

Algorithm notes

PCA

The standalone backend uses double-precision centered/scaled covariance or the smaller Gram matrix, Householder tridiagonalization and implicit QL. It accepts up to min(features, samples - 1) components. It has no graph compilation or iterative-subspace cutoff. It is covariance-based, not direct SVD; ill-conditioned small components remain subject to conditioning limits. Inputs and portable fitted arrays remain float32. maxIter and tolerance configure the Polygrad PCA solver, not the standalone direct solver.

Polygrad PCA supports up to 16 components. Up to 64 features it diagonalizes the full covariance with Jacobi rotations. Larger inputs start with reorthogonalized subspace iteration; up to 1,024 features, strict PCA reserves part of the same iteration budget for a full-covariance fallback if needed. Wider inputs keep the bounded-width iterative solver. All tensor computation stays in C-authored Polygrad graphs. This is float32 covariance-based PCA, not a full-precision SVD. Component signs and repeated-eigenvalue bases are not unique. PCA checks the residual against the original centered data and raises on nonconvergence. Nonlinear methods may use a finite approximate projection without requiring the full fallback. The optional backend requires Polygrad 0.7.1 or newer within 0.7.x. Its X86 launcher supports the wider PCA kernels that failed with Polygrad 0.7.0.

t-SNE

• barnes_hut: default on C, sparse affinities and quadtree repulsion with opening angle 0.5. Not supported by the Polygrad backend.
• exact: default on Polygrad; dense affinities, 4 to 2,000 points.
• sparse: affinities from the k nearest neighbours (k defaults to three times the perplexity), no fixed 2,000-point limit. Repulsion evaluates every pair in blocks of blockSize rows, so time stays quadratic; memory for repulsion is N x blockSize instead of N x N on Polygrad; the C path accumulates pairs directly. sparse never silently selects Barnes-Hut. No FFT implementation.
• Defaults: 500 iterations, early exaggeration 12 until iteration 250, momentum 0.5 then 0.8, learning rate 50.

UMAP

The calibration and pair forces follow umap-learn 0.5.7. The optimizers differ:

• nNeighbors excludes the point itself (umap-learn's n_neighbors is one more). Euclidean metric, local connectivity 1, fuzzy union.
• C uses sequential scheduled fuzzy edges and negative sampling, learning rate 1 by default. It follows the reference-style schedule, not identical reference coordinates: neighbor search, PCA initialization and RNG differ.
• Polygrad updates all weighted edges from the same layout snapshot, with five sampled negatives per edge, instead of umap-learn's sequential per-edge SGD. Learning rates are therefore not interchangeable.
• a and b correspond to min_dist = 0.1, spread = 1. Initialization uses PCA coordinates instead of a spectral embedding.
• C sampling uses xorshift32; Polygrad uses integer hashing of seed and step. Native and Wasm C builds share math for reproducible seeded fits. Polygrad results are not promised bit-identical across devices or with C.

Not implemented: transforming new points, densMAP, supervised UMAP, other metrics.

TriMap

The objective, momentum and delta-bar-delta gains follow the original implementation. Both backends use the same full-batch objective and PCA scaling. C uses direct gradient accumulation; Polygrad uses autograd. Neighbors and random sampling differ from Annoy/NumPy and between backends. Triplets and final coordinates differ from the reference implementation.

Neighbour search

C uses a balanced KD-tree for exact search. Above 2,000 points and 16 features, it uses bounded tree queries followed by neighbor-of-neighbor refinement; this is approximate and can change downstream layouts. Polygrad retains exact blocked search. k is at most 64 in the standalone neighbor API; the low-level Polygrad sparse t-SNE function also accepts precomputed neighbors with larger k. C sparse/Barnes-Hut t-SNE uses at most 64 neighbours and requires perplexity no greater than min(64, samples - 1).

Development

The base backend shares a pinned MIT-licensed musl subset for exp/log/log2/pow between native and Wasm. Builds disable fast-math and floating-point contraction; callers must retain round-to-nearest and gradual underflow. Long seeded fits and math vectors are compared byte-for-byte (excluding NaN payloads). This does not promise equality with the Polygrad backend or other library versions. Upstream hashes and adaptations are recorded in src/base/math/origin.json; redistribution notices are in NOTICE.

make base builds only src/base/, without Polygrad headers or libraries. make test-base PYTHON=/path/to/python runs sanitizer and independent numerical checks. Building optional graph authors additionally requires a Polygrad source tree (which need not be built), Clang and Python 3.10+ to generate bindings, a C compiler and Node headers for native modules, and Emscripten for WebAssembly. Set POLYGRAD_PATH if Polygrad is not at ../../polygrad/polygrad.

POLYGRAD_PATH selects build headers and the binding generator, not execution. Tests and demos resolve the installed Python/npm Polygrad packages by default. To test a locally built checkout explicitly, set POLYGRAD_RUNTIME=/path/to/polygrad. Python then uses that checkout's py/ and build/libpolygrad.so; JavaScript uses its js/ package. Browser tools resolve from this project or WLEARN_PATH. Workspace npm links are respected; unlink them or use isolated package checks when verifying registry installations.

make all POLYGRAD_PATH=/path/to/polygrad
make package          # stage the npm package and Python sdist
make test-api         # estimator API and Python <-> JS model files
make test-python PYTHON=/path/to/python
make test-node PYTHON=/path/to/python
make test-browser PYTHON=/path/to/python

Tests compare against reference implementations, so the Python environment needs NumPy, SciPy, scikit-learn and umap-learn 0.5.7. These are test dependencies only.

make demo-c-api serves an interactive demo on http://127.0.0.1:8081 (set PORT to change it). See examples/c_api/README.md. Benchmarks against npm packages are described in bench/README.md.

Source layout:

src/dimred.h         standalone public API and shared types
src/base/           plain C algorithms (backend: 'c')
src/polygrad/       optional tensor authors (backend: 'polygrad')
js/, py/            lifecycle, memory marshalling and artifact bindings

Only src/polygrad/ includes Polygrad headers. It may use base algorithms; the base code must never depend on Polygrad. Graph entry points use dr_polygrad_*_build; standalone entries use dr_pca_fit, dr_umap_fit, etc.

Polygrad's former X86 kernel-argument limit is fixed in 0.7.1, including the 31-argument neighbour-search kernel used by TriMap.

License

Apache-2.0. See NOTICE.