Performance vs scikit-learn

The numbers below are measured, not estimated. The same deterministic synthetic dataset (xorshift64, seed 42, values in [-100, 100)) is fed to both libraries, and the median fit_transform time over 15 runs (after one warmup) is reported.

Test setup: Apple M5 Pro (18 cores, arm64), Rust 1.96.0 (release), Python 3.9.6, scikit-learn 1.6.1, numpy 2.0.2, scipy 1.13.1. Times are in milliseconds. The Ratio column is sklearn_ms / datarust_ms — values > 1 mean datarust is faster.

Benchmark table

Feature flags make a difference

matrixmultiply. Enabling this feature dispatches covariance and matmuls to a tuned pure-Rust GEMM (no system BLAS). On 50 000 × 200, PCA drops from 206 ms → 111 ms (1.9× faster), and LinearRegression fit+predict from 263 ms → 91 ms (2.9× faster).

rayon. Parallel column/row processing. RobustScaler at 50 000 × 200 drops from 68.7 ms (default) to 14.7 ms (4.7× faster with rayon).

Scalar vs matrixmultiply kernels

The table below isolates what the optional matrixmultiply feature buys inside the numeric hot paths — medians from the criterion suite (cargo bench -p datarust --bench benchmarks -- --warm-up-time 0.5 --measurement-time 1.5 --sample-size 10), single-threaded release build on the same Apple M5 Pro hardware as above. +GEMM is the same build with features = ["matrixmultiply"]; Speedup is scalar / +GEMM. Each time carries its own unit (ms / µs / ns).

The GEMM pays off most on matmul-heavy kernels — Matrix::matmul 1.8–2.8×, LinearRegression fit 2.6–2.7×, TruncatedSVD ~2.0× — while predict paths are memory-bound matvecs and barely move (~1.0×). Pearson no longer needs GEMM for wide tables (the lower-triangle scalar covariance puts the kernel at 6.0 ms for 10 000 × 100); PCA barely moves at these small sizes and shines on larger matrices.

Where datarust wins

• Mixed numeric + categorical composition. ColumnTransformer is 153–380× faster than scikit-learn’s on large inputs. This is the headline result — it reflects the cost of sklearn’s per-column Python dispatch, dtype coercion, and object-array marshalling.
• String / categorical encoding. OneHotEncoder is ~2.9–4.0× faster because datarust operates on a native StrMatrix directly — no Python object-array overhead, no GIL.
• Numeric scalers with rayon. StandardScaler/RobustScaler/Pipeline beat sklearn by 2.1–13× at 50 000 × 200.
• Small data and startup latency. At 1 000 × 10, datarust is faster on every workload — up to 8.9× on StandardScaler and 153× on ColumnTransformer. No Python interpreter to spin up, no numpy import cost.

Where scikit-learn still wins

• PCA on tall-and-wide data (without the matrixmultiply feature). sklearn calls into LAPACK’s full SVD via shared-library BLAS; datarust uses a from-scratch Jacobi sweep. With matrixmultiply the gap narrows from ~17× to ~9×, and PCASolver::Randomized closes it further for low-rank inputs.

Criterion microbenchmark suite

Alongside the sklearn comparison, the criterion suite (cargo bench -p datarust --bench benchmarks and cargo bench -p datarust-profile --bench benchmarks) covers the whole API surface. The medians below are from the default (zero-dependency) build with --warm-up-time 0.5 --measurement-time 1.5 --sample-size 10 on the same Apple M5 Pro, and document the operations added most recently. Each time carries its own unit.

Feature-build comparisons: Criterion identifies a benchmark by group and ID, not Cargo features. Use a distinct CARGO_TARGET_DIR for the default and matrixmultiply runs; otherwise its change output compares unlike builds and falsely reports the feature switch as a regression.

CARGO_TARGET_DIR=target/criterion-default cargo bench -p datarust --bench benchmarks
CARGO_TARGET_DIR=target/criterion-matrixmultiply cargo bench -p datarust \
  --bench benchmarks --features matrixmultiply

datarust

The stats_nested group mirrors the flat-storage kernels over Vec<Vec<f64>> inputs; mode_column shares mode’s sort-then-scan, so a 100 000 × 20 mostly-distinct table modes in 27.7 ms. matrix_ops/from_flat (49 µs at 10 000 × 50) shows why flat construction is the fast path over the nested from_nested (~289 µs for the same shape).

The next pass added eight groups covering the production-time paths that fit_transform-only benchmarks hid — transform/inverse_transform on fitted transformers, predict_proba, fitted-encoder transforms, sparse one-hot, the remaining classification metrics, the splitters, and string-column gathering:

Three rows are the current bottlenecks. QuantileTransformer.transform is still the per-value outlier — ~7.3 ns per value (14.6 ms at 100 000 × 20, down from 23.4 ms) against ~0.85 ns for the linear scalers: each value maps into one of 512 pre-partitioned value spans over the 1 000-point reference and interpolates inside that span’s handful of entries — O(1) plus a few compares instead of a full log₂(1000) binary search, with bit-identical output. The remaining cost is the per-value interpolation itself (the nested-Vec transpose round-trips are gone; the transform now streams the flat buffer row-major). StrMatrix::column (clone) is ~8× slower than the borrowing column_refs (72.0 ms vs 8.12 ms at 100 000 × 20) — all four categorical encoders used to pay this clone tax in their fit paths, and switching them to column_refs made fit_transform 60–83% faster (ordinal/frequency fit at 10 000 × 20: ~21 ms → ~3.6–3.9 ms, with transform unchanged). StratifiedKFold.split used to rebuild each fold’s train set with a per-fold HashSet scan of every sample; a reusable boolean mask (mark → scan → reset) cut it from 6.31 ms to 1.86 ms at 100 000 rows (−71%), leaving the stratification setup itself as the remaining cost. onehot_sparse transform_sparse was 2× slower than the dense transform because it round-tripped through per-row triplet Vecs and SparseMatrix::from_triplets’ per-row re-sort; it now builds the CSR arrays directly — 18.6 ms vs 14.3 ms dense at 50 000 × 20 (down from 27.6 ms), and fit_transform_sparse dropped from 78.3 ms to 32.9 ms.

datarust-profile

The high-cardinality groups pin down the worst cases of wide-table profiling: near-unique identifier columns (10 000 × 20) take ~83 ms end-to-end — dominated by duplicate detection and per-column categorical tallies — and Cramér’s V over 1 000-level columns costs ~6 ms per 10-column table. report_json scales with the column count, not the row count (the profile is column-summarized), so 10 000 × 20 and 100 000 × 20 serialize in ~36–39 µs.

Reproduce the benchmarks

The harness lives in examples/bench_compare_rust.rs (Rust side) and benches/compare_sklearn.py (Python side). Run on your own hardware:

# Rust (all feature combos)
cargo run --release --features matrixmultiply --example bench_compare_rust 15

# Python (requires numpy, scikit-learn)
python3 benches/compare_sklearn.py 15