Changelog

All notable changes to datarust are documented in the project’s CHANGELOG.md on GitHub.

This page provides a summary. For the full, detailed changelog (including internal refactors and performance tables), see the canonical source.

Unreleased

• PowerTransformer: the lambda-search likelihood now uses a precomputed jacobian term and a single fused Welford mean+variance pass — fit_transform 1 000 × 20 drops from 36.4 ms to 20.3 ms (1.8×).
• QuantileTransformer: column-major transform with a single trailing transpose — 10 000 × 100 goes from 27.7 ms to 23.1 ms.
• Quantile statistics: introselect replaces the full sort in column_quantiles_many — 100 000 × 100 goes from 144.7 ms to 86.0 ms (1.7×).
• KnnImputer: precomputed observed features, borrow-based neighbor search (no per-row reference-matrix copy), and partial neighbor selection — fit_transform 1 000 × 20 drops from 72.9 ms to 18.1 ms (4.0×).
• KMeans: nearest-centroid assignment via precomputed squared norms (FMA dot product) — fit_predict 5 000 × 20 × 10 is ~7% faster.
• LogisticRegression: the IRLS weighted Gram matrix and multinomial Hessian are accumulated in place (rank-1 sums, lower triangle only) instead of via a materialised weighted design matrix and general matmul — fit 50 000 × 100 drops from 3.86 s to 1.56 s (2.5×).
• stats::mode: sort-then-scan replaces hash-map counting (one heap entry per distinct value) — SimpleImputer MostFrequent 50 000 × 20 drops from 32.9 ms to 17.0 ms (1.9×).
• Wide-table Pearson: the new stats::correlation_matrix_flat streams the mean/center/covariance passes over contiguous memory, and the scalar covariance accumulates only the lower triangle — correlation_matrix_flat 10 000 × 100 drops from 38.7 ms to 6.7 ms (5.8×); datarust-profile’s wide profile_matrix 10 000 × 100 drops from ~63 ms to ~31 ms.
• Performance docs: the Performance page and README now include a “Scalar vs matrixmultiply kernels” table with stable criterion medians (--warm-up-time 0.5 --measurement-time 1.5 --sample-size 10) isolating the GEMM feature’s effect on matmul-heavy kernels.
• datarust-profile: hash-based row deduplication, borrow-based categorical access (T: AsRef<str>), allocation-free missing detection, and integer-coded Cramér’s V — profile_str_matrix 100 000 × 20 drops from 269 ms to 132 ms.
• Encoder fit paths: the four categorical encoders now gather columns through the borrowing StrMatrix::column_refs instead of cloning every cell — fit_transform 10 000 × 20 drops from ~20.7 ms to 3.60 ms (ordinal, 5.7×).
• QuantileTransformer transform: streams the flat buffer row-major (no nested-Vec transpose round-trip) — 100 000 × 20 drops from 28.3 ms to 22.9 ms; the reference lookup is then pre-partitioned into 512 uniform value spans so each value skips the full binary search — 23.4 ms to 14.6 ms (1.6×), bit-identical output.
• OneHotEncoder sparse transform: builds the CSR arrays directly instead of per-row triplets plus a re-sort — transform_sparse 50 000 × 20 drops from 27.6 ms to 18.3 ms; fit_transform_sparse from 78.3 ms to 33.2 ms.
• StratifiedKFold: each fold’s training complement uses a reusable boolean mask instead of a per-fold HashSet scan — split 100 000 drops from 6.31 ms to 1.82 ms (3.5×).
• Fresh sklearn comparison: the “datarust vs scikit-learn” table was re-run end-to-end (scikit-learn 1.6.1 / numpy 2.0.2 / scipy 1.13.1) — OneHotEncoder 50 000 × 20 default 89 → 54.7 ms, PCA 50 000 × 200 838 → 206 ms, and the LinearRegression column is now filled in (263 ms default / 91 ms with matrixmultiply vs sklearn 118 ms).

For the full details and performance tables, see the canonical changelog.

0.6.5

• Restored enforceable Rust 1.70 compatibility by pinning MSRV-compatible Rayon and serde_json dependency lines.
• Hardened deserialized categorical encoder state against mismatched category lists, index maps, and output widths.
• Rejected fractional ordinal codes during inverse transformation.
• Completed fitted-state checks for KNN/Simple imputers, MaxAbsScaler, and QuantileTransformer.
• Aligned PCA solver and component-count documentation with the implemented behavior.

0.6.4

• Silhouette scoring compacts arbitrary cluster IDs, handles singleton clusters correctly, and rejects invalid one-cluster-per-sample input.
• Numerical estimators, decompositions, scalers, selectors, and regression metrics reject non-finite observations; imputers retain NaN missing-value support while rejecting infinity.
• Non-finite scaler, selector, encoder, and SVD configuration values fail before fitting.
• Inconsistent fitted state loaded from JSON returns a recoverable error instead of panicking during transform or prediction.
• CI now builds and validates the docs/blog site and denies rustdoc warnings explicitly.

0.6.3

• Ridge, Lasso, logistic regression, and KMeans now reject non-finite or otherwise invalid solver hyperparameters before optimization.
• Params::set_params validation no longer mutates KMeans or logistic regression when a candidate is invalid.
• CI now executes the intended OS × feature matrix, including datasets and an all-features build, and verifies the crates.io package.
• The published crate excludes the blog, book, website, npm, and Cloudflare deployment sources, reducing the compressed archive from about 3.0 MiB to 289 KiB.

0.6.2

• Safe dense access now checks bounds in release builds, while dense allocations reject dimension overflow.
• Raw and deserialized CSR matrices validate pointer ranges and sorted column indices; duplicate triplets are summed.
• Train/test and cross-validation reject undersized or mismatched inputs, and fractional test sizes now round up consistently.
• StratifiedKFold supports validated, non-contiguous multiclass labels.
• Binary log loss, ROC-AUC, and average precision validate labels and numeric inputs. New *_with_positive_label functions support label spaces such as {2, 5}, and average-precision ties are threshold-grouped.

0.6.1

• Classification labels are now compacted safely, so metrics and LogisticRegression support non-contiguous labels such as {2, 5, 9} while preserving original predictions and probability-column mappings.
• Precision, recall, and F1 now expose explicit binary, macro, weighted, and micro averaging, configurable zero-division handling, and per-class reports.
• The optional datasets feature adds embedded Iris, Breast Cancer, Wine, and Diabetes datasets with no runtime file or network dependency.

0.5.0

• Estimator hierarchy — Estimator, Predictor, Classifier, and PredictProba now define common supervised flows. Regressor is reserved for regression semantics.
• SupervisedPipeline — Pipeline::with_estimator combines preprocessing, target-aware feature selection, and a final predictor in a cloneable, serde-serializable pipeline.
• BREAKING: LogisticRegression classification semantics — predict returns hard labels; predict_proba returns a two-column probability matrix; predict_positive_proba exposes the positive-class vector.
• BREAKING: Predictor imports — import Predictor to call shared fit/predict methods on supervised models.
• BREAKING: Custom trait implementors — custom transformers and encoders must implement Estimator; custom regressors must implement Estimator and Predictor before Regressor.

0.4.0

Added

• model_selection module — train_test_split, KFold, StratifiedKFold, cross_val_score. Shared deterministic xorshift64 PRNG, now used by both model_selection and decomposition::randomized_svd.
• LogisticRegression — binary classification via IRLS (Cholesky/SVD). The crate’s first classifier.
• metrics::classification — accuracy_score, precision_score, recall_score, f1_score, confusion_matrix, log_loss.
• Ridge — L2-regularized regression (Cholesky/SVD). Succeeds on collinear inputs.
• Lasso — L1-regularized regression via coordinate descent. Produces sparse models (feature selection).
• Regressor trait — supervised counterpart of Transformer (fit(X, y) + predict).
• linalg::cholesky module — shared SPD solver foundation.
• LinearRegression — OLS regression, the crate’s first estimator.
• metrics::regression — MSE/RMSE, MAE, R², max_error, explained_variance.
• PCASolver enum (Auto / Full / Randomized) on PCA.
• jacobi::eigh_topk_flat — power-iteration + deflation for top-k eigenpairs.
• Flat-storage Jacobi eigensolver, flat matmul helpers, flat covariance.

Performance

• PCA 50 000 × 200 dropped from ~320 ms to ~104 ms with matrixmultiply.
• LinearRegression fit at 50 000 × 200: 258 ms → 84 ms with matrixmultiply.
• Scaler transform rayon threshold: scalar loop below 4 096 rows, parallel above.

0.3.0

• matrixmultiply feature: optional tuned pure-Rust GEMM (no system BLAS).
• BREAKING: Matrix internal storage switched from Vec<Vec<f64>> to a single contiguous Vec<f64>. ~13× on RobustScaler, ~5× on StandardScaler at 50 000 × 200.
• Flat-storage scalers, fused Welford statistics, NaN validation fused into transform loops.

Earlier versions

See the full changelog on GitHub for 0.1.x and 0.2.x history.