Skrub

Ever tried to run a scikit-learn pipeline on a dataframe with mixed types? You need one encoder for categorical columns, another for text, a third for dates... it gets verbose fast! skrub.TableVectorizer is the "fit once, vectorize any table" transformer. skrub-data.org/stable/refer...

The TableVectorizer inspects each column's dtype and cardinality, then dispatches to the right encoder: • Numeric → passthrough (cast to float32) • Low-cardinality strings (<40 unique) → OneHotEncoder • High-cardinality strings (≥40 unique) → StringEncoder • Dates → DatetimeEncoder

It also handles the messy stuff: null strings like "N/A", date format inference, numeric strings, and dropping all-null columns. Use it standalone or inside tabular_pipeline() for a full preprocessing + model pipeline in one call.

🪄 Text in. Features out. Turn information rich strings into meaningful encodings via your favorite pretrained language models from Hugging Face 🤗 Encode text columns for machine learning with skrub.LLMEncoder. pip install skrub[transformers]

skrub-data.org/stable/refer...

skrub-data.orgLLMEncoderEncode string features by applying a pretrained language model downloaded from the HuggingFace Hub.

Messy numeric strings → float32 in one line: skrub.ToFloat ✨ Comma decimals, thousands separators, (negative) parentheses -- all covered, in both pandas and polars.

skrub-data.org/stable/refer...

skrub-data.orgToFloatConvert a column to 32-bit floating-point numbers.

✨ Skrub 0.11 has been released ✨ Full changelog: skrub-data.org/stable/CHANG... Heads-up: skrub 0.11 requires Python ≥ 3.11 and scikit-learn ≥ 1.5.2. Highlights in thread ⤵️

Also in this release: faster construction of deep DataOps, ToCategorical for numeric columns, and more!

🫂13 new contributors helped with this release!

✨ Skrub 0.11 has been released ✨ Full changelog: skrub-data.org/stable/CHANG... Heads-up: skrub 0.11 requires Python ≥ 3.11 and scikit-learn ≥ 1.5.2. Highlights in thread ⤵️

📊 DataOp reports now link to your source code and show docstrings -- can be generated without executing calculations

🧠 TextEncoder is now LLMEncoder -- clearer name, same transformer embeddings

✨ Skrub 0.11 has been released ✨ Full changelog: skrub-data.org/stable/CHANG... Heads-up: skrub 0.11 requires Python ≥ 3.11 and scikit-learn ≥ 1.5.2. Highlights in thread ⤵️

🔍 describe_transformations() -- can now print a plain text explanation of what TableVectorizer and Cleaner did to each column

⚡ The SessionEncoder is now up to 15x faster

✨ Skrub 0.11 has been released ✨ Full changelog: skrub-data.org/stable/CHANG... Heads-up: skrub 0.11 requires Python ≥ 3.11 and scikit-learn ≥ 1.5.2. Highlights in thread ⤵️

🆕 CatEncoder -- OneHotEncoder + TargetEncoder, two encoders combined in a single column transformer to deal with columns with many infrequent classes

💾 Persistent caching for DataOps -- pipeline steps can now be cached across runs

github.comRelease 0.10.1 · skrub-data/skrub🌟 Skrub 0.10.1 has been released 🌟 Main features When a cross-validation splitter has been passed to .skb.mark_as_X(), it it now possible to select a specific split from it when calling .skb.train...

🐞 Bug fix release incoming 🐛 Version 0.10.1 of skrub is now live! In this release, we fixed a Cloudpickle import error that could be triggered by some updated dependencies, along with some other improvements. Release post: github.com/skrub-data/s... 1/4

- Improved granular access to split when using the DataOps splitter. Can now select a specific split based on index (i.e. the ith split out of N splits) - More bugfixes! 3/4

doi.orgskrub-data/skrub: 0.10.1🌟 Skrub 0.10.1 has been released 🌟 Main features When a cross-validation splitter has been passed to .skb.mark_as_X(), it it now possible to select a specific split from it when calling .skb.train_test_split() by passing split_index; for example data_op.skb.train_test_split(split_index=3) to get the third split. #2213 @jeromedockes. Added support in tabular_pipeline() for estimators instantiated from either tabicl.TabICLClassifier or tabicl.TabICLRegressor with recommended default parameters of TableVectorizer as the first step, and the estimator as the second step. #2222 by @ashwinvis. Changes TextEncoder's verbose parameter is now an int instead of a bool, where verbose=0 silences the progress bar and verbose>=1 shows it. The default is now 0. #2249 by @Jayant-kernel. patch_display() now uses a minimal, faster TableReport without plots or column associations by default. #2103 by @ashiorkornortey. Bugfixes DropSimilar now works with Polars dataframes when PyArrow is not installed. #2216 by @ShreyanshGoyal. The parallel coordinate plot created by ParamSearch.show_results() could have incorrect tick labels in some cases. This has been fixed in #2215 by @jeromedockes. GapEncoder with init="k-means" raised an error when the input column contained missing values. This has been fixed in #2238 by @Hrafz. DatetimeEncoder no longer raises UnboundLocalError when resolution=None is combined with periodic_encoding="circular" or "spline", and no longer fits an unused weekday periodic encoder when resolution is finer than "hour". #2240 by @Hrafz. When cols was not provided, AggJoiner and MultiAggJoiner selected the columns to aggregate through a Python set, so the order of the aggregated output columns varied between runs. They now keep the order in which the columns appear in the auxiliary table. This has been fixed in #2250 by @dylanpulver. A cloudpickle import error that could happen after updating some required dependencies was fixed in #2261 by @jeromedockes and @rcap107. Full changelog: https://github.com/skrub-data/skrub/compare/0.10.0...0.10.1

And long overdue, a Zenodo record for citing the library: doi.org/10.5281/zeno... 4/4

📋 Also included in this release: - Estimators from the Tabular In-Context Learning (TabICL) library are now accepted when running default pipelines in skrub, which the correct default parameters based on whether or not the estimator is a classifier or regressor 2/4

github.comRelease 0.10.0 · skrub-data/skrub✨ Skrub version 0.10.0 has been released ✨ Main Changes New transformers are now available: SessionEncoder groups timestamped data by sessions, DropSimilar helps with dropping redundant columns. T...

✨ Skrub version 0.10.0 has been released ✨ This is one of our biggest releases yet, with new transformers, improvements to the TableReport and the Data Ops, bug fixes and polishing of the docs. github.com/skrub-data/s... 🚀 Highlights below

More bugs have been squashed, and docs have been polished.

Last but not least, **17** new contributors helped with this release 🎉

The TableReport can now be exported in text format (Markdown formatting), or as a dictionary. In general, it is now easier to access the statistics measured by the TableReport in third party tools.

DataOps have been improved with better caching, as well as easier access and search of nodes in the graph. It is also possible to filter which columns are shown in the parallel coordinate plot.

New transformers are now available: SessionEncoder groups timestamped data by sessions, DropSimilar helps with dropping redundant columns.

ToFloat now includes decimal and thousand as parameters to parse numerical columns that use formatting different from the python default (such as "1 234,5"). Negative numbers represented with parentheses are also parsed ("(123)" becomes "-123").

github.comRelease 0.9.0 · skrub-data/skrub✨ Skrub version 0.9.0 has been released ✨ Main changes Scorers used by Data Ops can now take additional arguments (like sample weights). By @jeromedockes in #1995 The new methods .skb.find() and ....

✨ Skrub version 0.9.0 has been released ✨ This release adds some advanced features to the Data Ops, the has_dtype() selector, as well as some clarity improvements for the Cleaner and TableReport. Release post: github.com/skrub-data/s...

- The TableReport now uses plot_distributions and compute_associations to control the distribution and association tabs respectively. - The cleaner now allows to control whether numeric-looking strings ("['1', '2', '3']") should be parsed to float.

- fuzzy_join and Joiner now allow to choose the metric that should be used for matching. - ApplyToCols now has the exclude_cols parameter, to define which columns should not be transformed.

- It is now possible to pass arguments to the scorers in Data Ops, such as sample weights. - Diagrams for the Learner and parameter searches now include the full DataOp graph in their notebook repr. - It is now possible to find nodes by name in the DataOp graph.

skrub-data.orgRelease historyRelease 0.8.0: New Features: The eager_data_ops configuration option has been added. When set to False, no previews are computed and validation is deferred until the DataOp is actually used (e.g. w...

✨ skrub version 0.8.0 has been released ✨ This version includes several new features, including multiple improvements to the functionality and performance of the Data Ops, along with a few bug fixes and improvements to the docs. Changelog: skrub-data.org/stable/CHANG... Highlights below ⤵️

The TableReport custom filters have been improved and expanded: they can now take skrub selectors for filtering columns. The interface has also been simplified.

The minimum required version of polars has been increased from 0.20 to 1.5.

It is now possible to provide custom null values to the Cleaner, so that they are marked as nulls (for example, the string "unknown").

The has_nulls selector can now select columns based on a user-specified threshold of null values.

The SingleColumnTransformer and RejectColumn classes allow the construction of custom-made transformers for specific use cases.

The performance of DataOps with many computational nodes has been improved. Additionally, DataOps CV splitters can now take kwargs. For example, this allows to specify groups when creating train/test splits.

The ApplyToCols transformer is now a powerful alternative to the regular scikit-learn ColumnTransformer. It is now possible to apply any transformer to a subset of chosen columns using the skrub selectors.

As a follow-up, we would like to clarify how we’ll be crediting contributors moving forward. Currently, all contributions to the repository are tracked in the changelog and highlighted in the release notes, where each PR and the GitHub handle of its author are listed.

Skrub@skrub-data.bsky.social

Thanks to e-strauss for writing this example!

In addition, we will begin crediting specific contributors here on Bluesky when a contributor has worked on the subject of the post. We will use GitHub handles for this purpose. If you prefer your handle not to be used or would like to be credited by name instead, please let us know.

discord.ggJoin the Skrub Discord Server!Check out the Skrub community on Discord – hang out with 106 other members and enjoy free voice and text chat.

You can contact us either here or on our Discord server: discord.gg/ABaPnm7fDC

skrub-data.orgUsing PyTorch (via skorch) in DataOpsThis example shows how to wrap a PyTorch model with skorch and plug it into a skrub DataOps plan. The main goal here is to show the integration pattern: PyTorch defines the model (an nn.Module), sk...

While skrub Data Ops shine when preparing dataframes, their capabilities extend beyond that. For example, they can be used alongside libraries like PyTorch and skorch to work with images, and tune the model size to find the best set of hyperparameters: skrub-data.org/stable/auto_...

Thanks to e-strauss for writing this example!

github.comRelease Skrub release 0.7.2 · skrub-data/skrub✨ skrub version 0.7.2 has been released ✨ In this release we squashed more bugs, improved the API reference, and added a new example. Main changes: The StringEncoder now exposes the vocabulary par...

10 Feb

✨ skrub version 0.7.2 has been released ✨ In this release we squashed more bugs, improved the API reference, and added a new example. github.com/skrub-data/s...

10 Feb

Main changes: - The StringEncoder now exposes the vocabulary parameter, allowing it to be passed to the underlying TfidfVectorizer. - The function compute_ngram_distance has been made private to reduce clutter. - The repository wheel has been made smaller by removing some benchmarking material.

10 Feb

- A new example has been added to show how skrub Data Ops can be used with pytorch and skorch to solve an image classification task. skrub-data.org/stable/auto_...

5 Feb

Did you know that the skrub Data Ops support Optuna as backend to run hyperparameter search? It's as easy as writing "backend='optuna'": this will set up a default Optuna study (and the TPE sampler) to replace the standard random sampler.

5 Feb

At the end, you get a fully-fledged Optuna study to work with. Of course, that includes support for the Optuna dashboard and access to the Optuna reporting and plotting interfaces.