scikit-transformers is a very usefull package to enable and provide custom transformers such as LogColumnTransformer, BoolColumnTransformers and others fancy transformers.
It was created to provide a simple way to use custom transformers in scikit-learn pipelines, and allow to use them in a scikit-learn model, using GridSearchCV for testing and tuning hyperparameters.
The starting point was to provide a simple LogColumnTransformer, which is a simple wrapper around the numpy log function, making possible to use a skew threshold to apply the log transformation only on columns with a skew superior to a given threshold.
With scikit-transformers, it is now possible to use this LogColumnTransformer in transformer in a GridSearchCV using a skew threshold as hyperparameter to find what columns are good to log or not.
LogColumnTransformer is one of the many transformers implemented in scikit-transformers.
Using regular pip and venv tools :
python3 -m venv .venv
source .venv/bin/activate
pip install scikit-transformersFor a very basic usage :
import pandas as pd
from sktransf.trasnformer import LogColumnTransformer
df = pd.DataFrame(
{ "a": range(10),
"b": range(10)
}
)
logger = LogColumnTransformer()
logger.fit_transform(df)
df_transf = logger.transform(df)Using common transformers :
import pandas as pd
from sktransf.transformer import LogColumnTransformer, BoolColumnTransformer
from sktransf.selector import DropUniqueColumnSelector
df = pd.DataFrame(
{ "a": range(10),
"b": range(10)
}
)
df_bool = BoolColumnTransformer().fit_transform(df)
df_unique = DropUniqueColumnTransformer().fit_transform(df)
df_logged = LogColumnTransformer().fit_transform(df)Using a pipeline with a scikit-learn model :
import pandas as pd
from sklearn.pipeline import Pipeline
from sklearn.linear_model import LinearRegression
from sktransf.transformer import LogColumnTransformer, BoolColumnTransformer
from sktransf.selector import DropUniqueColumnSelector
pipe = Pipeline([
('bool', BoolColumnTransformer()),
('unique', DropUniqueColumnTransformer()),
('log', LogColumnTransformer()),
('model', LinearRegression())
])
X = pd.DataFrame(
{ "a": range(10),
"b": range(10)
}
)
y = range(10)
pipe.fit(X, y)
y_pred = pipe.predict(X)For more specific information, please refer to the notebooks:
- Transformers :
- Selectors :
- Pipelines :
A complete documentation is be available on the github page.
Please refer to the changelog page for more information.
Pull requests are welcome.
For major changes, please open an issue first to discuss what you would like to change.
For more information, please refer to the contributing page.