Layer Mechanics¶
ML-Ensemble is designed to provide an easy user interface. But it is also designed to be extremely flexible, all the wile providing maximum concurrency at minimal memory consumption. The lower-level API that builds the ensemble and manages the computations is constructed in as modular a fashion as possible.
The low-level API introduces a computational graph-like environment that you can
directly exploit to gain further control over your ensemble. In fact, building
your ensemble through the low-level API is almost as straight forward as using
the high-level API. In this tutorial, we will walk through how to use the
Group and Layer classes to fit several learners.
Suppose we want to fit several learners. The :ref:` learner tutorial <learner_tutorial` showed us how to fit a single learner, and so one approach would be to simple iterate over our learners and fit them one at a time. This however is a very slow approach since we don’t exploit the fact that learners can be trained in parallel. Moreover, any type of aggregation, like putting all predictions into an array, would have to be done manually.
The Layer API¶
To parallelize the implementation, we can use the Layer class. A layer is
a handle that will run any number of Group instances attached to it in parallel. Each
group in turn is a wrapper around a indexer-transformers-estimators triplet.
Basics¶
So, to fit our two learners in parallel, we first need a Group object to
handle them.
from mlens.parallel import Layer, Group, make_group, run
from mlens.utils.dummy import OLS, Scale
from mlens.index import FoldIndex
indexer = FoldIndex(folds=2)
group = make_group(indexer, [OLS(1), OLS(2)], None)
This group object is now a complete description of how to fit our two
learners using the prescribed indexing method.
To train the estimators, we need feed the group to a Layer instance:
import numpy as np
np.random.seed(2)
X = np.arange(20).reshape(10, 2)
y = np.random.rand(10)
layer = Layer(stack=group)
print(
run(layer, 'fit', X, y, return_preds=True)
)
Out:
[[ 1.3665537 2.3665535]
[ 5.363353 10.363353 ]
[ 9.360152 18.360151 ]
[13.356951 26.35695 ]
[17.35375 34.35375 ]
[21.486897 42.486897 ]
[25.524712 50.52471 ]
[29.562525 58.562527 ]
[33.60034 66.60034 ]
[37.638153 74.63815 ]]
To use some preprocessing before fitting the estimators, we can use the
transformers argument when creating our group:
group = make_group(indexer, [OLS(1), OLS(2)], [Scale()])
layer = Layer(stack=group)
print(
run(layer, 'fit', X, y, return_preds=True)
)
Out:
[[-27.977594 -55.977592]
[-23.980795 -47.980793]
[-19.983995 -39.983997]
[-15.987196 -31.987196]
[-11.990397 -23.990396]
[ 12.113442 24.113443]
[ 16.151257 32.151257]
[ 20.189072 40.18907 ]
[ 24.226885 48.226887]
[ 28.2647 56.264698]]
Multitasking¶
If we want our estimators two have different preprocessing, we can easily achieve this either by specifying different cases when making the group, or by making two separate groups. In the first case:
group = make_group(
indexer,
{'case-1': [OLS(1)], 'case-2': [OLS(2)]},
{'case-1': [Scale()], 'case-2': []}
)
layer = Layer(stack=group)
print(
run(layer, 'fit', X, y, return_preds=True)
)
Out:
[[-27.977594 2.3665535]
[-23.980795 10.363353 ]
[-19.983995 18.360151 ]
[-15.987196 26.35695 ]
[-11.990397 34.35375 ]
[ 12.113442 42.486897 ]
[ 16.151257 50.52471 ]
[ 20.189072 58.562527 ]
[ 24.226885 66.60034 ]
[ 28.2647 74.63815 ]]
In the latter case:
groups = [
make_group(indexer, OLS(1), Scale()), make_group(indexer, OLS(2), None)
]
layer = Layer(stack=groups)
print(
run(layer, 'fit', X, y, return_preds=True)
)
Out:
[[-27.977594 2.3665535]
[-23.980795 10.363353 ]
[-19.983995 18.360151 ]
[-15.987196 26.35695 ]
[-11.990397 34.35375 ]
[ 12.113442 42.486897 ]
[ 16.151257 50.52471 ]
[ 20.189072 58.562527 ]
[ 24.226885 66.60034 ]
[ 28.2647 74.63815 ]]
Which method to prefer depends on the application, but generally, it is
preferable to put all transformers and all estimators belonging to a
given indexing strategy into one group instance as it is easier to
separate groups based on indexer and using cases to distinguish between
different preprocessing pipelines.
Now, suppose we want to do something more exotic, like using different indexing strategies for different estimators. This can easily be achieved by creating groups for each indexing strategy we want:
groups = [
make_group(FoldIndex(2), OLS(1), Scale()),
make_group(FoldIndex(4), OLS(2), None)
]
layer = Layer(stack=groups)
print(
run(layer, 'fit', X, y, return_preds=True)
)
Out:
[[-27.977594 2.4661984]
[-23.980795 10.449842 ]
[-19.983995 18.433487 ]
[-15.987196 26.33962 ]
[-11.990397 34.341675 ]
[ 12.113442 42.343727 ]
[ 16.151257 50.33181 ]
[ 20.189072 58.324955 ]
[ 24.226885 66.458244 ]
[ 28.2647 74.47614 ]]
Some care needs to be taken here: if indexing strategies do not return the same number of rows, the output array will be zero-padded.
from mlens.index import BlendIndex
groups = [
make_group(FoldIndex(2), OLS(1), None),
make_group(BlendIndex(0.5), OLS(1), None)
]
layer = Layer(stack=groups)
print(
run(layer, 'fit', X, y, return_preds=True)
)
Out:
[[ 1.3665537 0. ]
[ 5.363353 0. ]
[ 9.360152 0. ]
[13.356951 0. ]
[17.35375 0. ]
[21.486897 21.486897 ]
[25.524712 25.524712 ]
[29.562525 29.562525 ]
[33.60034 33.60034 ]
[37.638153 37.638153 ]]
Note that even if mlens indexer output different shapes, they preserve
row indexing to ensure predictions are consistently mapped to their respective
input. If you build a custom indexer, make sure that it uses a strictly
sequential (with respect to row indexing) partitioning strategy.
Layer features¶
A layer does not have to be specified all in one go; you can instantiate
a layer and push and pop to its stack.
layer = Layer()
group = make_group(FoldIndex(4), OLS(), None)
layer.push(group)
If you push or pop to the stack, you must call fit before you can
use the layer for prediction.
run(layer, 'fit', X, y)
group = make_group(FoldIndex(2), OLS(1), None)
layer.push(group)
try:
run(layer, 'predict', X, y)
except Exception as exc:
print("Error: %s" % str(exc))
Out:
Error: Layer instance (layer-6) not fitted.
The Layer class can print the progress of a job, as well as inspect
data collected during the job. Note that the
printouts of the layer does not take group membership into account.
from mlens.metrics import rmse
layer = Layer()
group1 = make_group(
indexer,
{'case-1': [OLS(1)], 'case-2': [OLS(2)]},
{'case-1': [Scale()], 'case-2': []},
learner_kwargs={'scorer': rmse}
)
layer.push(group1)
run(layer, 'fit', X, y, return_preds=True)
print()
print("Collected data:")
print(layer.data)
Out:
Collected data:
score-m score-s ft-m ft-s pt-m pt-s
case-1 ols-1 20.88 0.23 0.00 0.00 0.00 0.00
ols-2 40.27 19.05 0.00 0.00 0.00 0.00
Total running time of the script: ( 0 minutes 2.157 seconds)