Overview

To build computational graphs, you need to specify an estimator, a (set of) indexer(s), and any preprocessing pipelines wanted. The indexer(s) determine the cross validation strategy to use during fit and transform calls. The FullIndex can be used if no cross-validation is wanted.

The basic node is the Learner. The learner contains the sub-graph pertaining to a specific indexer-estimator-preprocessing configuration. Similarly, the Transformer contains the sub-graph of a specific indexer-preprocessing configuration. To learn more, see the learner mechanics tutorial.

To build a graph of a set of Learner` and Transformer instances, several handles come in handy for ensuring efficient computation. The Group class is a handle for a set of learners and transformers sharing a specific indexer, while the Layer class is a handle for a set of groups. To learn more about how to efficiently parallelize independent estimations, see the layer mechanics tutorial.

Frequently, we want the output of some set of learners to be feed as input to some other set. The ParallelProcessing engine is a purpose-built parallel job-execution engine design to handle these types of scenarios. Together with the Sequential handle, designing deep computational graphs is straightforward. See the advanced graph mechanics tutorial for more information.