popmon.base package

Submodules

popmon.base.module module

class popmon.base.module.Module

Bases: object

Abstract base class used for modules in a pipeline.

__init__()

Module initialization

static get_datastore_object(datastore, feature, dtype, default=None)

Get object from datastore.

Bit more advanced than dict.get()

Parameters

• datastore (dict) – input datastore

• feature (str) – key of object to retrieve

• dtype (obj) – required datatype of object. Could be specific data type or tuple of dtypes

• default (obj) – object to default to in case key not found.

Returns

retrieved object

get_features(all_features)

Get all features that meet feature_begins_with and ignore_features requirements

Parameters

all_features (list) – input features list

Returns

pruned features list

Return type

list

set_logger(logger)

Set logger of module

Parameters

logger – input logger

Return type

None

transform(*args, **kwargs)

Central function of the module.

Typically transform() takes something from the datastore, does something to it, and puts the results back into the datastore again, to be passed on to the next module in the pipeline.

Parameters

datastore (dict) – input datastore

Returns

updated output datastore

Return type

dict

class popmon.base.module.ModuleMetaClass(name, bases, local)

Bases: type

Metaclass that wraps all transform() methods using the datastore_helper This obviates the need to decorate all methods in subclasses

popmon.base.module.combine_classes(*args)

Combine multiple metaclasses

popmon.base.module.datastore_helper(func)

Decorator for passing and storing only the relevant keys in the datastore to the transform() method.

popmon.base.pipeline module

class popmon.base.pipeline.Pipeline(modules, logger=None)

Bases: object

Base class used for to run modules in a pipeline.

__init__(modules, logger=None)

Initialization of the pipeline

Parameters

• modules (list) – modules of the pipeline.

• logger (logging.Logger | None) – logger to be used by each module.

add_modules(modules)

Add more modules to existing list of modules.

Parameters

modules (list) – list of more modules

Return type

None

set_logger(logger)

Set the logger to be used by each module

Parameters

logger (logging.Logger | None) – input logger

Return type

None

transform(datastore)

Central function of the pipeline.

Calls transform() of each module in the pipeline. Typically, transform() of a module takes something from the datastore, does something to it, and puts the results back into the datastore again, to be passed on to the next module in the pipeline.

Parameters

datastore (dict) – input datastore

Returns

updated output datastore

Return type

dict

popmon.base.registry module