popmon.base package
Submodules
popmon.base.module module
- class popmon.base.module.Module
Bases:
object
Abstract base class used for modules in a pipeline.
- __init__()
Module initialization
- static get_datastore_object(datastore, feature, dtype, default=None)
Get object from datastore.
Bit more advanced than dict.get()
- Parameters
datastore (dict) – input datastore
feature (str) – key of object to retrieve
dtype (obj) – required datatype of object. Could be specific data type or tuple of dtypes
default (obj) – object to default to in case key not found.
- Returns
retrieved object
- get_features(all_features)
Get all features that meet feature_begins_with and ignore_features requirements
- Parameters
all_features (list) – input features list
- Returns
pruned features list
- Return type
list
- set_logger(logger)
Set logger of module
- Parameters
logger – input logger
- transform(*args)
Central function of the module.
Typically transform() takes something from the datastore, does something to it, and puts the results back into the datastore again, to be passed on to the next module in the pipeline.
- Parameters
datastore (dict) – input datastore
- Returns
updated output datastore
- Return type
dict
- class popmon.base.module.ModuleMetaClass(name, bases, local)
Bases:
type
Metaclass that wraps all transform() methods using the datastore_helper This obviates the need to decorate all methods in subclasses
- popmon.base.module.combine_classes(*args)
Combine multiple metaclasses
- popmon.base.module.datastore_helper(func)
Decorator for passing and storing only the relevant keys in the datastore to the transform() method.
popmon.base.pipeline module
- class popmon.base.pipeline.Pipeline(modules, logger=None)
Bases:
object
Base class used for to run modules in a pipeline.
- __init__(modules, logger=None)
Initialization of the pipeline
- Parameters
modules (list) – modules of the pipeline.
logger (logging.Logger | None) – logger to be used by each module.
- add_modules(modules)
Add more modules to existing list of modules.
- Parameters
modules (list) – list of more modules
- set_logger(logger)
Set the logger to be used by each module
- Parameters
logger (logging.Logger | None) – input logger
- transform(datastore)
Central function of the pipeline.
Calls transform() of each module in the pipeline. Typically, transform() of a module takes something from the datastore, does something to it, and puts the results back into the datastore again, to be passed on to the next module in the pipeline.
- Parameters
datastore (dict) – input datastore
- Returns
updated output datastore
- Return type
dict