popmon package

Subpackages

Submodules

popmon.config module

class popmon.config.Comparison(**data)

Bases: BaseModel

Parameters related to comparisons

shift: int: shift of time-bins in rolling/expanding window. default is 1.

window: int: size of rolling window and/or trend detection. default is 10.

class popmon.config.Monitoring(**data)

Bases: BaseModel

Parameters related to monitoring

monitoring_rules: Dict[str, List[Union[float, int]]]

monitoring rules to generate traffic light alerts. The default setting is:

monitoring_rules = {
    "*_pull": [7, 4, -4, -7],
    "*_zscore": [7, 4, -4, -7],
    "[!p]*_unknown_labels": [0.5, 0.5, 0, 0],
}

Note that the (filename based) wildcards such as * apply to all statistic names matching that pattern. For example, "*_pull" applies for all features to all statistics ending on “_pull”. You can also specify rules for specific features and/or statistics by leaving out wildcard and putting the feature name in front. E.g.

monitoring_rules = {
    "featureA:*_pull": [5, 3, -3, -5],
    "featureA:nan": [4, 1, 0, 0],
    "*_pull": [7, 4, -4, -7],
    "nan": [8, 1, 0, 0],
}

In case of multiple rules could apply for a feature’s statistic, the most specific one applies. So in case of the statistic “nan”: “featureA:nan” is used for “featureA”, and the other “nan” rule for all other features.

pull_rules: Dict[str, List[Union[float, int]]]

red and yellow (possibly dynamic) boundaries shown in plots in the report. Default is:

pull_rules = {"*_pull": [7, 4, -4, -7]}

This means that the shown yellow boundaries are at -4, +4 standard deviations around the (reference) mean, and the shown red boundaries are at -7, +7 standard deviations around the (reference) mean. Note that the (filename based) wildcards such as * apply to all statistic names matching that pattern. (The same string logic applies as for monitoring_rules.)

class popmon.config.Report(**data)

Bases: BaseModel

Report-specific configuration

extended_report: bool: if True, show all the generated statistics in the report (optional) if set to False, then smaller show_stats (see below)

last_n: int: plot statistic data for last ‘n’ periods (optional)

online_report: bool: Use a CDN to host resources, or embed them into the report.

primary_color: str: Primary color used throughout the report

report_filepath: Union[str, Path, None]: the file path where to output the report (optional)

section: Section: Configuration for the individual sections

show_stats: List[str]: list of statistic name patterns to show in the report. If None, show all (optional)

skip_first_n: int: in plot skip first ‘n’ periods. last_n takes precedence (optional)

skip_last_n: int: in plot skip last ‘n’ periods. last_n takes precedence (optional)

title: str: Report title in browser and navbar. May contain HTML.

tl_colors: Dict[str, str]: “Configure line colors in barplots of Comparisons and Profiles section. Need to be hex format (full length)

class popmon.config.Section(**data)

Bases: BaseModel

Configuration for the individual sections

alerts: AlertSection: Configuration related to the alerts section

comparisons: ComparisonsSection: Configuration related to the comparisons section

histograms: HistogramSectionModel: Configuration related to the histogram section

overview: OverviewSection: Configuration related to the overview section

profiles: ProfilesSection: Configuration related to the profiles section

traffic_lights: TrafficLightsSection: Configuration related to the traffic lights section

class popmon.config.Settings(_env_file='<object object>', _env_file_encoding=None, _env_nested_delimiter=None, _secrets_dir=None, **values)

Bases: ValidatedSettings

bin_specs: Dict[str, Any]

dictionaries used for rebinning numeric or timestamp features. An example bin_specs dictionary is:

bin_specs = {
    "x": {"bin_width": 1, "bin_offset": 0},
    "y": {"num": 10, "low": 0.0, "high": 2.0},
    "x:y": [{}, {"num": 5, "low": 0.0, "high": 1.0}],
}

In the bin specs for x:y, x is not provided (here) and reverts to the 1-dim setting. The ‘bin_width’, ‘bin_offset’ notation makes an open-ended histogram (for that feature) with given bin width and offset. The notation ‘num’, ‘low’, ‘high’ gives a fixed range histogram from ‘low’ to ‘high’ with ‘num’ number of bins.

binning: Literal['auto', 'unit']: default binning to revert to in case bin_specs not supplied. When using “auto”, semi-clever binning is automatically done.

comparison: Comparison: Settings related to the comparisons

features: Optional[List[str]]

columns to pick up from input data. (default is all features). For multi-dimensional histograms, separate the column names with a ‘:’. Example features list is:

features = ["x", "date", "date:x", "date:y", "date:x:y"]

If time_axis is set or found, and if no features provided, features becomes: [‘date:x’, ‘date:y’, ‘date:z’] etc.

monitoring: Monitoring: Settings related to monitoring

reference_type: Literal['self', 'external', 'rolling', 'expanding', 'self_split']: type of reference used for comparisons

report: Report: Settings regarding the report

time_axis: str: name of datetime feature, used as time axis, e.g. ‘date’. (column should be timestamp, date(time) or numeric batch id) if empty string, will be auto-guessed.

class popmon.config.ValidatedSettings(_env_file='<object object>', _env_file_encoding=None, _env_nested_delimiter=None, _secrets_dir=None, **values): Bases: BaseSettings

popmon.resources module

popmon.resources.data(name)

Return the full path filename of a shipped data file.

Parameters: name (str) – The name of the data.
Returns: The full path filename of the data.
Return type: str
Raises: FileNotFoundError – If the data cannot be found.

popmon.resources.notebook(name)

Return the full path filename of a tutorial notebook.

Parameters: name (str) – The name of the notebook.
Returns: The full path filename of the notebook.
Return type: str
Raises: FileNotFoundError – If the notebook cannot be found.

popmon.resources.templates_env(filename=None, **kwargs)

Return visualization templates directory environment. If filename provided, the exact template is being retrieved and provided keyword arguments - rendered accordingly.

Parameters

filename (str) – the name of the template to get retrieved.
kwargs – residual keyword arguments which would be used for rendering

Returns

template if a filename is provided (rendered given that keyword arguments are provided) otherwise: environment of the templates directory

popmon.utils module

popmon.utils.filter_metrics(metrics, ignore_stat_endswith, show_stats=None): Filter metrics by excluding based on suffix and/or including when matching a pattern

popmon.utils.parallel(func, args_list, mode='args'): Routine for parallel processing

popmon.utils.short_date(date): Shorten date string to length of 22

popmon package

Subpackages

Submodules

popmon.config module

popmon.resources module

popmon.utils module

popmon.version module