popmon package
Subpackages
- popmon.alerting package
- popmon.analysis package
- popmon.base package
- popmon.extensions package
- popmon.hist package
- popmon.io package
- popmon.pipeline package
- Submodules
- popmon.pipeline.amazing_pipeline module
- popmon.pipeline.dataset_splitter module
- popmon.pipeline.metrics module
- popmon.pipeline.metrics_pipelines module
- popmon.pipeline.report module
- popmon.pipeline.report_pipelines module
- popmon.stats package
- popmon.stitching package
- popmon.visualization package
- Submodules
- popmon.visualization.alert_section_generator module
- popmon.visualization.histogram_section module
- popmon.visualization.overview_section module
- popmon.visualization.report_generator module
- popmon.visualization.section_generator module
- popmon.visualization.traffic_light_section_generator module
- popmon.visualization.utils module
Submodules
popmon.config module
- class popmon.config.Comparison(**data)
Bases:
BaseModel
Parameters related to comparisons
- shift: int
shift of time-bins in rolling/expanding window. default is 1.
- window: int
size of rolling window and/or trend detection. default is 10.
- class popmon.config.Monitoring(**data)
Bases:
BaseModel
Parameters related to monitoring
- monitoring_rules: Dict[str, List[Union[float, int]]]
monitoring rules to generate traffic light alerts. The default setting is:
monitoring_rules = { "*_pull": [7, 4, -4, -7], "*_zscore": [7, 4, -4, -7], "[!p]*_unknown_labels": [0.5, 0.5, 0, 0], }
Note that the (filename based) wildcards such as * apply to all statistic names matching that pattern. For example,
"*_pull"
applies for all features to all statistics ending on “_pull”. You can also specify rules for specific features and/or statistics by leaving out wildcard and putting the feature name in front. E.g.monitoring_rules = { "featureA:*_pull": [5, 3, -3, -5], "featureA:nan": [4, 1, 0, 0], "*_pull": [7, 4, -4, -7], "nan": [8, 1, 0, 0], }
In case of multiple rules could apply for a feature’s statistic, the most specific one applies. So in case of the statistic “nan”: “featureA:nan” is used for “featureA”, and the other “nan” rule for all other features.
- pull_rules: Dict[str, List[Union[float, int]]]
red and yellow (possibly dynamic) boundaries shown in plots in the report. Default is:
pull_rules = {"*_pull": [7, 4, -4, -7]}
This means that the shown yellow boundaries are at -4, +4 standard deviations around the (reference) mean, and the shown red boundaries are at -7, +7 standard deviations around the (reference) mean. Note that the (filename based) wildcards such as * apply to all statistic names matching that pattern. (The same string logic applies as for monitoring_rules.)
- class popmon.config.Report(**data)
Bases:
BaseModel
Report-specific configuration
- extended_report: bool
if True, show all the generated statistics in the report (optional) if set to False, then smaller show_stats (see below)
- last_n: int
plot statistic data for last ‘n’ periods (optional)
- online_report: bool
Use a CDN to host resources, or embed them into the report.
- primary_color: str
Primary color used throughout the report
- report_filepath: Optional[Union[str, Path]]
the file path where to output the report (optional)
- show_stats: List[str]
list of statistic name patterns to show in the report. If None, show all (optional)
- skip_first_n: int
in plot skip first ‘n’ periods. last_n takes precedence (optional)
- skip_last_n: int
in plot skip last ‘n’ periods. last_n takes precedence (optional)
- title: str
Report title in browser and navbar. May contain HTML.
- tl_colors: Dict[str, str]
“Configure line colors in barplots of Comparisons and Profiles section. Need to be hex format (full length)
- class popmon.config.Section(**data)
Bases:
BaseModel
Configuration for the individual sections
- alerts: AlertSection
Configuration related to the alerts section
- comparisons: ComparisonsSection
Configuration related to the comparisons section
- histograms: HistogramSectionModel
Configuration related to the histogram section
- overview: OverviewSection
Configuration related to the overview section
- profiles: ProfilesSection
Configuration related to the profiles section
- traffic_lights: TrafficLightsSection
Configuration related to the traffic lights section
- class popmon.config.Settings(_env_file='<object object>', _env_file_encoding=None, _env_nested_delimiter=None, _secrets_dir=None, **values)
Bases:
ValidatedSettings
- bin_specs: Dict[str, Any]
dictionaries used for rebinning numeric or timestamp features. An example bin_specs dictionary is:
bin_specs = { "x": {"bin_width": 1, "bin_offset": 0}, "y": {"num": 10, "low": 0.0, "high": 2.0}, "x:y": [{}, {"num": 5, "low": 0.0, "high": 1.0}], }
In the bin specs for x:y, x is not provided (here) and reverts to the 1-dim setting. The ‘bin_width’, ‘bin_offset’ notation makes an open-ended histogram (for that feature) with given bin width and offset. The notation ‘num’, ‘low’, ‘high’ gives a fixed range histogram from ‘low’ to ‘high’ with ‘num’ number of bins.
- binning: Literal['auto', 'unit']
default binning to revert to in case bin_specs not supplied. When using “auto”, semi-clever binning is automatically done.
- comparison: Comparison
Settings related to the comparisons
- features: Optional[List[str]]
columns to pick up from input data. (default is all features). For multi-dimensional histograms, separate the column names with a ‘:’. Example features list is:
features = ["x", "date", "date:x", "date:y", "date:x:y"]
If time_axis is set or found, and if no features provided, features becomes: [‘date:x’, ‘date:y’, ‘date:z’] etc.
- monitoring: Monitoring
Settings related to monitoring
- reference_type: Literal['self', 'external', 'rolling', 'expanding', 'self_split']
type of reference used for comparisons
- time_axis: str
name of datetime feature, used as time axis, e.g. ‘date’. (column should be timestamp, date(time) or numeric batch id) if empty string, will be auto-guessed.
- class popmon.config.ValidatedSettings(_env_file='<object object>', _env_file_encoding=None, _env_nested_delimiter=None, _secrets_dir=None, **values)
Bases:
BaseSettings
popmon.resources module
- popmon.resources.data(name)
Return the full path filename of a shipped data file.
- Parameters
name (str) – The name of the data.
- Returns
The full path filename of the data.
- Return type
str
- Raises
FileNotFoundError – If the data cannot be found.
- popmon.resources.notebook(name)
Return the full path filename of a tutorial notebook.
- Parameters
name (str) – The name of the notebook.
- Returns
The full path filename of the notebook.
- Return type
str
- Raises
FileNotFoundError – If the notebook cannot be found.
- popmon.resources.templates_env(filename=None, **kwargs)
Return visualization templates directory environment. If filename provided, the exact template is being retrieved and provided keyword arguments - rendered accordingly.
- Parameters
filename (str) – the name of the template to get retrieved.
kwargs – residual keyword arguments which would be used for rendering
- Returns
template if a filename is provided (rendered given that keyword arguments are provided) otherwise: environment of the templates directory
popmon.utils module
- popmon.utils.filter_metrics(metrics, ignore_stat_endswith, show_stats=None)
Filter metrics by excluding based on suffix and/or including when matching a pattern
- popmon.utils.parallel(func, args_list, mode='args')
Routine for parallel processing
- popmon.utils.short_date(date)
Shorten date string to length of 22
popmon.version module
THIS FILE IS AUTO-GENERATED BY SETUP.PY.