mindmeld.active_learning.plot_manager module

This module plots the results from the Active Learning Pipeline.

exception mindmeld.active_learning.plot_manager.MissingDataError

Bases: Exception

class mindmeld.active_learning.plot_manager.PlotManager(experiment_dir_path: str, aggregate_statistic: str, class_level_statistic: str, plot_entities: bool, plot_intents: bool)

Bases: object

Handles plotting. Plots supported: At the domain, intent and entity level:

1. Plot Single Epoch (Compares Strategies)
2. Plot Average Across Epochs (Compares Strategies)
3. Plot All Epochs (Compares Epochs)

At the domain and intent level:

1. Plot Stacked Bar (Compares Selection Distributions across Iterations)

create_plot_dir(y_keys: List)

Creates folders to support the generated path if they do not already exist. :param y_keys: Keys to access the data from a epoch to be used as y values for plotting. :type y_keys: list

generate_plots()

Entry point for generating plots as per the specifications provided in the config.

get_accuracies_json_data() → List[Dict]

Loads accuracies.json from the experiment directory path. :returns: Data loaded from accuracies.json. :rtype: data (List[dict])

static get_across_iterations(epoch_dict: Dict, selected_keys: List)

Filter data across all iterations in a single epoch as specified by a set of keys. :param epoch_dict: Dict containing accuracies across iterations for a single epoch. :type epoch_dict: dict :param selected_keys: List of keys used to filter the given dictionary. :type selected_keys: list

Returns:List of the selected data across iterations. Return type:data (list)

get_domain_list() → List

Method to get a list of domains included in training from the first epoch and iteration. :returns: List of domains for the current experiment. :rtype: domain_list (list)

get_entity_list(domain: str, intent: str) → List

Method to get a list of entities included in training from the first epoch and iteration. :param domain: The domain containing the intent to retreive entities from. :type domain: str :param intent: The intent to retreive entities from. :type intent: str

Returns:List of entities in the given intent. Return type:entity_list (list)

get_img_path(y_keys: List, file_name: str)

Parameters:

• y_keys (list) -- Keys to access the data from a epoch to be used as y values for plotting.
• file_name (str) -- Name of the file to save.

Returns:

Path of the image to be saved.

Return type:

img_path (str)

get_intent_list(domain: str) → List

Method to get a list of intents included in training from the first epoch and iteration. :param domain: The domain to get retrieve intents for. :type domain: str

Returns:List of intent for a given domain in the current experiment. Return type:intent_list (list)

static get_label_set_counter(all_counters: List, unique_labels: List) → Dict

Parameters:

• all_counters (list) -- List of Counters, each counter represents a single iteration.
• unique_labels (list) -- A List of unique and sorted keys across all_counters.

Returns:

Each unique label is mapped to a list, the value at each

index in the list corresponds to the count of the label in that iteration.

Return type:

label_set_counter (dict)

static get_nested(data_dict: Dict, selected_keys: List)

Filter data from a nested dictionary selecting from a set of keys. :param data_dict: Dictionary containing data to filter :type data_dict: dict :param selected_keys: List of keys used to filter the given dictionary. :type selected_keys: list

Returns:Dictionary containing the filtered nested data. Return type:data_dict (dict)

get_queries_json_data() → List[Dict]

Loads selected_queries.json from the experiment directory path. selected_queries.json stores the queries selected by active learning at each iteration. :returns: Data loaded from selected_queries.json. :rtype: data (List[dict])

static get_unique_labels(all_counters: List) → List

Parameters:all_counters (list) -- List of Counters, each counter represents a single iteration. Returns:A List of unique and sorted keys across all_counters. Return type:unique_labels (list)

plot_all_epochs(y_keys: List, display: bool = False, save: bool = True, use_aggregate_statistic: bool = True)

Plot all epochs. Creates a plot for each strategy. :param y_keys: Keys to access the data from a epoch to be used as y values for plotting. :type y_keys: list :param display: Whether to show the plot. :type display: bool :param save: Whether to save the plot. :type save: bool :param use_aggregate_statistic: If True, the aggregate_statistic will be used as the

label for the y_axis. If False, the class_level_statistic will be used.

plot_avg_across_epochs(y_keys: List, display: bool = False, save: bool = True, use_aggregate_statistic: bool = True)

Plot average accuracies across all epochs for each strategy. :param y_keys: Keys to access the data from a epoch to be used as y values for plotting. :type y_keys: list :param display: Whether to show the plot. :type display: bool :param save: Whether to save the plot. :type save: bool :param use_aggregate_statistic: If True, the aggregate_statistic will be used as the

label for the y_axis. If False, the class_level_statistic will be used.

plot_single_epoch(y_keys: List, epoch: int = 0, display: bool = False, save: bool = True, use_aggregate_statistic: bool = True)

Plot accuracies across a single epoch for each strategy. :param y_keys: Keys to access the data from a epoch to be used as y values for plotting. :type y_keys: list :param epoch: The epoch to plot. :type epoch: int :param display: Whether to show the plot. :type display: bool :param save: Whether to save the plot. :type save: bool :param use_aggregate_statistic: If True, the aggregate_statistic will be used as the

label for the y_axis. If False, the class_level_statistic will be used.

plot_stacked_bar(epoch: int = 0, plot_domains: bool = True, plot_intents: bool = True, **kwargs)

Plots a stacked bar graph of selection distributions across iterations for an epoch. :param epoch: The epoch to plot. :type epoch: int :param plot_domain: Whether to generate plots at the domain level. :type plot_domain: bool :param plot_intents: Whether to generate plots at the intent level. :type plot_intents: bool

plotting_wrapper(function, plot_domain: bool = True)

Plotting wrapper functions for plots that use data from accuracies.json :param function: plotting function. :type function: generator :param plot_domain: Whether to generate plots at the domain level. :type plot_domain: bool :param plot_intents: Whether to generate plots at the intent level. :type plot_intents: bool :param plot_entities: Whether to generate plots at the entity level. :type plot_entities: bool

queries_json_data_has_data() → Dict

Checks whether queries.json is empty {}. :returns: Whether queries.json has data. :rtype: has_data (bool)

strategies

**Returns* -- strategies (list)* -- List of selection strategies for the given experiment.