comet_ml.ConfusionMatrix ¶

Data structure for holding a confusion matrix of values and their labels.

Create the ConfusionMatrix data structure.

Parameters:

• y_true –

  (optional) list of vectors representing the targets, or a list of integers representing the correct label. If not provided, then matrix may be provided.

• y_predicted –

  (optional) list of vectors representing predicted values, or a list of integers representing the output. If not provided, then matrix may be provided.

• images –

  (optional) a list of data that can be passed to Experiment.log_image()

• labels –

  (optional) a list of strings that name of the columns and rows, in order. By default, it will be "0" through the number of categories (e.g., rows/columns).

• matrix –

  (optional) the confusion matrix (list of lists). Must be square, if given. If not given, then it is possible to provide y_true and y_predicted.

• examples –

  (optional) if matrix is given you may also give examples, a dictionary of {(x, y): [index1, index2, ...]} where x is column and y is row of matrix

• title –

  (optional) a custom name to be displayed. By default, it is "Confusion Matrix".

• row_label –

  (optional) label for rows. By default, it is "Actual Category".

• column_label –

  (optional) label for columns. By default, it is "Predicted Category".

• max_examples_per_cell –

  (optional) maximum number of examples per cell. By default, it is 25.

• max_categories –

  (optional) max number of columns and rows to use. By default, it is 25.

• winner_function –

  (optional) a function that takes in an entire list of rows of patterns, and returns the winning category for each row. By default, it is argmax.

• index_to_example_function –

  (optional) a function that takes an index and returns either a number, a string, a URL, or a {"sample": str, "assetId": str} dictionary. See below for more info. By default, the function returns a number representing the index of the example.

• cache –

  (optional) should the results of index_to_example_function be cached and reused? By default, cache is True.

• selected –

  (optional) None, or list of selected category indices. These are the rows/columns that will be shown. By default, select is None. If the number of categories is greater than max_categories, and selected is not provided, then selected will be computed automatically by selecting the most confused categories.

• experiment –

  (optional) passed automatically when using Experiment.create_confusion_matrix().

• kwargs –

  (optional) any extra keywords and their values will be passed onto the index_to_example_function.

# Typically, you can log the y_true/y_predicted or matrix:

>>> experiment = Experiment()

# If you have a y_true and y_predicted:
>>> y_predicted = model.predict(x_test)
>>> experiment.log_confusion_matrix(y_true, y_predicted)

# Or, if you have the categories for y_true or y_predicted
# you can just pass those in:
>>> experiment.log_confusion_matrix([0, 1, 2, 3],
                                    [2, 2, 2, 2]) # guess 2 for all

# Or, if you have already computed the matrix:
>>> experiment.log_confusion_matrix(labels=["one", "two", "three"],
                                    matrix=[[10, 0, 0],
                                            [ 0, 9, 1],
                                            [ 1, 1, 8],])

# However, if you want to reuse examples from previous runs,
# you can reuse a ConfusionMatrix instance. You might want to
# do this if you are creating a series of confusion matrices
# during the training of a model.
# See https://staging.comet.com/docs/quick-start/ for a tutorial.

>>> cm = ConfusionMatrix()
>>> y_predicted = model.predict(x_test)
>>> cm.compute_matrix(y_true, y_predicted)
>>> experiment.log_confusion_matrix(matrix=cm)

# Log again, using previously cached values:
>>> y_predicted = model.predict(x_test)
>>> cm.compute_matrix(y_true, y_predicted)
>>> experiment.log_confusion_matrix(matrix=cm)

For more details and example uses, please see:

Log Confusion Matrix

or:

comet_ml.Experiment.log_confusion_matrix

column_label = column_label

experiment = experiment

images = None

index_to_example_function = index_to_example_function

labels = labels

max_categories = max_categories

max_examples_per_cell = max_examples_per_cell

row_label = row_label

selected = sorted(selected) if selected is not None else None

title = title

use_cache = cache

winner_function = winner_function

clear()

clear_cache()

compute_matrix(
    y_true, y_predicted, index_to_example_function=None, images=None, **kwargs
)

default_index_to_example_function(index, **kwargs)

# Example index_to_example_function
def index_to_example_function(index):
    # x_test is user's inputs (just an example):
    image_array = x_test[index]
    # Make an asset name:
    image_name = "confusion-matrix-%05d.png" % index
    # Make an asset:
    results = experiment.log_image(
        image_array, name=image_name, image_shape=(28, 28, 1)
    )
    # Return the example name and assetId
    return {"sample": image_name, "assetId": results["imageId"]}

# Then, pass it to ConfusionMatrix(), or log_confusion_matrix()

default_winner_function(ndarray)

display(space=4)

image_index_to_example_function(
    index: int, **kwargs: Dict[str, Any]
) -> Optional[Dict[str, Any]]

initialize()

need_init()

to_json()