bacpipe.embedding_evaluation.benchmark

Functions

benchmark(model, dataset[, ...])	Benchmark a model's classifier performance for a dataset.
classification_report(y_true, y_pred, *[, ...])	Build a text report showing the main classification metrics.

bacpipe.embedding_evaluation.benchmark.benchmark(model, dataset, annotations_file=None, CustomModel=None, check_if_already_processed=True, **kwargs)

Benchmark a model’s classifier performance for a dataset. The dataset requires an annotation file that is located in the root directory of the dataset. This annotation file has needs to have the column names: start, end, audiofilename, label:species so that the ground truth can be extracted. Ground truth is mapped to the timestamps so that predictions and ground_truth have the same shape. If predictions have already been produced this function runs very quickly as it uses the saved data.

Finally the sklearn.metrics.classification_report function is used to quantify the performance. The results are printed as a report and returned as a dictionary. This function expects a threshold. Threshold-independent performance evaluation is currently not supported.

Parameters:

• model (string) – model name

• dataset (string) – path to audio dataset

• annotations_file (string, optional) – file name of annotations, by default None

• CustomModel (class, optional) – Custom model to use for the predictions, by default None

• check_if_already_processed (bool, optional) – if you want to force embeddings to be generated again, set to True, defaults to True

Returns:

dictionary containing report results, ground truth array, predictions array, index to label dict and a list of the species that weren’t found in the classifier class list

Return type:

dict