bacpipe.core package

Submodules

bacpipe.core.audio_processor module

class bacpipe.core.audio_processor.AudioHandler(model, padding, audio_dir, bool_slowdown=False, slowdown_rate=None, **kwargs)

Bases: object

Helper class for all methods related to loading and padding audio.

__init__(model, padding, audio_dir, bool_slowdown=False, slowdown_rate=None, **kwargs)

Helper class for all methods related to loading and padding audio.

Parameters:

• model (Model object) – has attributes for all the model characteristics like sample rate, segment length etc. as well as the methods to run the model

• padding (str) – padding function to use for where padding is necessary

• audio_dir (pathlib.Path object) – path to audio dir

prepare_audio(sample)

Use bacpipe pipeline to load audio file, window it according to model specific window length and preprocess the data, ready for batch inference computation. Also log file length and shape for metadata files.

Parameters:

sample (pathlib.Path or str) – path to audio file

Returns:

audio frames preprocessed with model specific preprocessing

Return type:

torch.Tensor

bacpipe.core.constants module

bacpipe.core.constants.EMBEDDING_DIMENSIONS = {'audiomae': 768, 'audioprotopnet': 1024, 'aves_especies': 768, 'avesecho_passt': 768, 'bat': 64, 'batdetect2_clip_avg': 32, 'batdetect2_dets_avg': 32, 'beats': 768, 'biolingual': 512, 'birdaves_especies': 1024, 'birdmae': 1280, 'birdnet': 1024, 'convnext_birdset': 1024, 'google_whale': 1280, 'hbdet': 2048, 'insect459': 1280, 'insect66': 1280, 'mix2': 960, 'naturebeats': 768, 'perch_bird': 1280, 'perch_v2': 1536, 'protoclr': 384, 'rcl_fs_bsed': 2048, 'surfperch': 1280, 'vggish': 128}

model names with their corresponding embedding size

Type:

dict

bacpipe.core.constants.NEEDS_CHECKPOINT = ['audiomae', 'avesecho_passt', 'aves_especies', 'bat', 'beats', 'birdaves_especies', 'birdnet', 'hbdet', 'insect66', 'insect459', 'mix2', 'naturebeats', 'protoclr', 'rcl_fs_bsed']

models that require a checkpoint to be downloaded to be run

Type:

list[str]

bacpipe.core.constants.TF_MODELS = ['birdnet', 'perch_v2', 'perch_bird', 'google_whale', 'surfperch', 'vggish', 'hbdet']

models that were built using tensorflow

Type:

list[str]

bacpipe.core.constants.models_needing_checkpoint = ['audiomae', 'avesecho_passt', 'aves_especies', 'bat', 'beats', 'birdaves_especies', 'birdnet', 'hbdet', 'insect66', 'insect459', 'mix2', 'naturebeats', 'protoclr', 'rcl_fs_bsed']

Models that require a checkpoint to be downloaded before use.

Type:

list[str]

bacpipe.core.constants.supported_models = ['audiomae', 'audioprotopnet', 'avesecho_passt', 'aves_especies', 'bat', 'batdetect2_clip_avg', 'batdetect2_dets_avg', 'beats', 'birdaves_especies', 'biolingual', 'birdnet', 'birdmae', 'convnext_birdset', 'hbdet', 'insect66', 'insect459', 'mix2', 'naturebeats', 'perch_bird', 'perch_v2', 'protoclr', 'rcl_fs_bsed', 'surfperch', 'google_whale', 'vggish']

Supported embedding models available in bacpipe.

Type:

list[str]

bacpipe.core.experiment_manager module

class bacpipe.core.experiment_manager.Loader(audio_dir, model_name=None, check_if_combination_exists=True, dim_reduction_model=False, use_folder_structure=False, testing=False, **kwargs)

Bases: object

Initiate the generation of embedding by creating a Loader object. This object will handles paths for loading and saving data. During this process it collects metadata which can be accessed as an attribute and will be saved after the successful run. kwargs that are not specifically passed will be taken from bacpipe.config and bacpipe.settings.

__init__(audio_dir, model_name=None, check_if_combination_exists=True, dim_reduction_model=False, use_folder_structure=False, testing=False, **kwargs)

Initiate the generation of embedding by creating a Loader object. This object will handles paths for loading and saving data. During this process it collects metadata which can be accessed as an attribute and will be saved after the successful run. kwargs that are not specifically passed will be taken from bacpipe.config and bacpipe.settings.

Parameters:

• audio_dir (string or pathlib.Path) – path to audio data

• model_name (string, optional) – Name of the model that should be used, by default None

• check_if_combination_exists (bool, optional) – If false new embeddings are created and the checking is skipped, by default True

• dim_reduction_model (bool, optional) – Either false if primary embeddings are created or the name of the dimensionaliry reduction model if dim reduction should be performed, by default False

• use_folder_structure (bool, optional) – If True data will be saved and the output folder structure will be created, by default False

• testing (bool, optional) – Testing yes or no?, by default False

classifier_should_be_run(paths, run_pretrained_classifier, testing, **kwargs)

embeddings(return_type='dict')

Load and return processed embeddings. This method can only be used to return already computed embeddings. Embeddings can be returned as np.array (array) or as dictionary (dict) in which case the keys will correspond to the corresponding embedding file name. In case of the array, all embeddings are concatenated so that the first dimension corresponds to the timestamp and the second dimension to the embedding dimension.

Parameters:

return_type (str, optional) – return type either array or dict, by default ‘dict’

Returns:

depending on return_type argument

Return type:

array or dict

get_annotations_parquet(**kwargs)

static get_audio_files(audio_dir, audio_suffixes=['.wav', '.WAV', '.aif', '.mp3', '.MP3', '.flac', '.ogg'], return_type='pathlib.Path')

Collect all audio files in a given directory that have file endings that can be processed by bacpipe.

Parameters:

• audio_dir (str) – path to audio data

• audio_suffixes (list, optional) – list of audio suffixes, by default settings.audio_suffixes

• return_type (str, optional) – specify if list should be returned as list of strings or list of pathlib.Path objects which comes in handy for some downstream processing, by default ‘pathlib.Path’

Returns:

list of audio files

Return type:

list

get_embedding_dir()

get_preds_array(return_type='dict', **kwargs)

predictions(return_type='dict')

Load and return classifier predictions. This method can only be used for already processed predictions. Predictions that have been processed will be returned based on the specified return_type: array for np.array, in which case all predictions are concatenated and a dictionary is passed referencing the index to the corresponding label. dict for a dictionary, in which case the keys correspond to the audio file name corresponding to the annotation and the values are np.arrays with all annotations of that file dataframe for a dataframe with columns for each species that was active and columns for filename, start and end times.

Parameters:

return_type (str, optional) – return either array, dict or dataframe, by default ‘dict’

Returns:

either tuples of (np.array, dict) for array or tuple of (dict, dict) for dict or pd.DataFrame

Return type:

tuple or pd.DataFrame

read_embedding_file(file)

save_embedding_file(file, embeds)

update_files()

write_metadata_file()

bacpipe.core.experiment_manager.replace_default_kwargs_with_user_kwargs(remove_keys=None, **kwargs)

bacpipe.core.experiment_manager.save_logs()

bacpipe.core.workflows module

bacpipe.core.workflows.cross_model_evaluation(dim_reduction_model, evaluation_task, models, **kwargs)

Generate plots to compare models by the specified tasks.

Parameters:

• dim_reduction_model (str) – name of dimensionality reduction model

• evaluation_task (list) – tasks to evaluate models by

• models (list) – embedding models

bacpipe.core.workflows.ensure_models_exist(model_base_path, model_names, repo_id='vskode/bacpipe_models')

Ensure that the model checkpoints for the selected models are available locally. Downloads from Hugging Face Hub if missing.

Parameters:

• model_base_path (Path) – Local base directory where the checkpoints should be stored.

• model_names (str or list) – Model name or list of model names to run

• repo_id (str, optional) – Hugging Face Hub repo ID, by default “vinikay/bacpipe_models”

Returns:

path to saved models

Return type:

str

bacpipe.core.workflows.evaluation_with_settings_already_exists(audio_dir, dim_reduction_model, models, testing=False, **kwargs)

Check if the evaluation with the specified settings already exists. The function checks if the embeddings, dimensionality reduction, probing and clustering evaluation results already exist in the specified directory. If any of these results do not exist, the function returns False. Otherwise, it returns True.

Parameters:

• audio_dir (string) – full path to audio files

• dim_reduction_model (string) – name of the dimensionality reduction model to be used

• models (list) – embedding models

Returns:

True if the evaluation with the specified settings

Return type:

bool

bacpipe.core.workflows.generate_embeddings(avoid_pipelined_gpu_inference=False, **kwargs)

Run the embedding generation pipeline including classification using the pretrained classifier (if included). All of this will be done for one model. The predefined folder structure will be created so that subsequent processing runs will be very fast, as they then only load the data. kwargs that are not specifically passed will be taken from bacpipe.config and bacpipe.settings.

Parameters:

avoid_pipelined_gpu_inference (bool, optional) – set to True to avoid multiprocessing, by default False

Returns:

loader object to access embeddings and classifier predictions

Return type:

bacpipe.Loader

Raises:

ValueError – if not model name is provided

bacpipe.core.workflows.get_model_names(models, audio_dir, main_results_dir, embed_parent_dir, already_computed=False, **kwargs)

Get the names of the models used for processing. This is either done by using already computed embeddings or by using the selected models from the config file. If already computed embeddings are used, the model names are extracted from the directory structure.

Parameters:

• models (list) – list of embedding models

• audio_dir (string) – full path to audio files

• main_results_dir (string) – top level directory for the results of the embedding evaluation

• embed_parent_dir (string) – parent directory for the embeddings

• already_computed (bool, Default is False) – ignore model list and use only models whos embeddings already have been computed and are saved in the results dir

Raises:

ValueError – If already computed embeddings are used, but no embeddings are found in the specified directory.

bacpipe.core.workflows.model_specific_evaluation(loader_dict, evaluation_task, probe_configs, models, dim_reduction_model=False, **kwargs)

Perform evaluation of the embeddings using the specified evaluation task. The evaluation task can be either probing or clustering. The evaluation is performed using the functions from the probing and clustering modules. The results of the evaluation are saved in the directory specified by the audio_dir parameter.

Parameters:

• loader_dict (dict) – dictionary containing the loader objects for each model

• evaluation_task (string) – name of the evaluation task to be performed.

• probe_configs (dict) – dictionary containing the configuration for the probing tasks. The configurations are specified in the bacpipe/settings.yaml file.

• models (list) – embedding models

bacpipe.core.workflows.play(bool_save_logs=False, **kwargs)

Play the bacpipe! The pipeline will run using the models specified in bacpipe.config.models and generate results in the directory bacpipe.settings.results_dir. For more details see the ReadMe file on the repository page https://github.com/bioacoustic-ai/bacpipe or the documentation under https://bacpipe.readthedocs.io/en/latest/.

Parameters:

bool_save_logs (bool, optional) – Save logs, config and settings file. This is important if you get a bug, sharing this will be very helpful to find the source of the problem, by default False

Raises:

FileNotFoundError – If no audio files are found we can’t compute any embeddings. So make sure the path is correct :)

bacpipe.core.workflows.run_pipeline_for_models(models, audio_dir, dim_reduction_model, **kwargs)

Generate embeddings for each model in the list of model names. The embeddings are generated using the generate_embeddings function from the generate_embeddings module. The embeddings are saved in the directory specified by the audio_dir parameter. The function returns a dictionary containing the loader objects for each model, by which metadata and paths are stored. kwargs that are not specifically passed will be taken from bacpipe.config and bacpipe.settings.

code example: ``` loader = bacpipe.run_pipeline_for_models(

models=[‘birdnet’, ‘naturebeats’], audio_dir=’bacpipe/tests/test_data’, dim_reduction_model=’umap’

)

# this call will initiate the embedding generation process, it will check if embeddings # already exist for the combination of each model and the dataset and if so it will # be ready to load them. The loader keys will be the model name and the values will # be the loader objects for each model. Each object contains all the information # on the generated embeddings. To name access them: loader[‘birdnet’].embeddings() # this will give you a dictionary with the keys corresponding to embedding files # and the values corresponding to the embeddings as numpy arrays

loader[‘birdnet’].metadata_dict # This will give you a dictionary overview of: # - where the audio data came from, # - where the embeddings were saved # - all the audio files, # - the embedding size of the model, # - the audio file lengths, # - the number of embeddings for each audio files # - the sample rate # - the number of samples per window # - and the total length of the processed dataset in seconds # Thic dictionary is also saved as a yaml file in the directory of the embeddings ```

Parameters:

• models (list) – embedding models

• audio_dir (string) – full path to audio files

• dim_reduction_model (string) – name of the dimensionality reduction model to be used for the embeddings. If “None” is selected, no dimensionality reduction is performed.

Returns:

loader_dict – dictionary containing the loader objects for each model

Return type:

dict

bacpipe.core.workflows.run_pipeline_for_single_model(model_name, audio_dir, dim_reduction_model='None', check_if_already_processed=True, check_if_already_dim_reduced=True, testing=False, **kwargs)

Run the bacpipe pipeline, including embedding generation, classification using the pretrained classifier (if included), dimensionality reduction (if passed), and plotting of visualization to files. All of this will be done for one model. The predefined folder structure will be created so that subsequent processing runs will be very fast, as they then only load the data. kwargs that are not specifically passed will be taken from bacpipe.config and bacpipe.settings.

Parameters:

• model_name (string) – model name

• audio_dir (str) – path to audio data

• dim_reduction_model (str, optional) – name of dimensionality reduction model, by default “None”

• check_if_already_processed (bool, optional) – set to False if you want to force recomputing of embeddings, by default True

• check_if_already_dim_reduced (bool, optional) – set to False if you want to force recomputing of dimensionality reduced embeddings, by default True

• overwrite (bool, optional) – set to True if you want default labels and ground truth labels to be processed again, by default False

• testing (bool, optional) – set to True for testing, by default False

Returns:

object to processed embeddings and classifier predictions

Return type:

bacpipe.Loader

Module contents