API Reference

class bacpipe.Embedder(model_name, loader=None, CustomModel=None, dim_reduction_model=False, **kwargs)

Bases: AudioHandler

This class takes care of loading the specified model and using it to process the audio data to create embeddings. This class is also used to create dimensinoality reductions from embeddings. At the end if instantiation, the selected model is loaded and the model is associated with the specified device. kwargs that are not specifically passed will be taken from bacpipe.config and bacpipe.settings.

Parameters:

AudioHandler (class) – Helper class that handles loading of audio

__init__(model_name, loader=None, CustomModel=None, dim_reduction_model=False, **kwargs)

This class takes care of loading the specified model and using it to process the audio data to create embeddings. This class is also used to create dimensinoality reductions from embeddings. At the end if instantiation, the selected model is loaded and the model is associated with the specified device. kwargs that are not specifically passed will be taken from bacpipe.config and bacpipe.settings.

Parameters:

• model_name (str) – name of selected embedding model

• loader (Loader object) – Object that has all the necessary path information and methods to load and save all the processed data

• CustomModel (class, optional) – custom model class to use for processing, by default None

• dim_reduction_model (bool, optional) – Can be bool or the string corresponding to the dimensionality reduction model, by default False

batch_inference(batched_samples, callback=None)

embeddings_using_multithreading(array_of_audios)

Generate embeddings for all files in a pipelined manner: - Producer thread loads and preprocesses audio - Consumer (main thread) embeds audio while producer prepares next batch Ensures metadata and embeddings are written exactly like in the sequential version.

Parameters:

fileloader_obj (Loader object) – contains all metadata of a model specific embedding creation session

Returns:

updated object with metadata on embedding creation session

Return type:

Loader object

get_embeddings_for_audio(sample)

Create a dataloader for the processed audio frames and run batch inference. Both are methods of the self.model class, which can be found in the utils.py file.

Parameters:

sample (torch.Tensor) – preprocessed audio frames

Returns:

embeddings from model

Return type:

np.array

get_embeddings_from_model(sample)

Run full embedding generation pipeline, both for generating embeddings from audio data or generating dimensionality reductions from embedding data. Depending on that sample can be an embedding array or a audio file path.

Parameters:

sample (np.array or string-like) – embedding array of path to audio file

Returns:

embeddings

Return type:

np.array

get_reduced_dimensionality_embeddings(embeds)

init_dataloader(audio)

run_dimensionality_reduction_pipeline()

run_inference_pipeline_sequentially()

run_inference_pipeline_using_multithreading()

Generate embeddings for all files in a pipelined manner: - Producer thread loads and preprocesses audio - Consumer (main thread) embeds audio while producer prepares next batch Ensures metadata and embeddings are written exactly like in the sequential version.

Parameters:

fileloader_obj (Loader object) – contains all metadata of a model specific embedding creation session

Returns:

updated object with metadata on embedding creation session

Return type:

Loader object

bacpipe.play(bool_save_logs=False, **kwargs)

Play the bacpipe! The pipeline will run using the models specified in bacpipe.config.models and generate results in the directory bacpipe.settings.results_dir. For more details see the ReadMe file on the repository page https://github.com/bioacoustic-ai/bacpipe or the documentation under https://bacpipe.readthedocs.io/en/latest/.

Parameters:

bool_save_logs (bool, optional) – Save logs, config and settings file. This is important if you get a bug, sharing this will be very helpful to find the source of the problem, by default False

Raises:

FileNotFoundError – If no audio files are found we can’t compute any embeddings. So make sure the path is correct :)

bacpipe.supported_models = ['audiomae', 'audioprotopnet', 'avesecho_passt', 'aves_especies', 'bat', 'batdetect2_clip_avg', 'batdetect2_dets_avg', 'beats', 'birdaves_especies', 'biolingual', 'birdnet', 'birdmae', 'convnext_birdset', 'hbdet', 'insect66', 'insect459', 'mix2', 'naturebeats', 'perch_bird', 'perch_v2', 'protoclr', 'rcl_fs_bsed', 'surfperch', 'google_whale', 'vggish']

Built-in mutable sequence.

If no argument is given, the constructor creates a new empty list. The argument must be an iterable if specified.

bacpipe.models_needing_checkpoint = ['audiomae', 'avesecho_passt', 'aves_especies', 'bat', 'beats', 'birdaves_especies', 'birdnet', 'hbdet', 'insect66', 'insect459', 'mix2', 'naturebeats', 'protoclr', 'rcl_fs_bsed']

Built-in mutable sequence.

If no argument is given, the constructor creates a new empty list. The argument must be an iterable if specified.

bacpipe.config = namespace(audio_dir='bacpipe/tests/test_data', overwrite=False, dashboard=True, models=['birdnet', 'perch_bird'], already_computed=False, dim_reduction_model='umap', evaluation_task=[])

A simple attribute-based namespace.

SimpleNamespace(**kwargs)

bacpipe.settings = namespace(main_results_dir='bacpipe_results', embed_parent_dir='embeddings', dim_reduc_parent_dir='dim_reduced_embeddings', evaluations_dir='evaluations', model_base_path='bacpipe/model_checkpoints', device='cpu', global_batch_size=8, audio_suffixes=['.wav', '.WAV', '.aif', '.mp3', '.MP3', '.flac', '.ogg'], padding='wrap', avoid_pipelined_gpu_inference=False, nr_parallel_workers=False, rm_embedding_on_keyboard_interrupt=True, label_column='species', annotations_filename='annotations.csv', only_embed_annotations=False, default_label_keys=['time_of_day', 'day_of_year', 'continuous_timestamp', 'parent_directory', 'audio_file_name'], min_label_occurrences=50, bool_filter_labels=False, run_pretrained_classifier=True, classifier_threshold=0.5, save_raven_tables=True, train_ratio=0.65, test_ratio=0.2, probe_configs={'config_1': {'bool': True, 'name': 'linear', 'learning_rate': 0.0005, 'batch_size': 64, 'num_epochs': 20, 'dataset_csv_path': 'probing_dataframe.csv', 'shuffle': True}, 'config_2': {'bool': True, 'name': 'knn', 'n_neighbors': 15, 'dataset_csv_path': 'probing_dataframe.csv'}}, save_probe=True, evaluate_with_silhouette=False, clust_configs={'config_1': {'bool': True, 'name': 'kmeans', 'params': {'n_clusters': 18}}, 'config_2': {'bool': False, 'name': 'hdbscan', 'params': {'min_cluster_size': 10, 'min_samples': 5, 'metric': 'euclidean'}}}, dashboard_port=5006, dashboard_address='localhost', dashboard_websocket_origin=False, embed_fig_height=700, spectrogram_plot_height=550, heatmap_fig_height=600, accordion_width=780, spec_colorscale='Viridis', color_continuous='Twilight', bool_change_speed=False, new_speed=0.1, testing=False)

A simple attribute-based namespace.

SimpleNamespace(**kwargs)