batch_load_hdf5_data
- ketos.neural_networks.dev_utils.detection.batch_load_hdf5_data(hdf5_file_path, batch_size, audio_representation, table_name=None, logger=None)[source]
Loads HDF5 data in batches from a specified file and yields it for processing.
This function assumes that the HDF5 dataset is named ‘data’. For each batch, it will yield a dictionary containing the batch’s data, filename, start and end offsets.
The ‘data’ key in the yielded dictionary corresponds to the batch’s data.
The ‘filename’ key in the yielded dictionary corresponds to the batch’s filename.
The ‘start’ key in the yielded dictionary corresponds to the start offset of the batch.
The ‘end’ key in the yielded dictionary corresponds to the end offset of the batch.
- Args:
- hdf5_file_pathstr
The file path of the HDF5 file to read from.
- batch_sizeint
The number of samples to include in each batch.
- audio_representationdict
A dictionary containing information about the audio representation.
- table_namestr
The name of the table in the HDF5 file to read the data from. If no name is provided, it defaults to ‘/data’.
- logger: logging.Logger or KetosLogger
A Logger instance to log errors encountered while loading audio file data.
- Returns:
- dict
A dictionary containing the batch data, its filename, start and end offsets. The ‘end’ key value is calculated based on the ‘duration’ key of the audio_representation parameter and may not be accurate, as the duration might change when actually computing the representation.