IO
datamate.io ¶
Module for handling file I/O operations for Directory objects.
This module provides functionality for reading, writing, and manipulating HDF5 files and directories.
ArrayFile ¶
Bases: Protocol
Protocol for single-array HDF5 file interface.
Attributes:
Name | Type | Description |
---|---|---|
path |
Path
|
Path to the HDF5 file. |
shape |
Tuple[int, ...]
|
Shape of the array data. |
dtype |
dtype
|
NumPy dtype of the array data. |
Source code in datamate/io.py
41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 |
|
H5Reader ¶
Bases: ArrayFile
Wrapper around HDF5 read operations with safe file handle management.
Ensures file handles are only open during access operations to prevent resource leaks.
Attributes:
Name | Type | Description |
---|---|---|
path |
Path to the HDF5 file. |
|
shape |
Shape of the array data. |
|
dtype |
NumPy dtype of the array data. |
|
n_retries |
Number of retry attempts for file operations. |
Source code in datamate/io.py
63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 |
|
directory_to_df ¶
directory_to_df(directory, dtypes=None)
Convert a directory to a pandas DataFrame.
Creates a DataFrame from HDF5 datasets in the directory. Single-element datasets are broadcast to match the most common length.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
directory
|
Directory
|
Directory object containing HDF5 datasets. |
required |
dtypes
|
Optional[Dict[str, dtype]]
|
Optional mapping of column names to NumPy dtypes. |
None
|
Returns:
Type | Description |
---|---|
DataFrame
|
DataFrame containing the directory data. |
Example
dir = Directory("path/to/dir")
df = directory_to_df(dir, {"col1": np.float32})
Source code in datamate/io.py
296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 |
|