python(feat): add MCAP imports - #755
Conversation
|
Python docs preview: https://sift-stack.github.io/sift/python/pr-755/ Deployed from |
cd5f9d4 to
3534540
Compare
| asset_name: The asset name to set on the config. | ||
|
|
||
| Returns: | ||
| A config whose ``data`` lists one channel per leaf field, with default |
There was a problem hiding this comment.
Maybe say "A config who data is a flattened list of fields...`?
| for topic in topics: | ||
| for leaf in topic.leaves: | ||
| name = f"{topic.topic}.{leaf.field_path}" | ||
| data_type = ChannelDataType.BYTES if leaf.kind == "complex" else leaf.sift_type() |
There was a problem hiding this comment.
I think we should default to ChannelDataType.BOTH since that's what we do elsewhere.
There was a problem hiding this comment.
the default is both: complex_types_import_mode: McapComplexTypesImportMode = McapComplexTypesImportMode.BOTH, BYTES here just flags it before expanding, updated docstring cbff58f
| record = next(records) | ||
| except StopIteration: | ||
| return | ||
| except Exception as e: |
There was a problem hiding this comment.
Is there a more specific Exception we can catch?
There was a problem hiding this comment.
the try only wraps the mcap library call, so every error here comes from the library (EndOfFile, struct.error, decompression and decode errors)
didn't list them explicitly since a library update could make the tuple stale, added a comment for this
| file.seek(0) | ||
| try: | ||
| summary = make_reader(file).get_summary() | ||
| except Exception: |
There was a problem hiding this comment.
Is there a more specific Exception we can catch?
There was a problem hiding this comment.
same as above, thetry only wraps the mcap call b0c641a
| return leaves | ||
|
|
||
|
|
||
| def _read_schemas_and_channels( |
There was a problem hiding this comment.
Pretty much everything in this module is private. But there's inconsistent usage of underscores. I think we can just get rid of them.
2 participants
Summary
Adds MCAP (
.mcap) import support tosift_clientfor ROS 2 topics (ros2msg/cdr)detect_configreads the file's channels locally without decoding messages, giving one entry per flattened field named<topic>.<field_path>. Edit it to select, rename, or retype channels; leavedataempty to import everything.Variable-cardinality fields are typed
BYTES, withcomplex_types_import_modedeciding what each becomes, matching Parquet. Undecodable topics are skipped with a warning unlessparse_error_policyisIGNORE_ERROR, which fails the import.Adds the MCAP config types,
DataTypeKey.MCAP, an example, and a newmcapextra.Testing
Unit tests covering schema flattening, type mapping, topic rules, the three scan paths, and each complex types mode.
Manual imports with MCAP files