exporters - #4672
Conversation
Drop the Test/trt import path so PI05, GR00T, and Nemotron export is self-contained, and rename GR00T action_context to context_projection.
Each family now has spec, helpers, and patches in one folder, with shared vision/language/action-step code in models/common.
github-actions
Bot
added
component: tests
narendasan
left a comment
narendasan
left a comment
There was a problem hiding this comment.
This is cool, for sure much more clean. Do you have like a guide on how to bring up your own model?
There was a problem hiding this comment.
You can just put the example of the exporter tool with the tool unless you want them rendered. Then we need to use the sphinx gallery format
There was a problem hiding this comment.
Do you have the inference side of these examples? (Ideally both python and C++ deployments)
There was a problem hiding this comment.
Can we start this out of tree for now like //tools/hf_exporters?
| num_kv = int(cfg.num_key_value_heads) | ||
| head_dim = int(getattr(cfg, "head_dim", cfg.hidden_size // cfg.num_attention_heads)) | ||
| try: | ||
| from torch_tensorrt.hf.exporters.rope import make_rope_rotary_cos_sin |
There was a problem hiding this comment.
Why do we need to guard around code we ship?
|
|
||
|
|
||
| @dynamo_tensorrt_converter( | ||
| torch.ops.trt.attention_plugin.default, |
There was a problem hiding this comment.
make this namespace torch.ops.tensorrt_edge_llm.*
|
|
||
|
|
||
| @dynamo_tensorrt_converter( | ||
| torch.ops.trt.vit_attention_plugin.default, |
|
|
||
|
|
||
| @dynamo_tensorrt_converter( | ||
| torch.ops.trt.causal_conv1d.default, |
|
|
||
|
|
||
| @dynamo_tensorrt_converter( | ||
| torch.ops.trt.update_ssm_state.default, |
|
|
||
|
|
||
| @dynamo_tensorrt_converter( | ||
| torch.ops.trt.nvfp4_moe_plugin.default, |
|
|
||
| import torch | ||
|
|
||
| _THOR_CUDA_LIB = Path("/usr/local/cuda-13.0/thor/targets/aarch64-linux/lib") |
There was a problem hiding this comment.
We should have some more robust discovery method
2 participants
Description
Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.
Fixes # (issue)
Type of change
Please delete options that are not relevant and/or add your own.
Checklist:
Stack created with GitHub Stacks CLI • Give Feedback 💬