Skip to content

Conversation

@ishwar-raut1
Copy link
Contributor

Description

The EP will reject the node with unsupported data types.

Motivation and Context

The user will face a crash if the model with an unsupported datatype is used.

@ishwar-raut1
Copy link
Contributor Author

@chilo-ms @jywu-msft to review

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds data type validation to the NV TensorRT RTX execution provider to prevent crashes when models contain unsupported data types. The EP will now gracefully reject nodes with unsupported data types instead of crashing.

  • Introduces data type checking functionality with supported type validation
  • Updates tensor binding operations to handle INT4 and FLOAT8E4M3FN data types
  • Integrates data type validation into the node capability assessment process

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@chilo-ms
Copy link
Contributor

chilo-ms commented Sep 4, 2025

/azp run Linux QNN CI Pipeline,Win_TRT_Minimal_CUDA_Test_CI,Windows ARM64 QNN CI Pipeline,Windows GPU Doc Gen CI Pipeline,Windows x64 QNN CI Pipeline

@azure-pipelines
Copy link

Azure Pipelines successfully started running 5 pipeline(s).

@jywu-msft
Copy link
Member

jywu-msft commented Sep 4, 2025

in the interest of time, i'm going to merge this and we can follow up on comments later.
I believe TRT supports int4 for weights only and that is why the casting to uint8 is done for I/O

@jywu-msft jywu-msft merged commit b49e69e into microsoft:main Sep 4, 2025
86 checks passed
tianleiwu pushed a commit that referenced this pull request Sep 5, 2025
### Description
<!-- Describe your changes. -->
The EP will reject the node with unsupported data types. 


### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->

The user will face a crash if the model with an unsupported datatype is
used.
@snnn
Copy link
Member

snnn commented Sep 5, 2025

This PR has been cherry-picked into the rel-1.23.0 branch in PR #25959. Removing the release:1.23.0 label.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants