-
Notifications
You must be signed in to change notification settings - Fork 3.5k
Introduce new C++ API for C interfaces #25762
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR introduces comprehensive C++ wrapper interfaces for ONNX Runtime's C APIs, providing exception-safe alternatives to the raw C API functions. The changes focus on creating modern C++ classes and methods that handle resource management automatically through RAII patterns.
Key changes include:
- Introduction of new C++ wrapper classes for provider options (CUDAProviderOptions, TensorRTProviderOptions)
- Addition of memory management and allocator interfaces
- Implementation of tensor copy and stream management APIs
- Enhanced session and environment interfaces with new C++ methods
Reviewed Changes
Copilot reviewed 22 out of 22 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
onnxruntime/test/util/include/api_asserts.h |
Updates test assertion macros to use new C++ Status class instead of raw C API |
onnxruntime/test/shared_lib/test_session_options.cc |
Replaces C API CUDA provider setup with new C++ CUDAProviderOptions wrapper |
onnxruntime/test/shared_lib/test_model_loading.cc |
Migrates CUDA provider initialization to use C++ wrapper class |
onnxruntime/test/shared_lib/test_inference.cc |
Extensive refactoring to use C++ provider options and session interfaces |
onnxruntime/test/shared_lib/test_data_copy.cc |
Updates data transfer tests to use new C++ tensor copy and memory APIs |
onnxruntime/test/shared_lib/test_allocator.cc |
Converts allocator tests to use C++ allocator management interfaces |
| Multiple provider test files | Similar migrations from C API to C++ wrapper patterns |
include/onnxruntime/core/session/onnxruntime_cxx_api.h |
Adds new C++ class definitions and method declarations |
include/onnxruntime/core/session/onnxruntime_cxx_inline.h |
Implements the inline methods for the new C++ interfaces |
| Core API files | Updates parameter annotations and minor interface corrections |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
…/onnxruntime into yuslepukhin/cxx_ep_api
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You can commit the suggested changes from lintrunner.
### Description <!-- Describe your changes. --> This PR provides C++ interfaces for the following: Env ==== CopyTensors() CreateSharedAllocator GetSharedAllocator ReleaseSharedAllocator CreateAndRegisterAllocatorV2 RegisterAllocator UnregisterAllocator EpDevice ====== EpDevice_MemoryInfo CreateSyncStreamForEpDevice MemoryInfo ======== CreateMemoryInfo_V2 MemoryInfoGetName MemoryInfoGetId MemoryInfoGetMemType MemoryInfoGetType MemoryInfoGetDeviceMemType MemoryInfoGetVendorId Session ========== SessionGetInputName SessionGetOutputName SessionGetMemoryInfoForInputs SessionGetMemoryInfoForOutputs SessionGetEpDeviceForInputs SyncStream =========== SyncStream_GetHandle ReleaseSyncStream OrtArenaCfg =========== CreateArenaCfgV2 TRT === CreateTensorRTProviderOptions and V2 UpdateTensorRTProviderOptions SessionOptions ============== OrtSessionOptionsAppendExecutionProvider_CPU Prepacked container ============= CUDA Options V2 =========== OrtCUDAProviderOptionsV2 CreateCUDAProviderOptions GetCUDAProviderOptionsByName UpdateCUDAProviderOptionsWithValue UpdateCUDAProviderOptions GetCUDAProviderOptionsAsString ### Motivation and Context <!-- - Why is this change required? What problem does it solve? - If it fixes an open issue, please link to the issue here. --> Provide a way to write exception safe code.
### Description Cherry-pick the following PRs into the `rel-1.23.0` branch: - #25592 - #25622 - #25688 - #25729 - #25743 - #25769 - #25745 - #25761 - #25751 - #25716 - #25228 - #25768 - #25788 - #25747 - #25800 - #25818 - #25762 - #25749 - #25831 ### Motivation and Context <!-- - Why is this change required? What problem does it solve? - If it fixes an open issue, please link to the issue here. --> --------- Co-authored-by: quic-tirupath <[email protected]> Co-authored-by: quic-calvnguy <[email protected]> Co-authored-by: qti-kromero <[email protected]> Co-authored-by: Jeff Kilpatrick <[email protected]> Co-authored-by: Scott McKay <[email protected]> Co-authored-by: David Fan <[email protected]> Co-authored-by: kuanyul-qti <[email protected]> Co-authored-by: Dmitri Smirnov <[email protected]> Co-authored-by: Chi Lo <[email protected]> Co-authored-by: Edward Chen <[email protected]> Co-authored-by: Chunye Wang@AMD <[email protected]> Co-authored-by: minfhong-qti <[email protected]> Co-authored-by: Vishal Agarwal <[email protected]> Co-authored-by: Maximilian Müller <[email protected]> Co-authored-by: Maximilian Müller <[email protected]> Co-authored-by: Changming Sun <[email protected]> Co-authored-by: adrastogi <[email protected]> Co-authored-by: Aditya Rastogi <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
### Description <!-- Describe your changes. --> This PR provides C++ interfaces for the following: Env ==== CopyTensors() CreateSharedAllocator GetSharedAllocator ReleaseSharedAllocator CreateAndRegisterAllocatorV2 RegisterAllocator UnregisterAllocator EpDevice ====== EpDevice_MemoryInfo CreateSyncStreamForEpDevice MemoryInfo ======== CreateMemoryInfo_V2 MemoryInfoGetName MemoryInfoGetId MemoryInfoGetMemType MemoryInfoGetType MemoryInfoGetDeviceMemType MemoryInfoGetVendorId Session ========== SessionGetInputName SessionGetOutputName SessionGetMemoryInfoForInputs SessionGetMemoryInfoForOutputs SessionGetEpDeviceForInputs SyncStream =========== SyncStream_GetHandle ReleaseSyncStream OrtArenaCfg =========== CreateArenaCfgV2 TRT === CreateTensorRTProviderOptions and V2 UpdateTensorRTProviderOptions SessionOptions ============== OrtSessionOptionsAppendExecutionProvider_CPU Prepacked container ============= CUDA Options V2 =========== OrtCUDAProviderOptionsV2 CreateCUDAProviderOptions GetCUDAProviderOptionsByName UpdateCUDAProviderOptionsWithValue UpdateCUDAProviderOptions GetCUDAProviderOptionsAsString ### Motivation and Context <!-- - Why is this change required? What problem does it solve? - If it fixes an open issue, please link to the issue here. --> Provide a way to write exception safe code.
Description
This PR provides C++ interfaces for the following:
Env
CopyTensors()
CreateSharedAllocator
GetSharedAllocator
ReleaseSharedAllocator
CreateAndRegisterAllocatorV2
RegisterAllocator
UnregisterAllocator
EpDevice
EpDevice_MemoryInfo
CreateSyncStreamForEpDevice
MemoryInfo
CreateMemoryInfo_V2
MemoryInfoGetName
MemoryInfoGetId
MemoryInfoGetMemType
MemoryInfoGetType
MemoryInfoGetDeviceMemType
MemoryInfoGetVendorId
Session
SessionGetInputName
SessionGetOutputName
SessionGetMemoryInfoForInputs
SessionGetMemoryInfoForOutputs
SessionGetEpDeviceForInputs
SyncStream
SyncStream_GetHandle
ReleaseSyncStream
OrtArenaCfg
CreateArenaCfgV2
TRT
CreateTensorRTProviderOptions and V2
UpdateTensorRTProviderOptions
SessionOptions
OrtSessionOptionsAppendExecutionProvider_CPU
Prepacked container
CUDA Options V2
OrtCUDAProviderOptionsV2
CreateCUDAProviderOptions
GetCUDAProviderOptionsByName
UpdateCUDAProviderOptionsWithValue
UpdateCUDAProviderOptions
GetCUDAProviderOptionsAsString
Motivation and Context
Provide a way to write exception safe code.