Skip to content

Conversation

@yuslepukhin
Copy link
Member

Description

This PR provides C++ interfaces for the following:

Env

CopyTensors()

CreateSharedAllocator
GetSharedAllocator
ReleaseSharedAllocator
CreateAndRegisterAllocatorV2

RegisterAllocator
UnregisterAllocator

EpDevice

EpDevice_MemoryInfo
CreateSyncStreamForEpDevice

MemoryInfo

CreateMemoryInfo_V2
MemoryInfoGetName
MemoryInfoGetId
MemoryInfoGetMemType
MemoryInfoGetType
MemoryInfoGetDeviceMemType
MemoryInfoGetVendorId

Session

SessionGetInputName
SessionGetOutputName

SessionGetMemoryInfoForInputs
SessionGetMemoryInfoForOutputs
SessionGetEpDeviceForInputs

SyncStream

SyncStream_GetHandle
ReleaseSyncStream

OrtArenaCfg

CreateArenaCfgV2

TRT

CreateTensorRTProviderOptions and V2
UpdateTensorRTProviderOptions

SessionOptions

OrtSessionOptionsAppendExecutionProvider_CPU

Prepacked container

CUDA Options V2

OrtCUDAProviderOptionsV2
CreateCUDAProviderOptions

GetCUDAProviderOptionsByName
UpdateCUDAProviderOptionsWithValue
UpdateCUDAProviderOptions
GetCUDAProviderOptionsAsString

Motivation and Context

Provide a way to write exception safe code.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR introduces comprehensive C++ wrapper interfaces for ONNX Runtime's C APIs, providing exception-safe alternatives to the raw C API functions. The changes focus on creating modern C++ classes and methods that handle resource management automatically through RAII patterns.

Key changes include:

  • Introduction of new C++ wrapper classes for provider options (CUDAProviderOptions, TensorRTProviderOptions)
  • Addition of memory management and allocator interfaces
  • Implementation of tensor copy and stream management APIs
  • Enhanced session and environment interfaces with new C++ methods

Reviewed Changes

Copilot reviewed 22 out of 22 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
onnxruntime/test/util/include/api_asserts.h Updates test assertion macros to use new C++ Status class instead of raw C API
onnxruntime/test/shared_lib/test_session_options.cc Replaces C API CUDA provider setup with new C++ CUDAProviderOptions wrapper
onnxruntime/test/shared_lib/test_model_loading.cc Migrates CUDA provider initialization to use C++ wrapper class
onnxruntime/test/shared_lib/test_inference.cc Extensive refactoring to use C++ provider options and session interfaces
onnxruntime/test/shared_lib/test_data_copy.cc Updates data transfer tests to use new C++ tensor copy and memory APIs
onnxruntime/test/shared_lib/test_allocator.cc Converts allocator tests to use C++ allocator management interfaces
Multiple provider test files Similar migrations from C API to C++ wrapper patterns
include/onnxruntime/core/session/onnxruntime_cxx_api.h Adds new C++ class definitions and method declarations
include/onnxruntime/core/session/onnxruntime_cxx_inline.h Implements the inline methods for the new C++ interfaces
Core API files Updates parameter annotations and minor interface corrections

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can commit the suggested changes from lintrunner.

@yuslepukhin yuslepukhin merged commit e6d3e08 into main Aug 22, 2025
92 checks passed
@yuslepukhin yuslepukhin deleted the yuslepukhin/cxx_ep_api branch August 22, 2025 17:55
adrianlizarraga pushed a commit that referenced this pull request Aug 22, 2025
### Description
<!-- Describe your changes. -->
This PR provides C++ interfaces for the following:

Env
====
CopyTensors()

CreateSharedAllocator
GetSharedAllocator
ReleaseSharedAllocator
CreateAndRegisterAllocatorV2

RegisterAllocator
UnregisterAllocator

EpDevice
======
EpDevice_MemoryInfo
CreateSyncStreamForEpDevice

MemoryInfo
========
CreateMemoryInfo_V2
MemoryInfoGetName 
MemoryInfoGetId 
MemoryInfoGetMemType
MemoryInfoGetType
MemoryInfoGetDeviceMemType
MemoryInfoGetVendorId

Session
==========
SessionGetInputName
SessionGetOutputName

SessionGetMemoryInfoForInputs
SessionGetMemoryInfoForOutputs
SessionGetEpDeviceForInputs

SyncStream
===========
SyncStream_GetHandle
ReleaseSyncStream

OrtArenaCfg
===========
CreateArenaCfgV2

TRT
===
CreateTensorRTProviderOptions and V2
UpdateTensorRTProviderOptions

SessionOptions
==============
OrtSessionOptionsAppendExecutionProvider_CPU

Prepacked container
=============

CUDA Options V2
===========
OrtCUDAProviderOptionsV2
CreateCUDAProviderOptions

GetCUDAProviderOptionsByName
UpdateCUDAProviderOptionsWithValue
UpdateCUDAProviderOptions
GetCUDAProviderOptionsAsString

### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->
Provide a way to write exception safe code.
adrianlizarraga added a commit that referenced this pull request Aug 25, 2025
### Description
Cherry-pick the following PRs into the `rel-1.23.0` branch:
- #25592
- #25622
- #25688
- #25729
- #25743
- #25769
- #25745
- #25761
- #25751
- #25716
- #25228
- #25768
- #25788
- #25747
- #25800
- #25818
- #25762
- #25749
- #25831


### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->

---------

Co-authored-by: quic-tirupath <[email protected]>
Co-authored-by: quic-calvnguy <[email protected]>
Co-authored-by: qti-kromero <[email protected]>
Co-authored-by: Jeff Kilpatrick <[email protected]>
Co-authored-by: Scott McKay <[email protected]>
Co-authored-by: David Fan <[email protected]>
Co-authored-by: kuanyul-qti <[email protected]>
Co-authored-by: Dmitri Smirnov <[email protected]>
Co-authored-by: Chi Lo <[email protected]>
Co-authored-by: Edward Chen <[email protected]>
Co-authored-by: Chunye Wang@AMD <[email protected]>
Co-authored-by: minfhong-qti <[email protected]>
Co-authored-by: Vishal Agarwal <[email protected]>
Co-authored-by: Maximilian Müller <[email protected]>
Co-authored-by: Maximilian Müller <[email protected]>
Co-authored-by: Changming Sun <[email protected]>
Co-authored-by: adrastogi <[email protected]>
Co-authored-by: Aditya Rastogi <[email protected]>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
gedoensmax pushed a commit to gedoensmax/onnxruntime that referenced this pull request Sep 2, 2025
### Description
<!-- Describe your changes. -->
This PR provides C++ interfaces for the following:

Env
====
CopyTensors()

CreateSharedAllocator
GetSharedAllocator
ReleaseSharedAllocator
CreateAndRegisterAllocatorV2

RegisterAllocator
UnregisterAllocator

EpDevice
======
EpDevice_MemoryInfo
CreateSyncStreamForEpDevice

MemoryInfo
========
CreateMemoryInfo_V2
MemoryInfoGetName 
MemoryInfoGetId 
MemoryInfoGetMemType
MemoryInfoGetType
MemoryInfoGetDeviceMemType
MemoryInfoGetVendorId

Session
==========
SessionGetInputName
SessionGetOutputName

SessionGetMemoryInfoForInputs
SessionGetMemoryInfoForOutputs
SessionGetEpDeviceForInputs

SyncStream
===========
SyncStream_GetHandle
ReleaseSyncStream

OrtArenaCfg
===========
CreateArenaCfgV2

TRT
===
CreateTensorRTProviderOptions and V2
UpdateTensorRTProviderOptions

SessionOptions
==============
OrtSessionOptionsAppendExecutionProvider_CPU

Prepacked container
=============

CUDA Options V2
===========
OrtCUDAProviderOptionsV2
CreateCUDAProviderOptions

GetCUDAProviderOptionsByName
UpdateCUDAProviderOptionsWithValue
UpdateCUDAProviderOptions
GetCUDAProviderOptionsAsString

### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->
Provide a way to write exception safe code.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants