Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
d9878a3
implement GetEPContextNodes()
thevishalagarwal May 28, 2025
bde3ce5
clean up
thevishalagarwal May 28, 2025
7165dfe
rebase to latest
thevishalagarwal May 29, 2025
7b1f5bc
remove ctx model to just add node
thevishalagarwal Jun 4, 2025
03c42fd
update GetCapabilities for multiple EP Context Nodes
thevishalagarwal Jun 9, 2025
43ac5d5
fix lint
thevishalagarwal Jun 11, 2025
313f4ce
add support for TRT external weights API
thevishalagarwal Jun 18, 2025
a7eadab
add new changes
thevishalagarwal Jul 14, 2025
23c6393
update external initializer fix
thevishalagarwal Jul 23, 2025
7b1320e
fix EP name
thevishalagarwal Jul 24, 2025
3b039fa
reorganize unittest helpers
gedoensmax Jul 31, 2025
28d211e
fix type tests
gedoensmax Jul 31, 2025
bd3d4ed
basic EP context support
gedoensmax Jul 31, 2025
d5151d7
large model test
gedoensmax Aug 1, 2025
8de13f9
remove support for weightless
gedoensmax Aug 1, 2025
e2b67a4
reduce header usages, cleanup and unify usage of windows ifdef
gedoensmax Aug 5, 2025
3728ce0
address review comments
gedoensmax Aug 8, 2025
80574d2
fix engine cache path with EP context
thevishalagarwal Aug 12, 2025
3996d9b
fix unit test to add seed for random tensors
thevishalagarwal Aug 12, 2025
a4f8c45
support sm86 and onwards RTX devices
thevishalagarwal Aug 13, 2025
b0bab1c
update cc check
thevishalagarwal Aug 13, 2025
f762d86
fix lint
thevishalagarwal Aug 15, 2025
b81a6d5
do not copy memory to EP owned memory for raw initializers
gedoensmax Aug 17, 2025
d0926f8
use ort values i data is already loaded in memory
gedoensmax Aug 20, 2025
f640430
remove unused var
gedoensmax Aug 20, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fix engine cache path with EP context
  • Loading branch information
thevishalagarwal authored and gedoensmax committed Aug 20, 2025
commit 80574d29b82c68c1e566658fa8d0e7bd24d236d9
Original file line number Diff line number Diff line change
Expand Up @@ -2611,8 +2611,6 @@ Status NvExecutionProvider::CreateNodeComputeInfoFromGraph(const GraphViewer& gr
cache_path = GetCachePath(cache_path_, fused_node.Name()) + ".engine";
;
}
auto cache_file_name = std::filesystem::path(cache_path).filename();
cache_path = std::filesystem::path(engine_cache_relative_path_to_context_model_dir).append(cache_file_name.string()).string();
// NV TRT EP per default generates hardware compatible engines for any RTX device with compute capability > 80
std::string compute_capability_hw_compat = "80+";
if (!ep_context_model_) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,8 @@ Status CreateCtxNode(const GraphViewer& graph_viewer,
}
attr_ep_cache_context->set_s(engine_data_str);
} else {
attr_ep_cache_context->set_s(engine_cache_path);
std::string engine_cache_filename = std::filesystem::path(engine_cache_path).filename().string();
attr_ep_cache_context->set_s(engine_cache_filename);
std::fstream engine_cache_file(engine_cache_path, std::ios::binary | std::ios::out);
if (engine_cache_file.is_open()) {
engine_cache_file.write(engine_data, size);
Expand Down