Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
578d06e
Refactor connection to autograd with new joint trace creation
beverlylytle May 20, 2025
b04654a
apply update_fusion_call_ctx
beverlylytle May 21, 2025
0c19438
check bw for None
beverlylytle May 21, 2025
f0182ee
don't fuse get_grad
beverlylytle May 23, 2025
440bc96
group get_grads together for torch compile fusions
beverlylytle May 23, 2025
6bb4293
remove torchex impl of get_grad in favor of OpExProcessor exception
beverlylytle May 23, 2025
01dea9d
Merge branch 'main' into reautograd2
beverlylytle May 26, 2025
f45c92d
hide behind flag and clean up
beverlylytle May 26, 2025
eb32063
Xfail test_ddp_grad_bucketing
IvanYashchuk May 27, 2025
32b9675
Xfail test_limit_in_flight_allgathers with bucketing
IvanYashchuk May 27, 2025
675ea02
Xfail test_fsdp_with_no_sync_grad_accumulation
IvanYashchuk May 27, 2025
aafc899
Xfail test_fsdp_grad_parity_with_without_bucketing
IvanYashchuk May 27, 2025
9b4b252
Fix test_rematerialize_all_gather
IvanYashchuk May 27, 2025
a27f390
Restore test_torch_compile_cat_rope_single_fusion
IvanYashchuk May 27, 2025
56625f6
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] May 27, 2025
971c52e
remove extra rematerialization
beverlylytle May 27, 2025
d059353
Merge branch 'main' into reautograd2
beverlylytle May 28, 2025
9cc8c78
remove outdated change
beverlylytle May 28, 2025
67cde6e
Merge branch 'main' into reautograd2
beverlylytle Jun 3, 2025
e1308e1
clean up after merge
beverlylytle Jun 3, 2025
b73702a
more clean up
beverlylytle Jun 3, 2025
d39f270
Merge branch 'main' into reautograd2
beverlylytle Jun 3, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
[pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
  • Loading branch information
pre-commit-ci[bot] committed May 27, 2025
commit 56625f6f9ebffa5b093429e1078b5e511a7c7532
1 change: 1 addition & 0 deletions thunder/core/rematerialization.py
Original file line number Diff line number Diff line change
Expand Up @@ -514,6 +514,7 @@ def rematerialize_all_gather(fw_trace: TraceCtx, bw_trace: TraceCtx) -> tuple[Tr
_update_backward_with_new_saved_for_backward,
_update_forward_with_new_saved_for_backward,
)

_update_backward_with_new_saved_for_backward(new_bw_trace, new_required_for_backward)

new_fw_trace = from_trace(fw_trace)
Expand Down
5 changes: 4 additions & 1 deletion thunder/tests/distributed/test_fsdp.py
Original file line number Diff line number Diff line change
Expand Up @@ -399,7 +399,10 @@ def reset_parameters(self):
assert model.buf.device.type == "cuda"

# This is not updated yet for joint forward-backward trace
@common_utils.decorateIf(unittest.expectedFailure, lambda params: params["bucketing_strategy"] in (FSDPBucketingStrategy.LAYER, FSDPBucketingStrategy.BLOCK))
@common_utils.decorateIf(
unittest.expectedFailure,
lambda params: params["bucketing_strategy"] in (FSDPBucketingStrategy.LAYER, FSDPBucketingStrategy.BLOCK),
)
@common_utils.parametrize(
"executor,bucketing_strategy,fsdptype",
product(
Expand Down
Loading