-
Notifications
You must be signed in to change notification settings - Fork 1.4k
First-pass IMT implementation of FlushBaskets. #277
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 1 commit
c247dcc
0be50e3
88def72
9235428
249cb14
e14e674
71b2161
cca9e30
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
- Loading branch information
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -163,11 +163,11 @@ class TTree : public TNamed, public TAttLine, public TAttFill, public TAttMarker | |
| TTree(const TTree& tt); // not implemented | ||
| TTree& operator=(const TTree& tt); // not implemented | ||
|
|
||
| #ifdef R__USE_IMT | ||
| // For simplicity, although fIMTFlush is always disabled in non-IMT builds, we don't #ifdef it out. | ||
| mutable Bool_t fIMTFlush{false}; ///<! True if we are doing a multithreaded flush. | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Odd to see a mutable non-atomic. Is that really the intention? (i.e. is it really never possible to modify/read it in parallel?)
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes, that's the intention. Well, the real problem here is that The I wanted to avoid making it atomic to fulfill the request that there is no performance penalty when ROOT is used in non-IMT mode.
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Fair enough, a comment here or at the use place along the line of this answer ought to be added. |
||
| mutable std::atomic<Long64_t> fIMTTotBytes; ///<! Total bytes for the IMT flush baskets | ||
| mutable std::atomic<Long64_t> fIMTZipBytes; ///<! Zip bytes for the IMT flush baskets. | ||
| #endif | ||
|
|
||
| void InitializeSortedBranches(); | ||
| void SortBranchesByTime(); | ||
|
|
||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -932,6 +932,12 @@ Int_t TBasket::WriteBuffer() | |
| return -1; | ||
| } | ||
| fMotherDir = file; // fBranch->GetDirectory(); | ||
|
|
||
| // This mutex prevents multiple TBasket::WriteBuffer invocations from interacting | ||
| // with the underlying TFile at once - TFile is assumed to *not* be thread-safe. | ||
| // | ||
| // The only parallelism we'd like to exploit (right now!) is the compression | ||
| // step - everything else should be serialized at the TFile level. | ||
| #ifdef R__USE_IMT | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Given the slightly unusual use pattern (take the lock then explicit unlock then lock). Can you add comment on what this is propecting (and why it will not provoke a dead lock). |
||
| std::unique_lock<std::mutex> sentry(file->fWriteMutex); | ||
| #endif // R__USE_IMT | ||
|
|
@@ -1000,7 +1006,9 @@ Int_t TBasket::WriteBuffer() | |
| for (Int_t i = 0; i < nbuffers; ++i) { | ||
| if (i == nbuffers - 1) bufmax = fObjlen - nzip; | ||
| else bufmax = kMAXZIPBUF; | ||
| //compress the buffer | ||
| // Compress the buffer. Note that we allow multiple TBasket compressions to occur at once | ||
| // for a given TFile: that's because the compression buffer when we use IMT is no longer | ||
| // shared amongst several threads. | ||
| #ifdef R__USE_IMT | ||
| sentry.unlock(); | ||
| #endif // R__USE_IMT | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -4792,6 +4792,8 @@ Int_t TTree::Fit(const char* funcname, const char* varexp, const char* selection | |
| return -1; | ||
| } | ||
|
|
||
| namespace { | ||
|
|
||
| struct BoolRAIIToggle { | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Consider putting this class in the anonymous namespace. |
||
|
|
||
| Bool_t &m_val; | ||
|
|
@@ -4801,6 +4803,7 @@ BoolRAIIToggle(Bool_t &val) : m_val(val) { m_val = true; } | |
| ~BoolRAIIToggle() { m_val = false; } | ||
| }; | ||
|
|
||
| } | ||
|
|
||
| //////////////////////////////////////////////////////////////////////////////// | ||
| /// Write to disk all the basket that have not yet been individually written. | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should be optional (so we need to add a command line argument) and default to not be enabled (as is the test now complain if ROOT was not built with IMT enabled).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ugh. Those were test modifications to
MainEvent.cxxthat I did not intend to commit. Will remove them all.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I disagree ;). I think it is a useful new testing swtich ... it just need to be made conditional. thanks.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would you prefer that it is automatically enabled or add it as a command line flag?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As a command line flag so that the default stay as is and one can test with and without it with the same build.