Skip to content

First-pass IMT implementation of FlushBaskets. #277

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
bbockelm wants to merge 8 commits into from
Closed

First-pass IMT implementation of FlushBaskets. #277

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
c247dcc
First-pass IMT implementation of FlushBaskets.
bbockelm Oct 6, 2016
0be50e3
Replace mutex with atomics while we are in the parallel flush basket …
bbockelm Oct 7, 2016
88def72
Switch out printf->Info for debug statements.
bbockelm Oct 7, 2016
9235428
Correct includes.
bbockelm Oct 7, 2016
249cb14
Add comment about pitfalls of IMT mode.
bbockelm Oct 7, 2016
e14e674
Fixups from code review.
bbockelm Nov 4, 2016
71b2161
Remove inadvertent inclusion of testing code to MainEvent.cxx.
bbockelm Nov 17, 2016
cca9e30
Allow user to toggle IMT mode explicitly.
bbockelm Nov 17, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions io/io/inc/TFile.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,11 @@ class TFilePrefetch;
class TFile : public TDirectoryFile {
friend class TDirectoryFile;
friend class TFilePrefetch;
// TODO: We need to make sure only one TBasket is being written at a time
// if we are writing multiple baskets in parallel.
#ifdef R__USE_IMT
friend class TBasket;
#endif

public:
/// Asynchronous open request status
Expand Down Expand Up @@ -106,6 +111,7 @@ class TFile : public TDirectoryFile {

#ifdef R__USE_IMT
static ROOT::TRWSpinLock fgRwLock; ///<!Read-write lock to protect global PID list
std::mutex fWriteMutex; ///<!Lock for writing baskets / keys into the file.
Copy link
Member

@Axel-Naumann Axel-Naumann Oct 7, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see #include <mutex>?

Copy link
Contributor Author

@bbockelm bbockelm Oct 7, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

#endif

static TList *fgAsyncOpenRequests; //List of handles for pending open requests
Expand Down
14 changes: 10 additions & 4 deletions tree/tree/inc/TTree.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,8 @@
#include "TVirtualTreePlayer.h"
#endif

#include <mutex>
Copy link
Member

@Axel-Naumann Axel-Naumann Oct 7, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd expect <atomic> instead?

Copy link
Contributor Author

@bbockelm bbockelm Oct 7, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. That was overlooked in changing the mutex to atomics. Fixed.


class TBranch;
class TBrowser;
class TFile;
Expand Down Expand Up @@ -99,6 +101,7 @@ class TTree : public TNamed, public TAttLine, public TAttFill, public TAttMarker

protected:
Long64_t fEntries; ///< Number of entries
// NOTE: cannot use std::atomic for these counters as it cannot be serialized.
Long64_t fTotBytes; ///< Total number of bytes in all branches before compression
Long64_t fZipBytes; ///< Total number of bytes in all branches after compression
Long64_t fSavedBytes; ///< Number of autosaved bytes
Expand Down Expand Up @@ -160,6 +163,9 @@ class TTree : public TNamed, public TAttLine, public TAttFill, public TAttMarker
TTree(const TTree& tt); // not implemented
TTree& operator=(const TTree& tt); // not implemented

#ifdef R__USE_IMT
mutable std::mutex fCounterMutex; ///<!Lock to protect counters
Copy link
Member

@pcanal pcanal Oct 6, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should probably using the TSpinMutex.

Copy link
Contributor Author

@bbockelm bbockelm Oct 6, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:) Yes - but the intent is to throw this away and avoid all locking whatsoever.

Copy link
Contributor Author

@bbockelm bbockelm Oct 7, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in latest version.

#endif
void InitializeSortedBranches();
void SortBranchesByTime();

Expand Down Expand Up @@ -303,8 +309,8 @@ class TTree : public TNamed, public TAttLine, public TAttFill, public TAttMarker
virtual TFriendElement *AddFriend(const char* treename, const char* filename = "");
virtual TFriendElement *AddFriend(const char* treename, TFile* file);
virtual TFriendElement *AddFriend(TTree* tree, const char* alias = "", Bool_t warn = kFALSE);
virtual void AddTotBytes(Int_t tot) { fTotBytes += tot; }
virtual void AddZipBytes(Int_t zip) { fZipBytes += zip; }
virtual void AddTotBytes(Int_t tot) { std::lock_guard<std::mutex> sentry(fCounterMutex); fTotBytes += tot; }
Copy link
Member

@pcanal pcanal Oct 6, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For production code, we would need to make those optional as we need to not penalize the serial code or even multi-thread code not using the IMT.

Copy link
Contributor Author

@bbockelm bbockelm Oct 7, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done - I believe I have gotten everything down to a single boolean flag.

virtual void AddZipBytes(Int_t zip) { std::lock_guard<std::mutex> sentry(fCounterMutex); fZipBytes += zip; }
virtual Long64_t AutoSave(Option_t* option = "");
virtual Int_t Branch(TCollection* list, Int_t bufsize = 32000, Int_t splitlevel = 99, const char* name = "");
virtual Int_t Branch(TList* list, Int_t bufsize = 32000, Int_t splitlevel = 99);
Expand Down Expand Up @@ -439,7 +445,7 @@ class TTree : public TNamed, public TAttLine, public TAttFill, public TAttMarker
virtual Long64_t GetSelectedRows() { return GetPlayer()->GetSelectedRows(); }
virtual Int_t GetTimerInterval() const { return fTimerInterval; }
TBuffer* GetTransientBuffer(Int_t size);
virtual Long64_t GetTotBytes() const { return fTotBytes; }
virtual Long64_t GetTotBytes() const { std::lock_guard<std::mutex> sentry(fCounterMutex); return fTotBytes; }
Copy link
Member

@pcanal pcanal Oct 6, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For correct operation, shouldn't you also force the assignment to a local variable:
std::lock_guardstd::mutex sentry(fCounterMutex);
auto retValue = fTotBytes;
return retValue;

Copy link
Contributor Author

@bbockelm bbockelm Oct 6, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this is correct ( @Dr15Jones - can you confirm); however, I prefer to remove this lock if possible.

What's the best advice for using std::atomic? Maybe use a RAII type like with GetEntry? Would this work:

  • When the parallel flush basket starts, set a (transient) flag with a scoped RAII-style object.
  • If the transient flag is set, use transient std::atomic-based counters. (This should have no overhead when IMT is disabled).
  • When parallel flush ends (no parallel work is going on), add the std::atomic-based counters to the existing, non-atomic fTotBytes and fZipBytes and zero them out.

Copy link
Collaborator

@Dr15Jones Dr15Jones Oct 6, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The use of a local variable depends on when compilers run destructors, before or after copying the value out to memory/register for returning from the function.

Copy link
Contributor Author

@bbockelm bbockelm Oct 6, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, ok: I thought C++ provided a guarantee.

(Good to know, but I think I can just remove the lock entirely)

Copy link
Contributor Author

@bbockelm bbockelm Oct 7, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in latest version - lock removed.

virtual TTree *GetTree() const { return const_cast<TTree*>(this); }
virtual TVirtualIndex *GetTreeIndex() const { return fTreeIndex; }
virtual Int_t GetTreeNumber() const { return 0; }
Expand Down Expand Up @@ -467,7 +473,7 @@ class TTree : public TNamed, public TAttLine, public TAttFill, public TAttMarker
virtual Double_t *GetV4() { return GetPlayer()->GetV4(); }
virtual Double_t *GetW() { return GetPlayer()->GetW(); }
virtual Double_t GetWeight() const { return fWeight; }
virtual Long64_t GetZipBytes() const { return fZipBytes; }
virtual Long64_t GetZipBytes() const { std::lock_guard<std::mutex> sentry(fCounterMutex); return fZipBytes; }
virtual void IncrementTotalBuffers(Int_t nbytes) { fTotalBuffers += nbytes; }
Bool_t IsFolder() const { return kTRUE; }
virtual Int_t LoadBaskets(Long64_t maxmemory = 2000000000);
Expand Down
12 changes: 12 additions & 0 deletions tree/tree/src/TBasket.cxx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -932,6 +932,9 @@ Int_t TBasket::WriteBuffer()
return -1;
}
fMotherDir = file; // fBranch->GetDirectory();
#ifdef R__USE_IMT
Copy link
Member

@pcanal pcanal Nov 4, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given the slightly unusual use pattern (take the lock then explicit unlock then lock). Can you add comment on what this is propecting (and why it will not provoke a dead lock).

std::unique_lock<std::mutex> sentry(file->fWriteMutex);
#endif // R__USE_IMT

if (R__unlikely(fBufferRef->TestBit(TBufferFile::kNotDecompressed))) {
// Read the basket information that was saved inside the buffer.
Expand Down Expand Up @@ -998,7 +1001,16 @@ Int_t TBasket::WriteBuffer()
if (i == nbuffers - 1) bufmax = fObjlen - nzip;
else bufmax = kMAXZIPBUF;
//compress the buffer
#ifdef R__USE_IMT
sentry.unlock();
#endif // R__USE_IMT
// NOTE this is declared with C linkage, so it shouldn't except. Also, when
// USE_IMT is defined, we are guaranteed that the compression buffer is unique per-branch.
// (see fCompressedBufferRef in constructor).
R__zipMultipleAlgorithm(cxlevel, &bufmax, objbuf, &bufmax, bufcur, &nout, cxAlgorithm);
#ifdef R__USE_IMT
sentry.lock();
#endif // R__USE_IMT

// test if buffer has really been compressed. In case of small buffers
// when the buffer contains random data, it may happen that the compressed
Expand Down
138 changes: 100 additions & 38 deletions tree/tree/src/TTree.cxx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1336,7 +1336,7 @@ Long64_t TTree::AutoSave(Option_t* option)
{
if (!fDirectory || fDirectory == gROOT || !fDirectory->IsWritable()) return 0;
if (gDebug > 0) {
printf("AutoSave Tree:%s after %lld bytes written\n",GetName(),fTotBytes);
printf("AutoSave Tree:%s after %lld bytes written\n",GetName(),GetTotBytes());
Copy link
Member

@pcanal pcanal Oct 6, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please use the ROOT printing routine (Info is likely to be the one you are looking for

Copy link
Contributor Author

@bbockelm bbockelm Oct 6, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe there are a few other printf statements - should I also try to grab those?

Copy link
Member

@pcanal pcanal Oct 6, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Possibly but then in a separate commit. thanks.

Copy link
Contributor Author

@bbockelm bbockelm Oct 7, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in latest version - all debugging is now a call to Info.

}
TString opt = option;
opt.ToLower();
Expand All @@ -1346,7 +1346,7 @@ Long64_t TTree::AutoSave(Option_t* option)
FlushBaskets();
}

fSavedBytes = fZipBytes;
fSavedBytes = GetZipBytes();

TKey *key = (TKey*)fDirectory->GetListOfKeys()->FindObject(GetName());
Long64_t nbytes;
Expand Down Expand Up @@ -4418,34 +4418,36 @@ Int_t TTree::Fill()
KeepCircular();
}
if (gDebug > 0) printf("TTree::Fill - A: %d %lld %lld %lld %lld %lld %lld \n",
nbytes, fEntries, fAutoFlush,fAutoSave,fZipBytes,fFlushedBytes,fSavedBytes);
nbytes, fEntries, fAutoFlush,fAutoSave,GetZipBytes(),fFlushedBytes,fSavedBytes);

if (fAutoFlush != 0 || fAutoSave != 0) {
// Is it time to flush or autosave baskets?
if (fFlushedBytes == 0) {
// Decision can be based initially either on the number of bytes
// or the number of entries written.
if ((fAutoFlush<0 && fZipBytes > -fAutoFlush) ||
(fAutoSave <0 && fZipBytes > -fAutoSave ) ||
Long64_t zipBytes = GetZipBytes();
if ((fAutoFlush<0 && zipBytes > -fAutoFlush) ||
(fAutoSave <0 && zipBytes > -fAutoSave ) ||
(fAutoFlush>0 && fEntries%TMath::Max((Long64_t)1,fAutoFlush) == 0) ||
(fAutoSave >0 && fEntries%TMath::Max((Long64_t)1,fAutoSave) == 0) ) {

//First call FlushBasket to make sure that fTotBytes is up to date.
FlushBaskets();
OptimizeBaskets(fTotBytes,1,"");
if (gDebug > 0) Info("TTree::Fill","OptimizeBaskets called at entry %lld, fZipBytes=%lld, fFlushedBytes=%lld\n",fEntries,fZipBytes,fFlushedBytes);
fFlushedBytes = fZipBytes;
OptimizeBaskets(GetTotBytes(),1,"");
if (gDebug > 0) Info("TTree::Fill","OptimizeBaskets called at entry %lld, fZipBytes=%lld, fFlushedBytes=%lld\n",fEntries,GetZipBytes(),fFlushedBytes);
fFlushedBytes = GetZipBytes();
fAutoFlush = fEntries; // Use test on entries rather than bytes

// subsequently in run
if (fAutoSave < 0) {
// Set fAutoSave to the largest integer multiple of
// fAutoFlush events such that fAutoSave*fFlushedBytes
// < (minus the input value of fAutoSave)
if (fZipBytes != 0) {
fAutoSave = TMath::Max( fAutoFlush, fEntries*((-fAutoSave/fZipBytes)/fEntries));
} else if (fTotBytes != 0) {
fAutoSave = TMath::Max( fAutoFlush, fEntries*((-fAutoSave/fTotBytes)/fEntries));
Long64_t totBytes = GetTotBytes();
if (zipBytes != 0) {
fAutoSave = TMath::Max( fAutoFlush, fEntries*((-fAutoSave/zipBytes)/fEntries));
} else if (totBytes != 0) {
fAutoSave = TMath::Max( fAutoFlush, fEntries*((-fAutoSave/totBytes)/fEntries));
} else {
TBufferFile b(TBuffer::kWrite, 10000);
TTree::Class()->WriteBuffer(b, (TTree*) this);
Expand All @@ -4462,24 +4464,24 @@ Int_t TTree::Fill()
if (fAutoSave != 0 && fEntries%fAutoSave == 0) {
//We are at an AutoSave point. AutoSave flushes baskets and saves the Tree header
AutoSave("flushbaskets");
if (gDebug > 0) Info("TTree::Fill","AutoSave called at entry %lld, fZipBytes=%lld, fSavedBytes=%lld\n",fEntries,fZipBytes,fSavedBytes);
if (gDebug > 0) Info("TTree::Fill","AutoSave called at entry %lld, fZipBytes=%lld, fSavedBytes=%lld\n",fEntries,GetZipBytes(),fSavedBytes);
} else {
//We only FlushBaskets
FlushBaskets();
if (gDebug > 0) Info("TTree::Fill","FlushBasket called at entry %lld, fZipBytes=%lld, fFlushedBytes=%lld\n",fEntries,fZipBytes,fFlushedBytes);
if (gDebug > 0) Info("TTree::Fill","FlushBasket called at entry %lld, fZipBytes=%lld, fFlushedBytes=%lld\n",fEntries,GetZipBytes(),fFlushedBytes);
}
fFlushedBytes = fZipBytes;
fFlushedBytes = GetZipBytes();
} else if (fNClusterRange == 0 && fEntries > 1 && fAutoFlush && fEntries%fAutoFlush == 0) {
if (fAutoSave != 0 && fEntries%fAutoSave == 0) {
//We are at an AutoSave point. AutoSave flushes baskets and saves the Tree header
AutoSave("flushbaskets");
if (gDebug > 0) Info("TTree::Fill","AutoSave called at entry %lld, fZipBytes=%lld, fSavedBytes=%lld\n",fEntries,fZipBytes,fSavedBytes);
if (gDebug > 0) Info("TTree::Fill","AutoSave called at entry %lld, fZipBytes=%lld, fSavedBytes=%lld\n",fEntries,GetZipBytes(),fSavedBytes);
} else {
//We only FlushBaskets
FlushBaskets();
if (gDebug > 0) Info("TTree::Fill","FlushBasket called at entry %lld, fZipBytes=%lld, fFlushedBytes=%lld\n",fEntries,fZipBytes,fFlushedBytes);
if (gDebug > 0) Info("TTree::Fill","FlushBasket called at entry %lld, fZipBytes=%lld, fFlushedBytes=%lld\n",fEntries,GetZipBytes(),fFlushedBytes);
}
fFlushedBytes = fZipBytes;
fFlushedBytes = GetZipBytes();
}
}
// Check that output file is still below the maximum size.
Expand Down Expand Up @@ -4802,6 +4804,48 @@ Int_t TTree::FlushBaskets() const
Int_t nerror = 0;
TObjArray *lb = const_cast<TTree*>(this)->GetListOfBranches();
Int_t nb = lb->GetEntriesFast();

#ifdef R__USE_IMT
if (ROOT::IsImplicitMTEnabled() && fIMTEnabled) {
if (fSortedBranches.empty()) { const_cast<TTree*>(this)->InitializeSortedBranches(); }

// Enable this IMT use case (activate its locks)
ROOT::Internal::TParBranchProcessingRAII pbpRAII;

std::atomic<Int_t> nerrpar(0);
std::atomic<Int_t> nbpar(0);
std::atomic<Int_t> pos(0);
tbb::task_group g;
Copy link
Collaborator

@Dr15Jones Dr15Jones Oct 7, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We must be extremely careful when embedding calls to TBB into legacy code. Holding any shared resource (which a C++ object itself may be viewed as such a resource) can lead to deadlocks. See

https://software.intel.com/en-us/forums/intel-threading-building-blocks/topic/401006

for details. This is one reason why the CMS threaded-framework is built using explicit tasks rather than implicit ones. For the case of writing, I can't think of anyone being dependent on the TFile, although we must be certain that no ROOT locks are being held by the thread calling g.wait().
The use of a tbb::task_arena would avoid the deadlock but does not appear to give us a way to limit the number of available threads.

Copy link
Contributor Author

@bbockelm bbockelm Oct 7, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi Chris,

Indeed.

My logic here is:

  • This function is high up enough in the call stack to verify it is not called with any global ROOT mutexes held.
  • The locking within the function was made per-TFile.
  • TTree and TFile are both thread-unsafe objects (cannot have two threads doing const or non-const calls into the same object), meaning there should not be multiple callers to this function regardless.

So, the remaining danger is if the caller (CMSSW) is holding a mutex of its own that other CMSSW tasks might need.

I will write these concerns up into a comment, provide an example of dangerous behavior, and add them to the function's documentation.

Brian

Copy link
Contributor Author

@bbockelm bbockelm Oct 7, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's an example of a deadlock that could have happened in older versions of CMSSW:

  • CMSSW takes a mutex on the output module.
  • CMSSW performs a ROOT call which causes a FlushBaskets operation.
  • ROOT creates many basket flushing tasks and then wait on the task group.
  • TBB's runtime notice the thread blocks in wait and schedules another CMSSW task on the same thread.
  • The other CMSSW also needs to access the output module, hence takes a mutex.
  • Since the thread already holds the output module's mutex, trying to acquire it again causes a deadlock.

Now, Chris tells me that CMSSW has switched away from the mutex for output module access - but this might be a good example.

I pushed a long comment for FlushBaskets outlining the above.

Copy link
Member

@pcanal pcanal Oct 7, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The other CMSSW also needs to access the output module, hence takes a mutex.
Since the thread already holds the output module's mutex, trying to acquire it again causes a deadlock.

This of course depend on whether the mutex that is used is recursive or not (TMutex is, std::mutex is not).

Copy link
Contributor Author

@bbockelm bbockelm Oct 7, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Certainly!

The point is that working code with IMT-disabled may stop working when IMT is enabled. One would hope that issues would be fixable once the user understands the underlying issue. Hence the extensive comment in the code.


for (Int_t i = 0; i < nb; i++) {
g.run([&]() {
// The branch to process is obtained when the task starts to run.
// This way, since branches are sorted, we make sure that branches
// leading to big tasks are processed first. If we assigned the
// branch at task creation time, the scheduler would not necessarily
// respect our sorting.
Int_t j = pos.fetch_add(1);

auto branch = fSortedBranches[j].second;
if (R__unlikely(!branch)) { return; }

if (R__unlikely(gDebug > 0)) {
std::stringstream ss;
ss << std::this_thread::get_id();
Info("FlushBaskets", "[IMT] Thread %s", ss.str().c_str());
Info("FlushBaskets", "[IMT] Running task for branch #%d: %s", j, branch->GetName());
}

Int_t nbtask = branch->FlushBaskets();

if (nbtask < 0) { nerrpar++; }
else { nbpar += nbtask; }
});
}
g.wait();
return nerrpar ? -1 : nbpar.load();
}
#endif
for (Int_t j = 0; j < nb; j++) {
TBranch* branch = (TBranch*) lb->UncheckedAt(j);
if (branch) {
Expand Down Expand Up @@ -5006,7 +5050,7 @@ Long64_t TTree::GetCacheAutoSize(Bool_t withDefault /* = kFALSE */ ) const

if (fAutoFlush < 0) cacheSize = Long64_t(-cacheFactor*fAutoFlush);
else if (fAutoFlush == 0) cacheSize = 0;
else cacheSize = Long64_t(cacheFactor*1.5*fAutoFlush*fZipBytes/(fEntries+1));
else cacheSize = Long64_t(cacheFactor*1.5*fAutoFlush*GetZipBytes()/(fEntries+1));

if (cacheSize >= (INT_MAX / 4)) {
cacheSize = INT_MAX / 4;
Expand All @@ -5019,7 +5063,7 @@ Long64_t TTree::GetCacheAutoSize(Bool_t withDefault /* = kFALSE */ ) const
if (cacheSize == 0 && withDefault) {
if (fAutoFlush < 0) cacheSize = -fAutoFlush;
else if (fAutoFlush == 0) cacheSize = 0;
else cacheSize = Long64_t(1.5*fAutoFlush*fZipBytes/(fEntries+1));
else cacheSize = Long64_t(1.5*fAutoFlush*GetZipBytes()/(fEntries+1));
}

return cacheSize;
Expand Down Expand Up @@ -6702,16 +6746,17 @@ void TTree::Print(Option_t* option) const
}
}
Long64_t total = skey;
if (fZipBytes > 0) {
total += fTotBytes;
Long64_t zipBytes = GetZipBytes();
if (zipBytes > 0) {
total += GetTotBytes();
}
TBufferFile b(TBuffer::kWrite, 10000);
TTree::Class()->WriteBuffer(b, (TTree*) this);
total += b.Length();
Long64_t file = fZipBytes + s;
Long64_t file = zipBytes + s;
Float_t cx = 1;
if (fZipBytes) {
cx = (fTotBytes + 0.00001) / fZipBytes;
if (zipBytes) {
cx = (GetTotBytes() + 0.00001) / zipBytes;
}
Printf("******************************************************************************");
Printf("*Tree :%-10s: %-54s *", GetName(), GetTitle());
Expand Down Expand Up @@ -7352,8 +7397,11 @@ void TTree::Refresh()

fAutoSave = tree->fAutoSave;
fEntries = tree->fEntries;
fTotBytes = tree->fTotBytes;
fZipBytes = tree->fZipBytes;
{
std::lock_guard<std::mutex> sentry(fCounterMutex);
fTotBytes = tree->GetTotBytes();
fZipBytes = tree->GetZipBytes();
}
fSavedBytes = tree->fSavedBytes;
fTotalBuffers = tree->fTotalBuffers.load();

Expand Down Expand Up @@ -7404,8 +7452,11 @@ void TTree::Reset(Option_t* option)
fNotify = 0;
fEntries = 0;
fNClusterRange = 0;
fTotBytes = 0;
fZipBytes = 0;
{
std::lock_guard<std::mutex> sentry(fCounterMutex);
fTotBytes = 0;
fZipBytes = 0;
}
fFlushedBytes = 0;
fSavedBytes = 0;
fTotalBuffers = 0;
Expand Down Expand Up @@ -7434,8 +7485,11 @@ void TTree::ResetAfterMerge(TFileMergeInfo *info)
{
fEntries = 0;
fNClusterRange = 0;
fTotBytes = 0;
fZipBytes = 0;
{
std::lock_guard<std::mutex> sentry(fCounterMutex);
fTotBytes = 0;
fZipBytes = 0;
}
fSavedBytes = 0;
fFlushedBytes = 0;
fTotalBuffers = 0;
Expand Down Expand Up @@ -8823,10 +8877,12 @@ void TTree::Streamer(TBuffer& b)
} else if (fAutoFlush != 0) {
// Estimate the cluster size.
// This will allow TTree::Process to enable the cache.
if (fZipBytes != 0) {
fCacheSize = fAutoFlush*(fZipBytes/fEntries);
} else if (fTotBytes != 0) {
fCacheSize = fAutoFlush*(fTotBytes/fEntries);
Long64_t zipBytes = GetZipBytes();
Long64_t totBytes = GetTotBytes();
if (zipBytes != 0) {
fCacheSize = fAutoFlush*(zipBytes/fEntries);
} else if (totBytes != 0) {
fCacheSize = fAutoFlush*(totBytes/fEntries);
} else {
fCacheSize = 30000000;
}
Expand All @@ -8852,16 +8908,22 @@ void TTree::Streamer(TBuffer& b)
b >> ijunk; fMaxEntryLoop = (Long64_t)ijunk;
b >> ijunk; fMaxVirtualSize = (Long64_t)ijunk;
b >> djunk; fEntries = (Long64_t)djunk;
b >> djunk; fTotBytes = (Long64_t)djunk;
b >> djunk; fZipBytes = (Long64_t)djunk;
{
std::lock_guard<std::mutex> sentry(fCounterMutex);
b >> djunk; fTotBytes = (Long64_t)djunk;
b >> djunk; fZipBytes = (Long64_t)djunk;
}
b >> ijunk; fAutoSave = (Long64_t)ijunk;
b >> ijunk; fEstimate = (Long64_t)ijunk;
if (fEstimate <= 10000) fEstimate = 1000000;
fBranches.Streamer(b);
if (fBranchRef) fBranchRef->SetTree(this);
TBranch__SetTree(this,fBranches);
fLeaves.Streamer(b);
fSavedBytes = fTotBytes;
{
std::lock_guard<std::mutex> sentry(fCounterMutex);
fSavedBytes = fTotBytes;
}
if (R__v > 1) fIndexValues.Streamer(b);
if (R__v > 2) fIndex.Streamer(b);
if (R__v > 3) {
Expand Down