Skip to content

TTreeCache: start a new fill at the end of the previous one. #8049

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
pcanal wants to merge 2 commits into
base: master
Choose a base branch
from
Open

TTreeCache: start a new fill at the end of the previous one. #8049

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
5003e27
Add entries range to TBranch::PrintCacheInfo
pcanal Apr 23, 2021
7da8b32
TTreeCache: start a new fill at the end of the previous one.
pcanal Apr 23, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
TTreeCache: start a new fill at the end of the previous one. 
See cms-sw/cmssw#33361

Fixes #8048

Now, set the start point of the filling to be the end of the previous filling rather than the start of the current cluster (which can sometimes
be before the end of the previous filling)

Issue: The error message was inaccurate, it did not take into account jagged filling of the TTreeCache.  In this case, the cache was filled with a little more than one cluster and when it needs to do the next refill it restarted from the cluster start boundary of that partially downloaded cluster which is “indeed” within the range of the last TreeCache fill (i.e. the error).

We did not see the problem with a local file because the TTreeCache usage is different.  CMSSW take note of whether prefetching (asynchronous reads) is available for a while or not.  In the setup CMSSW has, the prefetching (asynchronous reads) is available for the local file but not for the network/remote file.  In addition when prefetching (asynchronous reads) is not available, CMSSW uses multiple TTreeCache for a given TTree while it uses only one when prefetching (asynchronous reads) is available.  This results in the pattern of filling to be different between the 2 cases.
  • Loading branch information
@pcanal
pcanal committed Apr 29, 2021
commit 7da8b321ed45264c82af25fd74a9d588abe22984
2 changes: 2 additions & 0 deletions tree/tree/src/TTreeCache.cxx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1266,6 +1266,8 @@ Bool_t TTreeCache::FillBuffer()
if (showMore || gDebug > 6)
Info("FillBuffer", "Looking at cluster spanning from %lld to %lld", fEntryCurrent, fEntryNext);

if (0 <= fNextClusterStart && fEntryCurrent < fNextClusterStart && !(fEntryCurrent < fNextClusterStart))
Copy link
Contributor

@jblomer jblomer May 3, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fEntryCurrent < fNextClusterStart && !(fEntryCurrent < fNextClusterStart) doesn't seem to make sense.

Copy link
Member Author

@pcanal pcanal May 5, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right. This was intended to detect when the user is moving backward but a "typo/thinko" mess it up :( rendering the whole PR moot ... i.e. Test need to go in at the same time...

fEntryCurrent = fNextClusterStart;
if (fEntryCurrent < fEntryMin) fEntryCurrent = fEntryMin;
if (fEntryMax <= 0) fEntryMax = tree->GetEntries();
if (fEntryNext > fEntryMax) fEntryNext = fEntryMax;
Expand Down
Loading