Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
916823d
[cxxmodules] Implement global module indexing to improve performance.
vgvassilev Feb 20, 2019
e4bb1a8
[cxxmodules] Do not cache the file lookup failure.
vgvassilev Jun 30, 2019
7c14fcf
[cxxmodules] Preload only common modules.
vgvassilev Jun 30, 2019
3bda1e6
[cxxmodules][cling] Add a callback for start/finish code generation.
vgvassilev Jul 4, 2019
3db4b28
[tcling] Modernize header file virtual -> override.
vgvassilev Jul 4, 2019
e96522a
[tcling] Modernize header file: use inline initialization.
vgvassilev Jul 4, 2019
66d6e47
[cxxmodules] Tighten the findInGlobalModuleIndex routine.
vgvassilev Jul 5, 2019
9822d02
[cxxmodules] Preload Hist because it has the same issue as Gpad.
vgvassilev Jul 5, 2019
0766260
Preload Graf for the same reason as Hist and Gpad
arpi-r Jul 12, 2019
a77d5ed
Add GenVector FIXMEModules
arpi-r Jul 25, 2019
661f9dd
Do not resolve from require complete type.
vgvassilev Jan 25, 2020
1d59f46
If the identifier is unknown, return false.
vgvassilev Jan 28, 2020
4b38143
Rename the interface, add documentation, make it bool.
vgvassilev Jan 28, 2020
9c06b0d
Preload modules which are not in the index.
vgvassilev Jan 28, 2020
d84e4a0
Add help how to automatically incorporate clang-format changes.
vgvassilev Jan 28, 2020
da1ec92
Preload Tree to fix the messy roottest-root-io-newstl-make
vgvassilev Jan 29, 2020
c0014fd
Do not load recursively modules.
vgvassilev Feb 3, 2020
7d936ae
Preload module Physics.
vgvassilev Feb 3, 2020
0eea44a
Preload Smatrix, TreePlayer, Proof and Geom.
vgvassilev Feb 11, 2020
774c08f
[cxxmodules] Make the global module index opt-in.
vgvassilev Feb 23, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
[cxxmodules] Do not cache the file lookup failure.
In cases where we have the module cache path and prebuilt module path pointing
to the same location, the FileManager should not cache the module file lookup
failure because it may currently be building the module.

This patch is necessary because the global module index is built from the
module cache path and it is loaded from the prebuilt module path.

In a full explicit or implicit module build infrastructure this is not a problem.
However, in a mixed scenario (where modules for third-party dependencies are
built implicitly) such as ours this is problematic. One of the reasons is that
we cannot configure the prebuilt modules paths or module cache paths externally.
This is because the interpreter (at construction time) #includes RuntimeUniverse
which may trigger module build.

This patch allows us to refactor some of the code working around this issue.
  • Loading branch information
vgvassilev committed Feb 24, 2020
commit e4bb1a84711ce8dbe2f01514d9d23d0711e424dd
26 changes: 2 additions & 24 deletions interpreter/cling/lib/Interpreter/CIFactory.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -529,34 +529,12 @@ namespace {
return Paths;
}

/// \brief Prepares a file path for string comparison with another file path.
/// This easily be tricked by a malicious user with hardlinking directories
/// and so on, but for a comparison in good faith this should be enough.
static std::string normalizePath(StringRef path) {
SmallVector<char, 256> AbsolutePath, Result;
AbsolutePath.insert(AbsolutePath.begin(), path.begin(), path.end());
llvm::sys::fs::make_absolute(AbsolutePath);
llvm::sys::fs::real_path(AbsolutePath, Result, true);
return llvm::Twine(Result).str();
}

/// \brief Adds all the paths to the prebuilt module paths of the given
/// HeaderSearchOptions.
static void addPrebuiltModulePaths(clang::HeaderSearchOptions& Opts,
const SmallVectorImpl<StringRef>& Paths) {
for (StringRef ModulePath : Paths) {
// FIXME: If we have a prebuilt module path that is equal to our module
// cache we fail to compile the clang builtin modules for some reason.
// This makes clang to think it failed to build a dependency module, i.e.
// if we are building module C, clang goes off and builds B and A first.
// If the module cache points to the same location as the prebuilt module
// path, clang errors out on building module A, however, it builds it.
// Next time we run, it will build module B and issue diagnostics.
// If we run third time, it'd build successfully C and continue.
// For now it is fixed by just checking those two paths are not identical.
if (normalizePath(ModulePath) != normalizePath(Opts.ModuleCachePath))
Opts.AddPrebuiltModulePath(ModulePath);
}
for (StringRef ModulePath : Paths)
Opts.AddPrebuiltModulePath(ModulePath);
}

static std::string getIncludePathForHeader(const clang::HeaderSearch& HS,
Expand Down
14 changes: 10 additions & 4 deletions interpreter/llvm/src/tools/clang/lib/Lex/HeaderSearch.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -138,6 +138,7 @@ std::string HeaderSearch::getModuleFileName(Module *Module) {
std::string HeaderSearch::getModuleFileName(StringRef ModuleName,
StringRef ModuleMapPath,
bool UsePrebuiltPath) {
std::string ModuleCachePath = getModuleCachePath();
if (UsePrebuiltPath) {
if (HSOpts->PrebuiltModulePaths.empty())
return std::string();
Expand All @@ -146,20 +147,25 @@ std::string HeaderSearch::getModuleFileName(StringRef ModuleName,
for (const std::string &Dir : HSOpts->PrebuiltModulePaths) {
SmallString<256> Result(Dir);
llvm::sys::fs::make_absolute(Result);

bool CacheFailure = true;
// If we have the same ModuleCachePath and PrebuiltModulePath pointing
// to the same folder we should not cache the file lookup failure as it
// may be currently building an implicit module.
if (!ModuleCachePath.empty() && ModuleCachePath == Result)
CacheFailure = false;
llvm::sys::path::append(Result, ModuleName + ".pcm");
if (getFileMgr().getFile(Result.str()))
if (getFileMgr().getFile(Result.str(), /*Open=*/false, CacheFailure))
return Result.str().str();
}
return std::string();
}

// If we don't have a module cache path or aren't supposed to use one, we
// can't do anything.
if (getModuleCachePath().empty())
if (ModuleCachePath.empty())
return std::string();

SmallString<256> Result(getModuleCachePath());
SmallString<256> Result(ModuleCachePath);
llvm::sys::fs::make_absolute(Result);

if (HSOpts->DisableModuleHash) {
Expand Down