[cxxmodules] Enable the semantic global module index to boost performance.

README/README.CXXMODULES.md

@@ -309,10 +309,6 @@ different. There are several differences which can be noticed:
   * rootcling -cxxmodule creates a single artifact *Name.pcm* after the library
   name. At a final stage, ROOT might be able to integrate the Name.pcm with the
   shared library itself.
-  * Preloads all \*pcm files at start up time -- this currently is the only
-  remaining bottleneck which introduces a relatively small performance overhead
-  at startup time and is described bellow. It will be negligible for third-
-  party code (dominated by header parsing).
   * Improved correctness in number of cases -- in a few cases ROOT is more
   correct. In particular, when resolving global variables and function
   declarations which are not part of the ROOT PCH.
@@ -323,6 +319,21 @@ different. There are several differences which can be noticed:
   the LD_LIBRARY_PATH descending to the system libraries. The algorithm is very
   efficient because it uses bloom filters[[5]]. This in turn allows ROOT symbol
   to be extended to system libraries.
+
+### Module Registration Approaches
+
+  The C++ modules system supports /*preloading*/ of all modules at startup time.
+  The current implementation of loading of C++ modules in clang has an overhead
+  and is between 40-60 MB depending on the ROOT configuration while there might
+  be 2x slowdown depending on the workflow. These issues are very likely to be
+  addressed by the LLVM community in midterm.
+
+  Preloading of all C++ modules is semantically the closest to C++ behavior.
+  However, in order to achieve performance ROOT loads them on demand using
+  a global module index file. It has sufficient information to map a looked up
+  identifier to the module which contains the corresponding definition. Switching
+  back to preloading of all C++ modules is done by setting the `ROOT_USE_GMI`
+  environment variable to false.
 
 ### Supported Platforms
 
@@ -349,14 +360,15 @@ different. There are several differences which can be noticed:
 
 ## State of the union
 
-C++ Modules-aware ROOT preloads all modules at start up time. Our motivating
-example:
+Preloading all modules at start up time turn our motivating example into:
 
 ```cpp
 // ROOT prompt
 root [] S *s;           // #1: does not require a definition.
 root [] foo::bar *baz1; // #2: does not require a definition.
 root [] foo::bar baz2;  // #3: requires a definition.
+root [] TCanvas* c = new TCanvas(); // #4 requires a definition
+
 ```
 
 becomes equivalent to
@@ -368,12 +380,29 @@ root [] import Foo.*;
 root [] S *s;           // #1: does not require a definition.
 root [] foo::bar *baz1; // #2: does not require a definition.
 root [] foo::bar baz2;  // #3: requires a definition.
+root [] TCanvas* c = new TCanvas(); // #4 requires a definition
 ```
 
 The implementation avoids recursive actions and relies on a well-defined (by
 the C++ standard) behavior. Currently, this comes with a constant performance
 overhead which we go in details bellow.
 
+ROOT uses the global module index (GMI) to avoid the performance overhead. ROOT
+only preloads the set of C++ modules which are not present in the GMI. The
+example becomes equivalent to:
+
+```cpp
+// ROOT prompt
+root [] import Foo.*;   // Preload Foo if it is not in the GMI.
+root [] S *s;           // #1: does not require a definition.
+root [] foo::bar *baz1; // #2: does not require a definition.
+root [] foo::bar baz2;  // #3: requires a definition.
+root [] TCanvas* c = new TCanvas(); // #4 requires a definition
+```
+
+Line #4 forces cling to send ROOT a callback that TCanvas in unknown but
+the GMI resolves it to module Gpad, loads it and returns the control to cling.
+
 
 ### Performance
 This section compares ROOT PCH technology with C++ Modules which is important but
@@ -385,16 +414,9 @@ is not available.
 The comparisons are to give a good metric when we are ready to switch ROOT to use
 C++ Modules by default. However, since it is essentially the same technology,
 optimizations of C++ Modules also affect the PCH. We have a few tricks up in
-the slaves to but they come with given trade-offs. For example, we can avoid
-preloading of all modules at the cost of introducing recursive behavior in
-loading. This requires to build a global module index which is an on-disk
-hash table. It will contain information about the mapping between an
-identifier and a module name. Upon failed identifier lookup we will use the
-map to decide which set of modules should be loaded. Another optimization
-includes building some of the modules without `-fmodules-local-submodule-visibility`.
-In turn, this would flatten the C++ modules structure and give us performance
-comparable to the ROOT PCH. The trade-off is that we will decrease the
-encapsulation and leak information about implementation-specific header files.
+the sleeves to but they come with given trade-offs.
+
+#### Preloading of C++ Modules
 
 The main focus for the technology preview was not in performance until recently.
 We have invested some resources in optimizations and we would like to show you
@@ -413,6 +435,14 @@ The performance is dependent on many factors such as configuration of ROOT and
 workflow. You can read more at our Intel IPCC-ROOT Showcase presentation
 here (pp 25-33)[[8]].
 
+#### Loading C++ Modules on Demand
+
+In long term, we should optimize the preloading of modules to be a no-op and
+avoid recursive behavior based on identifier lookup callbacks. Unfortunately,
+at the moment the loading of C++ modules on demand shows significantly better
+performance results.
+
+
 You can visit our continuous performance monitoring tool where we compare
 the performance of ROOT against ROOT with a PCH [[9]].
 *Note: if you get error 400, clean your cache or open a private browser session.*

cmake/modules/RootMacros.cmake

@@ -1671,10 +1671,15 @@ function(ROOT_ADD_UNITTEST_DIR)
 endfunction()
 
 #----------------------------------------------------------------------------
-# function ROOT_ADD_GTEST(<testsuite> source1 source2... COPY_TO_BUILDDIR file1 file2 LIBRARIES)
-#
+# function ROOT_ADD_GTEST(<testsuite> source1 source2...
+#                        [WILLFAIL]
+#                        [COPY_TO_BUILDDIR file1 file2...] -- files to copy in the build directory
+#                        [LIBRARIES lib1 lib2...] -- Libraries to link against
+#                        [LABELS label1 label2...]) -- Labels to annotate the test
+#                        [INCLUDE_DIRS label1 label2...]) -- Extra target include directories
+
 function(ROOT_ADD_GTEST test_suite)
-  CMAKE_PARSE_ARGUMENTS(ARG "WILLFAIL" "" "COPY_TO_BUILDDIR;LIBRARIES;LABELS" ${ARGN})
+  CMAKE_PARSE_ARGUMENTS(ARG "WILLFAIL" "" "COPY_TO_BUILDDIR;LIBRARIES;LABELS;INCLUDE_DIRS" ${ARGN})
 
   # ROOTUnitTestSupport
   if(NOT TARGET ROOTUnitTestSupport)
@@ -1691,6 +1696,10 @@ function(ROOT_ADD_GTEST test_suite)
   ROOT_EXECUTABLE(${test_suite} ${source_files} LIBRARIES ${ARG_LIBRARIES})
   target_link_libraries(${test_suite} gtest gtest_main gmock gmock_main ROOTUnitTestSupport)
   target_include_directories(${test_suite} PRIVATE ${CMAKE_CURRENT_BINARY_DIR})
+  if (ARG_INCLUDE_DIRS)
+    target_include_directories(${test_suite} PRIVATE ${ARG_INCLUDE_DIRS})
+  endif(ARG_INCLUDE_DIRS)
+
   if(MSVC)
     set(test_exports "/EXPORT:_Init_thread_abort /EXPORT:_Init_thread_epoch
         /EXPORT:_Init_thread_footer /EXPORT:_Init_thread_header /EXPORT:_tls_index")

core/dictgen/src/rootcling_impl.cxx

@@ -4325,6 +4325,8 @@ int RootClingMain(int argc,
                                       GetModuleNameFromRdictName(DepMod).str().data());
          }
          DepMod = GetModuleNameFromRdictName(DepMod);
+         // We might deserialize.
+         cling::Interpreter::PushTransactionRAII RAII(&interp);
          if (!interp.loadModule(DepMod, /*complain*/false)) {
             ROOT::TMetaUtils::Error(0, "Module '%s' failed to load.\n",
                                     DepMod.data());

core/foundation/res/ROOT/FoundationUtils.hxx

@@ -9,7 +9,7 @@
 /// \date June, 2019
 ///
 /*************************************************************************
- * Copyright (C) 1995-2019, Rene Brun and Fons Rademakers.               *
+ * Copyright (C) 1995-2020, Rene Brun and Fons Rademakers.               *
  * All rights reserved.                                                  *
  *                                                                       *
  * For the licensing terms see $ROOTSYS/LICENSE.                         *
@@ -74,6 +74,13 @@ namespace FoundationUtils {
    ///\returns the sysconfig directory in the installation.
    const std::string& GetEtcDir();
 
+   ///\returns true if lowercase \c value is 1, on, true, 0, off, false
+   bool CanConvertEnvValueToBool(const std::string& value);
+
+   ///\returns true if the lowercase string is 1, on, true; false if 0, off,
+   /// false
+   bool ConvertEnvValueToBool(const std::string& value);
+
    } // namespace FoundationUtils
 } // namespace ROOT
 

core/foundation/src/FoundationUtils.cxx

@@ -21,6 +21,7 @@
 #include <RConfigure.h>
 
 #include <algorithm>
+#include <cassert>
 
 #include <errno.h>
 #include <string.h>
@@ -178,5 +179,32 @@ const std::string& GetEtcDir() {
    return rootetcdir;
 }
 
+static std::string str_tolower(std::string s) {
+   std::transform(s.begin(), s.end(), s.begin(),
+                  [](unsigned char c){ return std::tolower(c); });
+   return s;
+}
+
+bool CanConvertEnvValueToBool(const std::string& value) {
+   std::string lowercase = str_tolower(value);
+   if (lowercase == "1" || lowercase == "on" || lowercase == "true")
+      return true;
+   if (lowercase == "0" || lowercase == "off" || lowercase == "false")
+      return true;
+
+   return false;
+}
+
+bool ConvertEnvValueToBool(const std::string& value) {
+   assert(CanConvertEnvValueToBool(value));
+   std::string lowercase = str_tolower(value);
+   if (lowercase == "1" || lowercase == "on" || lowercase == "true")
+      return true;
+   if (lowercase == "0" || lowercase == "off" || lowercase == "false")
+      return false;
+   // FIXME: Implement a wrapper around __builtin_unreachable() and use it here
+   return false;
+}
+
 } // namespace FoundationUtils
 } // namespace ROOT

core/foundation/test/CMakeLists.txt

@@ -8,3 +8,4 @@ ROOT_ADD_GTEST(testMake_unique testMake_unique.cxx LIBRARIES Core)
 ROOT_ADD_GTEST(testTypeTraits testTypeTraits.cxx LIBRARIES Core)
 ROOT_ADD_GTEST(testNotFn testNotFn.cxx LIBRARIES Core)
 ROOT_ADD_GTEST(testClassEdit testClassEdit.cxx LIBRARIES Core)
+ROOT_ADD_GTEST(FoundationUtilsTests FoundationUtilsTests.cxx LIBRARIES Core INCLUDE_DIRS ../res)

core/foundation/test/FoundationUtilsTests.cxx

@@ -0,0 +1,52 @@
+/// \file FoundationUtilsTests.h
+///
+/// \brief The file contain unit tests which test the ROOT::FoundationUtils
+///
+/// \author Vassil Vassilev <[email protected]>
+///
+/// \date Jun, 2020
+///
+/*************************************************************************
+ * Copyright (C) 1995-2020, Rene Brun and Fons Rademakers.               *
+ * All rights reserved.                                                  *
+ *                                                                       *
+ * For the licensing terms see $ROOTSYS/LICENSE.                         *
+ * For the list of contributors see $ROOTSYS/README/CREDITS.             *
+ *************************************************************************/
+
+#include <ROOT/FoundationUtils.hxx>
+
+#include "gtest/gtest.h"
+
+using namespace ROOT::FoundationUtils;
+
+TEST(FoundationUtilsTests, CanConvertEnvValueToBool)
+{
+  ASSERT_FALSE(CanConvertEnvValueToBool(""));
+  ASSERT_TRUE(CanConvertEnvValueToBool("0"));
+  ASSERT_TRUE(CanConvertEnvValueToBool("false"));
+  ASSERT_TRUE(CanConvertEnvValueToBool("False"));
+  ASSERT_TRUE(CanConvertEnvValueToBool("FALSE"));
+  ASSERT_TRUE(CanConvertEnvValueToBool("Off"));
+  ASSERT_TRUE(CanConvertEnvValueToBool("off"));
+  ASSERT_TRUE(CanConvertEnvValueToBool("OFF"));
+
+  ASSERT_TRUE(CanConvertEnvValueToBool("1"));
+  ASSERT_TRUE(CanConvertEnvValueToBool("true"));
+  ASSERT_TRUE(CanConvertEnvValueToBool("True"));
+  ASSERT_TRUE(CanConvertEnvValueToBool("TRUE"));
+  ASSERT_TRUE(CanConvertEnvValueToBool("On"));
+  ASSERT_TRUE(CanConvertEnvValueToBool("on"));
+  ASSERT_TRUE(CanConvertEnvValueToBool("ON"));
+}
+
+TEST(FoundationUtilsTests, ConvertEnvValueToBool)
+{
+  ASSERT_TRUE(ConvertEnvValueToBool("1"));
+  ASSERT_TRUE(ConvertEnvValueToBool("TruE"));
+  ASSERT_TRUE(ConvertEnvValueToBool("oN"));
+
+  ASSERT_FALSE(ConvertEnvValueToBool("0"));
+  ASSERT_FALSE(ConvertEnvValueToBool("FalSe"));
+  ASSERT_FALSE(ConvertEnvValueToBool("oFf"));
+}

core/metacling/src/TCling.cxx

@@ -1041,6 +1041,7 @@ static bool LoadModule(const std::string &ModuleName, cling::Interpreter &interp
       ::Info("TCling::__LoadModule", "Preloading module %s. \n",
              ModuleName.c_str());
 
+   cling::Interpreter::PushTransactionRAII deserRAII(&interp);
    return interp.loadModule(ModuleName, /*Complain=*/true);
 }
 
@@ -1112,26 +1113,45 @@ static GlobalModuleIndex *loadGlobalModuleIndex(SourceLocation TriggerLoc, cling
          struct DefinitionFinder : public RecursiveASTVisitor<DefinitionFinder> {
             DefinitionFinder(clang::GlobalModuleIndex::UserDefinedInterestingIDs& IDs,
                              clang::TranslationUnitDecl* TU) : DefinitionIDs(IDs) {
+               // for (const Decl* D : TU->decls()) {
+               //    if (!isa<NamedDecl>(D))
+               //       continue;
+               //    const NamedDecl* ND = cast<NamedDecl>(D);
+               //    if (!ND->isFromASTFile())
+               //       continue;
+
+               //    if (const TagDecl *TD = llvm::dyn_cast<TagDecl>(ND)) {
+               //       if (TD->isCompleteDefinition())
+               //          Register(TD);
+               //    } else if (const NamespaceDecl *NSD = llvm::dyn_cast<NamespaceDecl>(ND)) {
+               //       // if (!NSD->getParent()->isTranslationUnit())
+               //       //    return false;
+               //       Register(NSD, /*AddSingleEntry=*/ false);
+               //    }
+               //    else if (const TypedefNameDecl *TND = dyn_cast<TypedefNameDecl>(ND))
+               //       Register(TND);
+               //    // FIXME: Add the rest...
+               // }
                TraverseDecl(TU);
             }
             bool VisitNamedDecl(NamedDecl *ND) {
-               for (auto R : ND->redecls()) {
-                  if (!R->isFromASTFile())
-                     continue;
-                  if (TagDecl *TD = llvm::dyn_cast<TagDecl>(R)) {
-                     if (TD->isCompleteDefinition())
-                        Register(TD);
-                  } else if (NamespaceDecl *NSD = llvm::dyn_cast<NamespaceDecl>(R))
-                     Register(NSD, /*AddSingleEntry=*/ false);
-                  else if (TypedefNameDecl *TND = dyn_cast<TypedefNameDecl>(R))
-                     Register(TND);
-                  // FIXME: Add the rest...
+               if (!ND->isFromASTFile())
+                  return true;
+
+               if (TagDecl *TD = llvm::dyn_cast<TagDecl>(ND)) {
+                  if (TD->isCompleteDefinition())
+                     Register(TD);
+               } else if (NamespaceDecl *NSD = llvm::dyn_cast<NamespaceDecl>(ND)) {
+                  Register(NSD, /*AddSingleEntry=*/ false);
                }
+               else if (TypedefNameDecl *TND = dyn_cast<TypedefNameDecl>(ND))
+                  Register(TND);
+               // FIXME: Add the rest...
                return true; // continue decending
             }
          private:
             clang::GlobalModuleIndex::UserDefinedInterestingIDs &DefinitionIDs;
-            void Register(NamedDecl* ND, bool AddSingleEntry = true) {
+            void Register(const NamedDecl* ND, bool AddSingleEntry = true) {
                assert(ND->isFromASTFile());
                // FIXME: All decls should have an owning module once rootcling
                // updates its generated decls from within the LookupHelper & co.
@@ -1220,13 +1240,27 @@ static void RegisterCxxModules(cling::Interpreter &clingInterp)
       clang::CompilerInstance &CI = *clingInterp.getCI();
       GlobalModuleIndex *GlobalIndex = nullptr;
       // Conservatively enable platform by platform.
-      bool supportedPlatform = false;
-      // Allow forcefully enabling the GMI.
-      const char *experimentalGMI = gSystem->Getenv("ROOT_EXPERIMENTAL_GMI");
-      if (experimentalGMI && strcmp(experimentalGMI,"false") != 0)
-         supportedPlatform = true;
+      bool supportedPlatform =
+#ifdef R__LINUX
+         true
+#elif defined(R__MACOSX)
+         true
+#else // Windows
+         false
+#endif
+         ;
+      // Allow forcefully enabling/disabling the GMI.
+      llvm::Optional<std::string> envUseGMI = llvm::sys::Process::GetEnv("ROOT_USE_GMI");
+      if (envUseGMI.hasValue()) {
+         bool value = envUseGMI->empty() || ROOT::FoundationUtils::ConvertEnvValueToBool(*envUseGMI);
+
+         if (supportedPlatform == value)
+            ::Warning("TCling__RegisterCxxModules", "Global module index is%sused already!",
+                     (value) ? " " :" not ");
+         supportedPlatform = value;
+      }
 
-      if (supportedPlatform && !gSystem->Getenv("ROOT_DISABLE_GMI")) {
+      if (supportedPlatform) {
          loadGlobalModuleIndex(SourceLocation(), clingInterp);
          // FIXME: The ASTReader still calls loadGlobalIndex and loads the file
          // We should investigate how to suppress it completely.