[cxxmodules] Refactor LoadCoreModules. #1394

Teemperor · 2017-11-29T10:35:49Z

Refactored LoadCoreModules that it now prints a warning if a module isn't found. Also fixes some nullptr-derefs from the old code where we accessed pointers before checking if they're null.

phsft-bot · 2017-11-29T10:36:02Z

Starting build on centos7/gcc49, mac1012/native, slc6/gcc49, slc6/gcc62, slc6/gcc62, ubuntu14/native, ubuntu14/native, windows10/vc15 with flags -Dvc=OFF -Dimt=ON -Dccache=ON
How to customize builds

phsft-bot · 2017-11-29T10:39:10Z

Build failed on ubuntu14/native.
See console output.

Errors:

/mnt/vdb/lsf/workspace/root-pullrequests-build/root/core/metacling/src/TCling.cxx:1130:11: error: ‘CI’ was not declared in this scope

phsft-bot · 2017-11-29T10:40:13Z

Build failed on mac1012/native.
See console output.

Errors:

/Volumes/MacintoshHD/build/jenkins/workspace/root-pullrequests-build/root/core/metacling/src/TCling.cxx:1130:11: error: use of undeclared identifier 'CI'

phsft-bot · 2017-11-29T10:41:50Z

Build failed on slc6/gcc62.
See console output.

Errors:

../root/core/metacling/src/TCling.cxx:1130:11: error: ‘CI’ was not declared in this scope

phsft-bot · 2017-11-29T10:42:50Z

Build failed on slc6/gcc49.
See console output.

Errors:

../root/core/metacling/src/TCling.cxx:1130:11: error: ‘CI’ was not declared in this scope

phsft-bot · 2017-11-29T10:43:55Z

Build failed on centos7/gcc49.
See console output.

Errors:

../root/core/metacling/src/TCling.cxx:1130:11: error: ‘CI’ was not declared in this scope

phsft-bot · 2017-11-29T10:53:03Z

Starting build on centos7/gcc49, mac1012/native, slc6/gcc49, slc6/gcc62, slc6/gcc62, ubuntu14/native, ubuntu14/native, windows10/vc15 with flags -Dvc=OFF -Dimt=ON -Dccache=ON
How to customize builds

Teemperor · 2017-11-29T10:54:20Z

@phsft-bot build just on slc6/gcc62 with flags -Druntime_cxxmodules=On -Dctest_test_exclude_none=on

phsft-bot · 2017-11-29T10:54:57Z

Starting build on slc6/gcc62 with flags -Dvc=OFF -Dimt=ON -Dccache=ON -Druntime_cxxmodules=On -Dctest_test_exclude_none=on
How to customize builds

phsft-bot · 2017-11-29T11:59:49Z

Build failed on slc6/gcc62.
See console output.

Errors:

293/1651 Test Fix GCC 6 compilation (tested on ArchLinux) #188: tutorial-dataframe-tdf013_InspectAnalysis .........................................................***Failed Error regular expression found in output. Regex=[: error:] 3.21 sec
544/1651 Test [cxxmodules] Don't use the VFS overlay for the dictionaries #447: tutorial-math-quasirandom .........................................................................***Failed Error regular expression found in output. Regex=[: error:] 2.00 sec
570/1651 Test [CLANG] Increase default value for -ftemplate-depth to 1024 #478: tutorial-multicore-mtbb201_parallelHistoFill ......................................................***Failed Error regular expression found in output. Regex=[: error:] 4.14 sec
685/1651 Test Add 1-vs-rest and 1-vs-1 performance measures to TMVA multiclass output #597: tutorial-roostats-Zbi_Zgamma ......................................................................***Failed Error regular expression found in output. Regex=[: error:] 7.62 sec
698/1651 Test Supress R warnings in ROOT build #609: tutorial-roostats-rs_numbercountingutils ..........................................................***Failed Error regular expression found in output. Regex=[: error:] 4.35 sec
732/1651 Test Several fixes and improvements for the vectorized implementations #649: tutorial-tmva-keras-LaunchClasGUI .................................................................***Failed Error regular expression found in output. Regex=[: error:] 1.66 sec
734/1651 Test Fixing an O(N^2) scaling problem caused by TTree::Draw() #650: tutorial-tmva-keras-LaunchMultiGUI ................................................................***Failed Error regular expression found in output. Regex=[: error:] 7.82 sec
736/1651 Test Fix buffer overflow. #651: tutorial-tmva-keras-LaunchRegGUI ..................................................................***Failed Error regular expression found in output. Regex=[: error:] 8.51 sec
776/1651 Test Always generate a ROOT modulemap and install it. #700: tutorial-dataframe-tdf002_dataModel-py ............................................................***Failed Error regular expression found in output. Regex=[: error:] 3.49 sec

Warnings:

Warning in <GenerateModule>: warning: Couldn't find the following specified headers in the module Core:
Warning in <GenerateModule>: warning: Couldn't find the following specified headers in the module valarrayDict:
Warning in <GenerateModule>: warning: Couldn't find the following specified headers in the module setDict:
Warning in <GenerateModule>: warning: Couldn't find the following specified headers in the module dequeDict:
Warning in <GenerateModule>: warning: Couldn't find the following specified headers in the module multimapDict:
Warning in <GenerateModule>: warning: Couldn't find the following specified headers in the module multisetDict:
Warning in <GenerateModule>: warning: Couldn't find the following specified headers in the module multimap2Dict:
Warning in <GenerateModule>: warning: Couldn't find the following specified headers in the module unordered_multisetDict:
Warning in <GenerateModule>: warning: Couldn't find the following specified headers in the module listDict:
Warning in <GenerateModule>: warning: Couldn't find the following specified headers in the module vectorDict:

And 7 more

Failing tests:

And 274 more

phsft-bot · 2017-11-29T21:38:19Z

Starting build on slc6/gcc62 with flags -Dvc=OFF -Dimt=ON -Dccache=ON -Druntime_cxxmodules=On -Dctest_test_exclude_none=on
How to customize builds

phsft-bot · 2017-11-29T22:32:14Z

Build failed on slc6/gcc62.
See console output.

Errors:

295/1653 Test documentation: fix legacy definition name #190: tutorial-dataframe-tdf013_InspectAnalysis .........................................................***Failed Error regular expression found in output. Regex=[: error:] 1.52 sec
545/1653 Test [TDF] Add ranges to user guide + a little bit of code formatting #449: tutorial-math-quasirandom .........................................................................***Failed Error regular expression found in output. Regex=[: error:] 0.92 sec
573/1653 Test Fixes to 10 coverity defect reports #480: tutorial-multicore-mtbb201_parallelHistoFill ......................................................***Failed Error regular expression found in output. Regex=[: error:] 3.46 sec
687/1653 Test Utilize git submodules for roottest #599: tutorial-roostats-Zbi_Zgamma ......................................................................***Failed Error regular expression found in output. Regex=[: error:] 1.66 sec
699/1653 Test Resolve warnings reported by -Wnon-virtual-dtor #611: tutorial-roostats-rs_numbercountingutils ..........................................................***Failed Error regular expression found in output. Regex=[: error:] 1.35 sec
734/1653 Test Fix buffer overflow. #651: tutorial-tmva-keras-LaunchClasGUI .................................................................***Failed Error regular expression found in output. Regex=[: error:] 2.20 sec
736/1653 Test Increased tolerance #652: tutorial-tmva-keras-LaunchMultiGUI ................................................................***Failed Error regular expression found in output. Regex=[: error:] 7.07 sec
737/1653 Test Make cling build again on Windows (following the recent changes). To … #653: tutorial-tmva-keras-LaunchRegGUI ..................................................................***Failed Error regular expression found in output. Regex=[: error:] 7.17 sec
777/1653 Test cling PR 162 #702: tutorial-dataframe-tdf002_dataModel-py ............................................................***Failed Error regular expression found in output. Regex=[: error:] 2.13 sec

Warnings:

Warning in <GenerateModule>: warning: Couldn't find the following specified headers in the module Core:
Warning in <GenerateModule>: warning: Couldn't find the following specified headers in the module map2Dict:
Warning in <GenerateModule>: warning: Couldn't find the following specified headers in the module dequeDict:
Warning in <GenerateModule>: warning: Couldn't find the following specified headers in the module listDict:
Warning in <GenerateModule>: warning: Couldn't find the following specified headers in the module valarrayDict:
Warning in <GenerateModule>: warning: Couldn't find the following specified headers in the module multisetDict:
Warning in <GenerateModule>: warning: Couldn't find the following specified headers in the module setDict:
Warning in <GenerateModule>: warning: Couldn't find the following specified headers in the module vectorDict:
Warning in <GenerateModule>: warning: Couldn't find the following specified headers in the module unordered_setDict:
Warning in <GenerateModule>: warning: Couldn't find the following specified headers in the module multimapDict:

And 7 more

Failing tests:

And 274 more

Teemperor · 2017-11-30T09:06:17Z

Uff, clang format doesn't like the RegisterModule way of doing code formatting...

phsft-bot · 2017-11-30T09:07:15Z

Starting build on slc6/gcc62 with flags -Dvc=OFF -Dimt=ON -Dccache=ON -Druntime_cxxmodules=On -Dctest_test_exclude_none=on
How to customize builds

phsft-bot · 2017-11-30T09:20:02Z

Starting build on slc6/gcc62 with flags -Dvc=OFF -Dimt=ON -Dccache=ON -Druntime_cxxmodules=On -Dctest_test_exclude_none=on
How to customize builds

phsft-bot · 2017-11-30T10:17:03Z

Build failed on slc6/gcc62.
See console output.

Errors:

297/1653 Test documentation: fix legacy definition name #190: tutorial-dataframe-tdf013_InspectAnalysis .........................................................***Failed Error regular expression found in output. Regex=[: error:] 1.26 sec
549/1653 Test [TDF] Add ranges to user guide + a little bit of code formatting #449: tutorial-math-quasirandom .........................................................................***Failed Error regular expression found in output. Regex=[: error:] 1.65 sec
573/1653 Test Fixes to 10 coverity defect reports #480: tutorial-multicore-mtbb201_parallelHistoFill ......................................................***Failed Error regular expression found in output. Regex=[: error:] 4.49 sec
687/1653 Test Utilize git submodules for roottest #599: tutorial-roostats-Zbi_Zgamma ......................................................................***Failed Error regular expression found in output. Regex=[: error:] 1.83 sec
700/1653 Test Resolve warnings reported by -Wnon-virtual-dtor #611: tutorial-roostats-rs_numbercountingutils ..........................................................***Failed Error regular expression found in output. Regex=[: error:] 1.72 sec
734/1653 Test Fix buffer overflow. #651: tutorial-tmva-keras-LaunchClasGUI .................................................................***Failed Error regular expression found in output. Regex=[: error:] 0.91 sec
736/1653 Test Increased tolerance #652: tutorial-tmva-keras-LaunchMultiGUI ................................................................***Failed Error regular expression found in output. Regex=[: error:] 0.91 sec
738/1653 Test Make cling build again on Windows (following the recent changes). To … #653: tutorial-tmva-keras-LaunchRegGUI ..................................................................***Failed Error regular expression found in output. Regex=[: error:] 2.03 sec
777/1653 Test cling PR 162 #702: tutorial-dataframe-tdf002_dataModel-py ............................................................***Failed Error regular expression found in output. Regex=[: error:] 2.02 sec

Warnings:

Warning in <GenerateModule>: warning: Couldn't find the following specified headers in the module Core:
Warning in <GenerateModule>: warning: Couldn't find the following specified headers in the module listDict:
Warning in <GenerateModule>: warning: Couldn't find the following specified headers in the module dequeDict:
Warning in <GenerateModule>: warning: Couldn't find the following specified headers in the module valarrayDict:
Warning in <GenerateModule>: warning: Couldn't find the following specified headers in the module multimapDict:
Warning in <GenerateModule>: warning: Couldn't find the following specified headers in the module vectorDict:
Warning in <GenerateModule>: warning: Couldn't find the following specified headers in the module multisetDict:
Warning in <GenerateModule>: warning: Couldn't find the following specified headers in the module setDict:
Warning in <GenerateModule>: warning: Couldn't find the following specified headers in the module unordered_setDict:
Warning in <GenerateModule>: warning: Couldn't find the following specified headers in the module unordered_multimapDict:

And 7 more

Failing tests:

And 48 more

Teemperor · 2017-12-04T10:31:08Z

Additional runtime modules failures are because of the (legitimate) warning that we can't find modules sometimes. Will be fixed in a future PR where we add more paths to clang's PrebuiltModulePaths.

vgvassilev

LGTM!

If we generate a dictionary with a C++ module with rootcling we should write in the payload that we generated a C++ module. This way we can actually print a *reliable* warning when we fail to load the C++ module during runtime. Without this we can only speculate if a missing C++ module is intentional.

This refactors the LoadCoreModules function into a LoadModule function that centralizes the loading/diagnosing module loading in TCling. It also now prints out warnings when a module can't be found, which brings diagnostic to the module loading in `RegisterModule`.

phsft-bot · 2017-12-13T16:57:38Z

Starting build on centos7/gcc49, mac1012/native, slc6/gcc49, slc6/gcc62, slc6/gcc62, ubuntu14/native, ubuntu14/native, windows10/vc15 with flags -Dvc=OFF -Dimt=ON -Dccache=ON
How to customize builds

Teemperor · 2017-12-14T07:37:21Z

@phsft-bot build just on slc6/clang_gcc62 with flags -Dcxxmodules=Off -Druntime_cxxmodules=On -Dctest_test_exclude_none=on

phsft-bot · 2017-12-14T07:41:36Z

Starting build on slc6/clang_gcc62 with flags -Dvc=OFF -Dimt=ON -Dccache=ON -Dcxxmodules=Off -Druntime_cxxmodules=On -Dctest_test_exclude_none=on
How to customize builds

phsft-bot · 2017-12-14T08:05:15Z

Build failed on slc6/clang_gcc62.
See console output.

Teemperor · 2017-12-14T08:15:33Z

@phsft-bot build just on centos7/gcc62 with flags -Druntime_cxxmodules=On -Dctest_test_exclude_none=on

phsft-bot · 2017-12-14T08:15:41Z

Starting build on centos7/gcc62 with flags -Dvc=OFF -Dimt=ON -Dccache=ON -Druntime_cxxmodules=On -Dctest_test_exclude_none=on
How to customize builds

phsft-bot · 2017-12-14T08:15:46Z

Build failed on centos7/gcc62.
See console output.

Teemperor · 2017-12-14T08:18:08Z

@phsft-bot build just on centos7/gcc62 with flags -Druntime_cxxmodules=On -Dctest_test_exclude_none=on

phsft-bot · 2017-12-14T08:18:14Z

Starting build on centos7/gcc62 with flags -Dvc=OFF -Dimt=ON -Dccache=ON -Druntime_cxxmodules=On -Dctest_test_exclude_none=on
How to customize builds

phsft-bot · 2017-12-14T08:18:18Z

Build failed on centos7/gcc62.
See console output.

Teemperor · 2017-12-14T08:21:38Z

@phsft-bot build just on slc6/gcc62 with flags -Druntime_cxxmodules=On -Dctest_test_exclude_none=on

phsft-bot · 2017-12-14T08:21:44Z

Starting build on slc6/gcc62 with flags -Dvc=OFF -Dimt=ON -Dccache=ON -Druntime_cxxmodules=On -Dctest_test_exclude_none=on
How to customize builds

Teemperor · 2017-12-14T12:12:54Z

@phsft-bot build just on slc6/gcc62 with flags -Druntime_cxxmodules=On -Dctest_test_exclude_none=on -Droofit=Off

phsft-bot · 2017-12-14T12:13:01Z

Starting build on slc6/gcc62 with flags -Dvc=OFF -Dimt=ON -Dccache=ON -Druntime_cxxmodules=On -Dctest_test_exclude_none=on -Droofit=Off
How to customize builds

Teemperor · 2017-12-14T12:18:43Z

@phsft-bot build just on slc6/gcc62 with flags -Druntime_cxxmodules=On -Dctest_test_exclude_none=on -Droofit=Off

phsft-bot · 2017-12-14T12:18:50Z

Starting build on slc6/gcc62 with flags -Dvc=OFF -Dimt=ON -Dccache=ON -Druntime_cxxmodules=On -Dctest_test_exclude_none=on -Droofit=Off
How to customize builds

phsft-bot · 2017-12-14T13:37:35Z

Build failed on slc6/gcc62.
See console output.

Errors:

289/1523 Test Update PR of Oliver #182: tutorial-dataframe-tdf013_InspectAnalysis .........................................................***Failed Error regular expression found in output. Regex=[: error:] 0.91 sec
539/1523 Test Replaced <math.h> with <cmath> to fix clang compilation. #440: tutorial-math-quasirandom .........................................................................***Failed Error regular expression found in output. Regex=[: error:] 1.24 sec
564/1523 Test Make TSeq iterator standard-compliant #471: tutorial-multicore-mtbb201_parallelHistoFill ......................................................***Failed Error regular expression found in output. Regex=[: error:] 3.34 sec
614/1523 Test Coverity fixes 82576, 82575, 68746, 68745, 68856, 61279, 61577 #526: tutorial-tmva-keras-LaunchClasGUI .................................................................***Failed Error regular expression found in output. Regex=[: error:] 0.92 sec
615/1523 Test Print a warning when rootcling can't find the ROOT modulemap. #527: tutorial-tmva-keras-LaunchMultiGUI ................................................................***Failed Error regular expression found in output. Regex=[: error:] 0.51 sec
616/1523 Test Be more specific on what is a header file to be moved and installed #528: tutorial-tmva-keras-LaunchRegGUI ..................................................................***Failed Error regular expression found in output. Regex=[: error:] 1.03 sec
657/1523 Test Provide actions for text-based streamers #578: tutorial-dataframe-tdf002_dataModel-py ............................................................***Failed Error regular expression found in output. Regex=[: error:] 1.36 sec

Warnings:

Warning in <GenerateModule>: warning: Couldn't find the following specified headers in the module Core:
Warning in <GenerateModule>: warning: Couldn't find the following specified headers in the module listDict:
Warning in <GenerateModule>: warning: Couldn't find the following specified headers in the module valarrayDict:
Warning in <GenerateModule>: warning: Couldn't find the following specified headers in the module unordered_setDict:
Warning in <GenerateModule>: warning: Couldn't find the following specified headers in the module unordered_multisetDict:
Warning in <GenerateModule>: warning: Couldn't find the following specified headers in the module multimap2Dict:
Warning in <GenerateModule>: warning: Couldn't find the following specified headers in the module map2Dict:
Warning in <GenerateModule>: warning: Couldn't find the following specified headers in the module mapDict:
Warning in <GenerateModule>: warning: Couldn't find the following specified headers in the module unordered_mapDict:
Warning in <GenerateModule>: warning: Couldn't find the following specified headers in the module setDict:

And 7 more

Failing tests:

And 55 more

amadio · 2018-01-29T10:16:45Z

What is the status here? Should this be updated and merged, or closed?

Teemperor · 2018-01-29T19:42:32Z

Well, it mostly just exposes a bug more visible (the lack of codegen when loading code with LoadModule) which causes more tests to fail. And fixing codegen problems isn't very straightforward :)

vgvassilev · 2018-02-16T21:28:01Z

@Teemperor can you rebase or shall I?

vgvassilev · 2018-02-23T22:44:48Z

That's PR #1665.

Teemperor assigned vgvassilev Nov 29, 2017

Teemperor requested a review from vgvassilev November 29, 2017 10:35

Teemperor requested a review from pcanal as a code owner November 29, 2017 10:35

This was referenced Nov 29, 2017

[cxxmodules] Also preload TMVA/TreePlayer/Graf #1396

Closed

Refactor load module #1392

Closed

Teemperor force-pushed the RemoveLoadCoreModules branch from fdaf551 to 5b3c6d5 Compare November 29, 2017 10:52

vgvassilev mentioned this pull request Nov 29, 2017

[cxxmodules] Allow rootcling to load system modules using the overlays. #1395

Closed

Teemperor force-pushed the RemoveLoadCoreModules branch from 5b3c6d5 to 9dd4fec Compare November 29, 2017 21:38

Teemperor force-pushed the RemoveLoadCoreModules branch from 9dd4fec to a49e8db Compare November 30, 2017 09:03

Teemperor requested a review from amadio as a code owner November 30, 2017 09:03

Teemperor changed the title ~~[cxxmodules] Refactor LoadCoreModules.~~ [testing][cxxmodules] Refactor LoadCoreModules. Nov 30, 2017

Teemperor added the do not merge label Nov 30, 2017

Teemperor force-pushed the RemoveLoadCoreModules branch from a49e8db to ff9a4ad Compare November 30, 2017 09:05

Teemperor force-pushed the RemoveLoadCoreModules branch from ff9a4ad to bb286fe Compare November 30, 2017 09:19

Teemperor removed the do not merge label Dec 4, 2017

vgvassilev approved these changes Dec 4, 2017

View reviewed changes

Teemperor and others added 2 commits December 13, 2017 17:08

Teemperor force-pushed the RemoveLoadCoreModules branch from 97fbd59 to 76d87f1 Compare December 13, 2017 16:57

Teemperor changed the title ~~[testing][cxxmodules] Refactor LoadCoreModules.~~ [cxxmodules] Refactor LoadCoreModules. Dec 13, 2017

Teemperor mentioned this pull request Jan 15, 2018

[cxxmodules] Only load modules if one was generated #1511

Merged

vgvassilev closed this Feb 23, 2018

[cxxmodules] Refactor LoadCoreModules. #1394

[cxxmodules] Refactor LoadCoreModules. #1394

Uh oh!

Conversation

Teemperor commented Nov 29, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

phsft-bot commented Nov 29, 2017

Uh oh!

phsft-bot commented Nov 29, 2017

Errors:

Uh oh!

phsft-bot commented Nov 29, 2017

Errors:

Uh oh!

phsft-bot commented Nov 29, 2017

Errors:

Uh oh!

phsft-bot commented Nov 29, 2017

Errors:

Uh oh!

phsft-bot commented Nov 29, 2017

Errors:

Uh oh!

phsft-bot commented Nov 29, 2017

Uh oh!

Teemperor commented Nov 29, 2017

Uh oh!

phsft-bot commented Nov 29, 2017

Uh oh!

phsft-bot commented Nov 29, 2017

Errors:

Warnings:

Failing tests:

Uh oh!

phsft-bot commented Nov 29, 2017

Uh oh!

phsft-bot commented Nov 29, 2017

Errors:

Warnings:

Failing tests:

Uh oh!

Teemperor commented Nov 30, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

phsft-bot commented Nov 30, 2017

Uh oh!

phsft-bot commented Nov 30, 2017

Uh oh!

phsft-bot commented Nov 30, 2017

Errors:

Warnings:

Failing tests:

Uh oh!

Teemperor commented Dec 4, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vgvassilev left a comment

Choose a reason for hiding this comment

Uh oh!

phsft-bot commented Dec 13, 2017

Uh oh!

Teemperor commented Dec 14, 2017

Uh oh!

phsft-bot commented Dec 14, 2017

Uh oh!

phsft-bot commented Dec 14, 2017

Uh oh!

Teemperor commented Dec 14, 2017

Uh oh!

phsft-bot commented Dec 14, 2017

Uh oh!

phsft-bot commented Dec 14, 2017

Uh oh!

Teemperor commented Dec 14, 2017

Uh oh!

phsft-bot commented Dec 14, 2017

Uh oh!

phsft-bot commented Dec 14, 2017

Uh oh!

Teemperor commented Nov 29, 2017 •

edited

Loading

Teemperor commented Nov 30, 2017 •

edited

Loading

Teemperor commented Dec 4, 2017 •

edited

Loading