Skip to content

Conversation

@HJLeee
Copy link
Contributor

@HJLeee HJLeee commented Dec 7, 2023

When multiple threads enter Module::AllocateDynamicEntry(MethodTable *pMT), the address of m_pDynamicStaticsInfo might be changed and one of the threads could register pMT in the old array.

Here is a sample crash.

  • Callstack
#0  0xb604bc20 in raise () from /lib/libc.so.6
#1  0xb4889a04 in onSigsegv(int) () from /usr/share/dotnet.tizen/lib/libdotnet_plugin.so
#2  0xb398bb52 in invoke_previous_action (action=<optimized out>, code=11, siginfo=0xb4876c88, context=0xb4876d08, signalRestarts=255) at /usr/src/debug/coreclr-6.0.9-1.arm/src/coreclr/pal/src/exception/signal.cpp:415
#3  sigsegv_handler (code=11, siginfo=0xb4876c88, context=<optimized out>) at /usr/src/debug/coreclr-6.0.9-1.arm/src/coreclr/pal/src/exception/signal.cpp:627
#4  <signal handler called>
#5  0xb372936e in MethodTable::ContainsGenericVariables (this=0x0, methodVarsOnly=0) from /usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.9/libcoreclr.so
#6  MethodTable::IsClassPreInited (this=0x0) at /usr/src/debug/coreclr-6.0.9-1.arm/src/coreclr/vm/methodtable.cpp:2242
#7  MethodTable::CheckRunClassInitThrowing (this=0x0) at /usr/src/debug/coreclr-6.0.9-1.arm/src/coreclr/vm/methodtable.cpp:3492
#8  0xb37c80ac in JIT_ClassInitDynamicClass_Helper (pLocalModule=0xb3e095d8, dwDynamicClassDomainID=30) at /usr/src/debug/coreclr-6.0.9-1.arm/src/coreclr/vm/jithelpers.cpp:1496
#9  0xb2ace210 in ?? ()
  • Info from pLocalModule in frame 8
(gdb) p *pLocalModule
$2 = {m_pDomainFile = 0x17f2a48, m_pDynamicClassTable = {<Volatile<DomainLocalModule::DynamicClassInfo*>> = {m_val = 0xb2e163c0}, <No data fields>}, m_aDynamicEntries = {m_val = 32},
  m_pADThunkTable = {<Volatile<UMEntryThunk*>> = {m_val = 0x0}, <No data fields>}, m_pGCStatics = 0xb1900018, m_ModuleIndex = {m_dwIndex = 0}, m_pDataBlob = 0xb3e095f0 ""}

(gdb) p *pLocalModule.m_pDomainFile
$3 = {_vptr$DomainFile = 0xb3a9bee8 <vtable for DomainAssembly+8>, m_pDomain = 0x17cffc8, m_pFile = 0x17f2838, m_pOriginalFile = 0x0, m_pModule = 0xb485a000, m_level = FILE_ACTIVE, m_hExposedModuleObject = 2979009105,  m_pError = 0x0, m_notifyflags = 7, m_loading = 0, m_pDynamicMethodTable = 0x0, m_pUMThunkHash = 0x0, m_bDisableActivationCheck = 0, m_dwReasonForRejectingNativeImage = 0}

(gdb) p *pLocalModule.m_pDomainFile.m_pModule
$4 = {_vptr$Module = 0xb3a9b834 <vtable for Module+8>, m_pSimpleName = 0x3077e48c "System.Private.CoreLib", m_file = 0x17f2838, m_dwTransientFlags = {m_val = 2131985}, m_dwPersistedFlags = {m_val = 16446},
  m_pVASigCookieBlock = 0x0, m_pAssembly = 0x17f2b70, m_moduleRef = 637534208, m_Crst = {<CrstStatic> = {<CrstBase> = {m_criticalsection = {DebugInfo = 0x0, LockCount = 0, RecursionCount = 0, OwningThread = 0x0,
          SpinCount = 0, dwInitState = 1, csnds = {rgNativeDataStorage = '\000' <repeats 79 times>, pvAlign = 0x0}}, m_dwFlags = 3221225472}, <No data fields>}, <No data fields>},
  m_FixupCrst = {<CrstStatic> = {<CrstBase> = {m_criticalsection = {DebugInfo = 0x0, LockCount = 0, RecursionCount = 0, OwningThread = 0x0, SpinCount = 0, dwInitState = 1, csnds = {
            rgNativeDataStorage = '\000' <repeats 79 times>, pvAlign = 0x0}}, m_dwFlags = 3221225505}, <No data fields>}, <No data fields>}, m_pISymUnmanagedReader = 0x0,
  m_ISymUnmanagedReaderCrst = {<CrstStatic> = {<CrstBase> = {m_criticalsection = {DebugInfo = 0x0, LockCount = 0, RecursionCount = 0, OwningThread = 0x0, SpinCount = 0, dwInitState = 1, csnds = {
            rgNativeDataStorage = '\000' <repeats 79 times>, pvAlign = 0x0}}, m_dwFlags = 3221225488}, <No data fields>}, <No data fields>}, m_pIStreamSym = 0x0, m_LookupTableCrst = {<CrstStatic> = {<CrstBase> = {
        m_criticalsection = {DebugInfo = 0x0, LockCount = 0, RecursionCount = 0, OwningThread = 0x0, SpinCount = 0, dwInitState = 1, csnds = {rgNativeDataStorage = '\000' <repeats 79 times>, pvAlign = 0x0}},
        m_dwFlags = 3221225496}, <No data fields>}, <No data fields>}, m_TypeDefToMethodTableMap = {<LookupMapBase> = {pNext = 0x0, pTable = 0xb3e15000, dwCount = 2235, supportedFlags = 1}, <No data fields>},
  m_TypeRefToMethodTableMap = {<LookupMapBase> = {pNext = 0x0, pTable = 0xb3e172ec, dwCount = 1, supportedFlags = 0}, <No data fields>}, m_MethodDefToDescMap = {<LookupMapBase> = {pNext = 0x0, pTable = 0xb3e172f0,
      dwCount = 27784, supportedFlags = 0}, <No data fields>}, m_FieldDefToDescMap = {<LookupMapBase> = {pNext = 0x0, pTable = 0xb3e32510, dwCount = 8266, supportedFlags = 0}, <No data fields>},
  m_pMemberRefToDescHashTable = 0xb3e3cb74, m_GenericParamToDescMap = {<LookupMapBase> = {pNext = 0x0, pTable = 0xb3e3a638, dwCount = 2335, supportedFlags = 0}, <No data fields>},
  m_GenericTypeDefToCanonMethodTableMap = {<LookupMapBase> = {pNext = 0x0, pTable = 0xb3e3cab4, dwCount = 0, supportedFlags = 1}, <No data fields>}, m_FileReferencesMap = {<LookupMapBase> = {pNext = 0x0,
      pTable = 0xb3e3cab4, dwCount = 1, supportedFlags = 0}, <No data fields>}, m_ManifestModuleReferencesMap = {<LookupMapBase> = {pNext = 0x0, pTable = 0xb3e3cab8, dwCount = 1,
      supportedFlags = 0}, <No data fields>}, m_MethodDefToPropertyInfoMap = {<LookupMapBase> = {pNext = 0x0, pTable = 0xb3e3cabc, dwCount = 0, supportedFlags = 0}, <No data fields>}, m_pILStubCache = 0x0,
  m_DefaultDllImportSearchPathsAttributeValue = 0, m_pValidatedEmitter = {<Volatile<IMetaDataEmit*>> = {m_val = 0x0}, <No data fields>}, m_pAvailableClasses = 0x0, m_pAvailableParamTypes = 0xb3e3cabc,
  m_InstMethodHashTableCrst = {<CrstStatic> = {<CrstBase> = {m_criticalsection = {DebugInfo = 0x0, LockCount = 0, RecursionCount = 0, OwningThread = 0x0, SpinCount = 0, dwInitState = 1, csnds = {
            rgNativeDataStorage = '\000' <repeats 79 times>, pvAlign = 0x0}}, m_dwFlags = 3221225473}, <No data fields>}, <No data fields>}, m_pInstMethodHashTable = 0xb3e3cb30, m_dwDebuggerJMCProbeCount = 0,
  m_pAvailableClassesCaseIns = 0x0, m_pBinder = 0xb3abf180 <g_CoreLib>, m_pReadyToRunInfo = 0xb485a3f8, m_pNativeImage = 0x0, m_pProfilingBlobTable = 0x0, m_pProfileData = 0x0, m_nativeImageProfiling = 0,
  m_methodProfileList = 0x0, m_dwTypeCount = 2233, m_dwExportedTypeCount = 0, m_dwCustomAttributeCount = 14665, m_tokenProfileData = 0x0, m_propertyNameSet = 0x0, m_nPropertyNameSet = 0, m_pThunkHeap = 0x0,
  m_ModuleID = 0xb3e095d8, m_ModuleIndex = {m_dwIndex = 0}, m_pRegularStaticOffsets = 0xb485a554, m_pThreadStaticOffsets = 0xb3e05000, m_maxTypeRidStaticsAllocated = 2234, m_dwMaxGCRegularStaticHandles = 960,
  m_dwMaxGCThreadStaticHandles = 16, m_dwRegularStaticsBlockSize = 2652, m_dwThreadStaticsBlockSize = 2265, m_cDynamicEntries = 33, m_maxDynamicEntries = 64, m_pDynamicStaticsInfo = 0xb2e0cb28,
  m_debuggerSpecificData = {m_pDynamicILCrst = 0x0, m_pDynamicILBlobTable = 0x0, m_pTemporaryILBlobTable = 0x0, m_pILOffsetMappingTable = 0x0, m_cTotalJMCFuncs = 0, m_fDefaultJMCStatus = false},
  m_pPersistentInlineTrackingMapNGen = 0x0, m_pJitInlinerTrackingMap = 0x17f3340, m_AssemblyRefByNameTable = 0x0, m_AssemblyRefByNameCount = 0, m_NativeMetadataAssemblyRefMap = 0x0,
  m_DictionaryCrst = {<CrstStatic> = {<CrstBase> = {m_criticalsection = {DebugInfo = 0x0, LockCount = 0, RecursionCount = 0, OwningThread = 0x0, SpinCount = 0, dwInitState = 1, csnds = {
            rgNativeDataStorage = '\000' <repeats 79 times>, pvAlign = 0x0}}, m_dwFlags = 3221225472}, <No data fields>}, <No data fields>}}

(gdb) p *pLocalModule.m_pDomainFile.m_pModule.m_pDynamicStaticsInfo@32
$6 = {{pEnclosingMT = 0xb322ffa0}, {pEnclosingMT = 0xb3230288}, {pEnclosingMT = 0xb32307cc}, {pEnclosingMT = 0xb3216374}, {pEnclosingMT = 0xb321668c}, {pEnclosingMT = 0xb32178a4}, {pEnclosingMT = 0xb321ada0}, {
    pEnclosingMT = 0xb321dab8}, {pEnclosingMT = 0xb2ef1c08}, {pEnclosingMT = 0xb2ea2314}, {pEnclosingMT = 0xb2eaab44}, {pEnclosingMT = 0xb2e80814}, {pEnclosingMT = 0xb2e11944}, {pEnclosingMT = 0xb2e14874}, {
    pEnclosingMT = 0xb2e15378}, {pEnclosingMT = 0xb2e15648}, {pEnclosingMT = 0xb2e15da8}, {pEnclosingMT = 0xb2e16230}, {pEnclosingMT = 0xb2e1a6d8}, {pEnclosingMT = 0xb2e1abc4}, {pEnclosingMT = 0xb2e03aac}, {
    pEnclosingMT = 0xb2e05308}, {pEnclosingMT = 0xb2e05fd8}, {pEnclosingMT = 0xb2e077ec}, {pEnclosingMT = 0xb2e08498}, {pEnclosingMT = 0xb2e09144}, {pEnclosingMT = 0xb2e0a5b0}, {pEnclosingMT = 0xb2e0ac68}, {
    pEnclosingMT = 0xb2e0b374}, {pEnclosingMT = 0xb2e0ba80}, {pEnclosingMT = 0x0}, {pEnclosingMT = 0xb2e0c2d8}}

You can see that 30th item is {pEnclosingMT = 0x0}.
I think this happens the address of m_pDynamicStaticsInfo is not protected during array memcpy() and switching.

  • Thread 1 gets newId 30 --> context switching happens.
  • Thread 2 gets newId 31, allocate new array and copy the array --> context switching happens.
  • Thread 1 records pMT in the array.
  • Thread 2 switches the array with newly allocated one.

@ghost ghost added community-contribution Indicates that the PR has been added by a community member area-VM-coreclr labels Dec 7, 2023
@HJLeee
Copy link
Contributor Author

HJLeee commented Dec 7, 2023

I also removed VolatileLoad & VolatileStore in this PR as the problem description in #53556 looks race condition b/w threads.
Please review if my assumption is correct, otherwise I'll rollback them.

Copy link
Member

@jkotas jkotas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@jkotas jkotas merged commit 292d035 into dotnet:main Dec 8, 2023
@HJLeee HJLeee deleted the protect_array branch December 8, 2023 08:12
@github-actions github-actions bot locked and limited conversation to collaborators Jan 7, 2024
@davidwrighton
Copy link
Member

/backport to release/8.0-staging

@github-actions github-actions bot unlocked this conversation Jan 22, 2024
@github-actions
Copy link
Contributor

Started backporting to release/8.0-staging: https://github.com/dotnet/runtime/actions/runs/7618727880

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Jan 22, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

area-VM-coreclr community-contribution Indicates that the PR has been added by a community member

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants