AppVerif found this issue "Microsoft.BuildXL.win-x64 / 0.1.0-20190912.1" in DetoursServices!Detoured_NtCreateFile

[malkia]

<avrf:logfile xmlns:avrf="Application Verifier">
<avrf:logSession TimeStarted="2019-11-18 : 03:15:27" PID="1888" Version="2">
<avrf:logEntry Time="2019-11-18 : 03:15:30" LayerName="Heaps" StopCode="0x13" Severity="Error">
avrf:messageFirst chance access violation for current stack trace.</avrf:message>
avrf:parameter110 - Invalid address causing the exception.</avrf:parameter1>
avrf:parameter27ff83c709c15 - Code address executing the invalid access.</avrf:parameter2>
avrf:parameter38f0e4b0 - Exception record.</avrf:parameter3>
avrf:parameter48f0dfc0 - Context record.</avrf:parameter4>
avrf:stackTrace
avrf:tracevrfcore!VerifierDisableVerifier+700 ( @ 0)</avrf:trace>
avrf:traceverifier!VerifierStopMessage+b9 ( @ 0)</avrf:trace>
avrf:tracentdll!RtlApplicationVerifierStop+96 ( @ 0)</avrf:trace>
avrf:tracevfbasics!+7ff836982669 ( @ 0)</avrf:trace>
avrf:tracevfbasics!+7ff83698335a ( @ 0)</avrf:trace>
avrf:tracevfbasics!+7ff8369829aa ( @ 0)</avrf:trace>
avrf:tracentdll!RtlInitializeCriticalSectionAndSpinCount+1c6 ( @ 0)</avrf:trace>
avrf:tracentdll!RtlWalkFrameChain+1119 ( @ 0)</avrf:trace>
avrf:tracentdll!KiUserExceptionDispatcher+2e ( @ 0)</avrf:trace>
avrf:traceDetoursServices!Detoured_NtCreateFile+75 (b:\public\src\sandbox\windows\detoursservices\detouredfunctions.cpp @ 5344)</avrf:trace>
avrf:traceDetoursServices!Detoured_NtOpenFile+3e (b:\public\src\sandbox\windows\detoursservices\detouredfunctions.cpp @ 5926)</avrf:trace>
avrf:traceKERNEL32!CreateActCtxWWorker+11b9 ( @ 0)</avrf:trace>
avrf:traceKERNEL32!CreateActCtxWWorker+39f ( @ 0)</avrf:trace>
avrf:traceKERNELBASE!CreateActCtxW+20 ( @ 0)</avrf:trace>
avrf:traceKERNEL32!QueryPerformanceCounter+112 ( @ 0)</avrf:trace>
avrf:tracentdll!RtlIsCriticalSectionLockedByThread+7ea ( @ 0)</avrf:trace>
avrf:tracentdll!RtlActivateActivationContextUnsafeFast+f3 ( @ 0)</avrf:trace>
avrf:tracentdll!RtlIsCriticalSectionLockedByThread+38b ( @ 0)</avrf:trace>
avrf:tracentdll!RtlImageNtHeader+3d5 ( @ 0)</avrf:trace>
avrf:tracentdll!RtlGetSuiteMask+28e ( @ 0)</avrf:trace>
avrf:tracentdll!RtlDosPathNameToNtPathName_U_WithStatus+3ef ( @ 0)</avrf:trace>
avrf:tracentdll!RtlDosPathNameToNtPathName_U_WithStatus+2b3 ( @ 0)</avrf:trace>
avrf:tracentdll!RtlInitializeCriticalSection+1f0 ( @ 0)</avrf:trace>
avrf:tracentdll!RtlReleaseSRWLockExclusive+694 ( @ 0)</avrf:trace>
avrf:traceKERNEL32!BaseThreadInitThunk+14 ( @ 0)</avrf:trace>
avrf:tracentdll!RtlUserThreadStart+21 ( @ 0)</avrf:trace>
</avrf:stackTrace>
</avrf:logEntry>
<avrf:logEntry Time="2019-11-18 : 03:15:30" LayerName="Heaps" StopCode="0x13" Severity="Error">
avrf:messageFirst chance access violation for current stack trace.</avrf:message>
avrf:parameter10 - Invalid address causing the exception.</avrf:parameter1>
avrf:parameter2140f05e7a - Code address executing the invalid access.</avrf:parameter2>
avrf:parameter38f0bab0 - Exception record.</avrf:parameter3>
avrf:parameter48f0b5c0 - Context record.</avrf:parameter4>
avrf:stackTrace
avrf:tracevrfcore!VerifierDisableVerifier+700 ( @ 0)</avrf:trace>
avrf:traceverifier!VerifierStopMessage+b9 ( @ 0)</avrf:trace>
avrf:tracentdll!RtlApplicationVerifierStop+96 ( @ 0)</avrf:trace>
avrf:tracevfbasics!+7ff836982669 ( @ 0)</avrf:trace>
avrf:tracevfbasics!+7ff83698335a ( @ 0)</avrf:trace>
avrf:tracevfbasics!+7ff8369829aa ( @ 0)</avrf:trace>
avrf:tracentdll!RtlInitializeCriticalSectionAndSpinCount+1c6 ( @ 0)</avrf:trace>
avrf:tracentdll!RtlWalkFrameChain+1119 ( @ 0)</avrf:trace>
avrf:tracentdll!KiUserExceptionDispatcher+2e ( @ 0)</avrf:trace>
avrf:traceSCRUBBED_APP!+140f05e7a ( @ 0)</avrf:trace>
avrf:traceSCRUBBED_APP!+14027a7ae ( @ 0)</avrf:trace>
avrf:traceSCRUBBED_APP!+1418a3bd8 ( @ 0)</avrf:trace>
avrf:traceSCRUBBED_APP!+1418adc7e ( @ 0)</avrf:trace>
avrf:traceSCRUBBED_APP!+1418b0f48 ( @ 0)</avrf:trace>
avrf:traceKERNELBASE!UnhandledExceptionFilter+1bc ( @ 0)</avrf:trace>
avrf:tracentdll!TpDbgDumpHeapUsage+129 ( @ 0)</avrf:trace>
avrf:tracentdll!TpDbgDumpHeapUsage+bda ( @ 0)</avrf:trace>
avrf:tracentdll!memset+136e ( @ 0)</avrf:trace>
avrf:tracentdll!_C_specific_handler+96 ( @ 0)</avrf:trace>
avrf:tracentdll!RtlRaiseStatus+7e2 ( @ 0)</avrf:trace>
avrf:tracentdll!_chkstk+11f ( @ 0)</avrf:trace>
avrf:tracentdll!RtlWalkFrameChain+14bf ( @ 0)</avrf:trace>
avrf:tracentdll!KiUserExceptionDispatcher+2e ( @ 0)</avrf:trace>
avrf:tracevrfcore!VerifierStopMessageEx+7dc ( @ 0)</avrf:trace>
avrf:tracevrfcore!VerifierDisableVerifier+700 ( @ 0)</avrf:trace>
avrf:traceverifier!VerifierStopMessage+b9 ( @ 0)</avrf:trace>
avrf:tracentdll!RtlApplicationVerifierStop+96 ( @ 0)</avrf:trace>
avrf:tracevfbasics!+7ff836982669 ( @ 0)</avrf:trace>
avrf:tracevfbasics!+7ff83698335a ( @ 0)</avrf:trace>
avrf:tracevfbasics!+7ff8369829aa ( @ 0)</avrf:trace>
avrf:tracentdll!RtlInitializeCriticalSectionAndSpinCount+1c6 ( @ 0)</avrf:trace>
avrf:tracentdll!RtlWalkFrameChain+1119 ( @ 0)</avrf:trace>
avrf:tracentdll!KiUserExceptionDispatcher+2e ( @ 0)</avrf:trace>
avrf:traceDetoursServices!Detoured_NtCreateFile+75 (b:\public\src\sandbox\windows\detoursservices\detouredfunctions.cpp @ 5344)</avrf:trace>
avrf:traceDetoursServices!Detoured_NtOpenFile+3e (b:\public\src\sandbox\windows\detoursservices\detouredfunctions.cpp @ 5926)</avrf:trace>
avrf:traceKERNEL32!CreateActCtxWWorker+11b9 ( @ 0)</avrf:trace>
avrf:traceKERNEL32!CreateActCtxWWorker+39f ( @ 0)</avrf:trace>
avrf:traceKERNELBASE!CreateActCtxW+20 ( @ 0)</avrf:trace>
avrf:traceKERNEL32!QueryPerformanceCounter+112 ( @ 0)</avrf:trace>
avrf:tracentdll!RtlIsCriticalSectionLockedByThread+7ea ( @ 0)</avrf:trace>
avrf:tracentdll!RtlActivateActivationContextUnsafeFast+f3 ( @ 0)</avrf:trace>
avrf:tracentdll!RtlIsCriticalSectionLockedByThread+38b ( @ 0)</avrf:trace>
avrf:tracentdll!RtlImageNtHeader+3d5 ( @ 0)</avrf:trace>
avrf:tracentdll!RtlGetSuiteMask+28e ( @ 0)</avrf:trace>
avrf:tracentdll!RtlDosPathNameToNtPathName_U_WithStatus+3ef ( @ 0)</avrf:trace>
avrf:tracentdll!RtlDosPathNameToNtPathName_U_WithStatus+2b3 ( @ 0)</avrf:trace>
avrf:tracentdll!RtlInitializeCriticalSection+1f0 ( @ 0)</avrf:trace>
avrf:tracentdll!RtlReleaseSRWLockExclusive+694 ( @ 0)</avrf:trace>
avrf:traceKERNEL32!BaseThreadInitThunk+14 ( @ 0)</avrf:trace>
avrf:tracentdll!RtlUserThreadStart+21 ( @ 0)</avrf:trace>
</avrf:stackTrace>
</avrf:logEntry>
</avrf:logSession>
</avrf:logfile>

[malkia]

FYI - Investigating now, but seem like enabling "Locks" check in AppVerif may cause this. So far just enabling "Heaps" checking does not. I'll have more as time goes on.

It maybe, after all that AppVerif + DetoursService.dll - may not work as expected - it'll be good to know if Microsoft themselves (or other company using BuildXL) have run into this?

[malkia]

Some more findings. Our AppVerif was set with:

C:\Windows\System32\appverif.exe -enable Heaps Locks Memory TLS DirtyStacks Threadpool Leak SRWLock Cuzz -for SCRUBBED_APP.exe

I was able to narrow down a successful run, if I remove all of these: Locks, Memory, TLS, DirtyStacks, Threadpool, Leak and SRWLock

With this command line it works:

C:\Windows\System32\appverif.exe -enable Heaps Cuzz -for SCRUBBED_APP.exe

There are other options to try out, but these were not enabled for us anyway.
I have a feeling, that anything which AppVerif needs to further hook somehow messes up with DetoursService.dll used by BuildXL