Fix a dead lock between garbage colletion and sample-based tracing.

We have been observing some rare deadlock in run-test 099-vmdebug
between
- a thread doing garbage collection, which had disabled weak
  references access and was trying to (re-)acquire the mutator
  lock;
- a sampling profiling thread used in sample-based tracing, which
  had acquired the mutator lock exclusively and was blocking on a
  condition variable regarding weak references access.

This change prevents garbage collection from occurring when
sampling the thread stacks in order to avoid this deadlock.

Test: for i in $(seq 1 10); do art/test/testrunner/testrunner.py --ndebuggable --debuggable --host --64 --verbose --debug --ndebug --gcstress -t 099-vmdebug; done
Bug: 73624630
Change-Id: I73272c2b1d7aa2554a88bb128d3646193bd1db04

1 files changed, 6 insertions, 0 deletions

diff --git a/runtime/trace.cc b/runtime/trace.cc
index bdc6757c89..d97dcb5a3d 100644
--- a/runtime/trace.cc
+++ b/runtime/trace.cc
@@ -303,6 +303,12 @@ void* Trace::RunSamplingThread(void* arg) {
       }
     }
     {
+      // Avoid a deadlock between a thread doing garbage collection
+      // and the profile sampling thread, by blocking GC when sampling
+      // thread stacks (see b/73624630).
+      gc::ScopedGCCriticalSection gcs(self,
+                                      art::gc::kGcCauseInstrumentation,
+                                      art::gc::kCollectorTypeInstrumentation);
       ScopedSuspendAll ssa(__FUNCTION__);
       MutexLock mu(self, *Locks::thread_list_lock_);
       runtime->GetThreadList()->ForEach(GetSample, the_trace);