Avoid eating SEGVs when performing on-demand dequicken

When using on-demand dequickening we will register SEGV handlers to
catch access to the dequickened dex-file. In order to prevent a
deadlock we need to be careful about how recursive SEGVs are handled.
We initially simply aborted if any occurred. While this is generally
correct it can obscure bugs by changing the stack-trace and possibly
break some tools which rely on being able to intercept SEGVs. To fix
this we will instead just pass-along SEGVs to later handlers when they
happen.

Test: ./test.py --host
Bug: 158737055
Change-Id: Ib5eebff54c2cd00565e5b1619a6794f1543f39e8

1 files changed, 10 insertions, 7 deletions

diff --git a/openjdkjvmti/transform.cc b/openjdkjvmti/transform.cc
index 613368525e..715a98c932 100644
--- a/openjdkjvmti/transform.cc
+++ b/openjdkjvmti/transform.cc
@@ -92,15 +92,18 @@ class TransformationFaultHandler final : public art::FaultHandler {
   bool Action(int sig, siginfo_t* siginfo, void* context ATTRIBUTE_UNUSED) override {
     DCHECK_EQ(sig, SIGSEGV);
     art::Thread* self = art::Thread::Current();
+    uintptr_t ptr = reinterpret_cast<uintptr_t>(siginfo->si_addr);
     if (UNLIKELY(uninitialized_class_definitions_lock_.IsExclusiveHeld(self))) {
-      if (self != nullptr) {
-        LOG(FATAL) << "Recursive call into Transformation fault handler!";
-        UNREACHABLE();
-      } else {
-        LOG(ERROR) << "Possible deadlock due to recursive signal delivery of segv.";
-      }
+      // It's possible this is just some other unrelated segv that should be
+      // handled separately, continue to later handlers. This is likely due to
+      // running out of memory somewhere along the FixedUpDexFile pipeline and
+      // is likely unrecoverable. By returning false here though we will get a
+      // better, more accurate, stack-trace later that points to the actual
+      // issue.
+      LOG(WARNING) << "Recursive SEGV occurred during Transformation dequickening at 0x" << std::hex
+                   << ptr;
+      return false;
     }
-    uintptr_t ptr = reinterpret_cast<uintptr_t>(siginfo->si_addr);
     ArtClassDefinition* res = nullptr;
 
     {