Reduce pauses for weak reference access

Remove the "preserve references" machinery. Instead let the reference
processor inform GetReferent about its state, so that it can leverage
knowledge about whether reachable memory has in fact been completely
marked.

Restructure the ReferenceProcessor interface by adding Setup to
ensure that ReferenceProcessor fields are properly set up before
we disable the normal fast path through GetReferent.

For the CC collector, forward essentially all SoftReferences as
part of normal marking, so we don't stall weak reference access
for those.

Note briefly in the log if we encounter References that are only
reachable from finalizers.

SS and MS collectors are only minimally updated to keep them working.

We now block in GetReferent only for the hopefully very brief period of
marking objects that were initially missed as a result of a mutator
collector race. This should hopefully eliminate multi-millisecond delays
here. For 2043-reference-pauses from aosp/1952438, it reduces blocking
from over 100 msecs to under 1 on host. This is mostly due to the
better SoftReference treatment; 100 msec pauses in GetReferent()
were never near-typical.

We iteratively mark through SoftReferences now. Previously we could
mistakenly clear SoftReferences discovered while marking from the top
level ones. (Lokesh pointed this out.) To make this work, we change
ForwardSoftReferences to actually remove References from the queue,
as the comment always said it did.

This also somewhat prepares us for a much less complete solution for
pauses to access WeakGlobalRefs or other "system weaks".

This fixes a memory ordering issue for the per-thread weak reference
access flags used with the CC collector. I think the issue is still
there for the CMS collector. That requires further discussion.

Bug: 190867430
Bug: 189738006
Bug: 211784084

Test: Build and boot aosp & Treehugger; aosp/195243

Change-Id: I02f12ac481db4c4e400d253662a7a126318d4bec

4 files changed, 10 insertions, 7 deletions

diff --git a/compiler/optimizing/intrinsics_arm64.cc b/compiler/optimizing/intrinsics_arm64.cc
index 77b55e455e..0236f0d5a9 100644
--- a/compiler/optimizing/intrinsics_arm64.cc
+++ b/compiler/optimizing/intrinsics_arm64.cc
@@ -3475,7 +3475,8 @@ void IntrinsicCodeGeneratorARM64::VisitReferenceGetReferent(HInvoke* invoke) {
     Register temp = temps.AcquireW();
     __ Ldr(temp,
            MemOperand(tr, Thread::WeakRefAccessEnabledOffset<kArm64PointerSize>().Uint32Value()));
-    __ Cbz(temp, slow_path->GetEntryLabel());
+    static_assert(enum_cast<int32_t>(WeakRefAccessState::kVisiblyEnabled) == 0);
+    __ Cbnz(temp, slow_path->GetEntryLabel());
   }
 
   {
diff --git a/compiler/optimizing/intrinsics_arm_vixl.cc b/compiler/optimizing/intrinsics_arm_vixl.cc
index a4a3457c37..303ac171a7 100644
--- a/compiler/optimizing/intrinsics_arm_vixl.cc
+++ b/compiler/optimizing/intrinsics_arm_vixl.cc
@@ -2517,8 +2517,8 @@ void IntrinsicCodeGeneratorARMVIXL::VisitReferenceGetReferent(HInvoke* invoke) {
     vixl32::Register temp = temps.Acquire();
     __ Ldr(temp,
            MemOperand(tr, Thread::WeakRefAccessEnabledOffset<kArmPointerSize>().Uint32Value()));
-    __ Cmp(temp, 0);
-    __ B(eq, slow_path->GetEntryLabel());
+    __ Cmp(temp, enum_cast<int32_t>(WeakRefAccessState::kVisiblyEnabled));
+    __ B(ne, slow_path->GetEntryLabel());
   }
 
   {
diff --git a/compiler/optimizing/intrinsics_x86.cc b/compiler/optimizing/intrinsics_x86.cc
index 7c2537495a..3a3886432a 100644
--- a/compiler/optimizing/intrinsics_x86.cc
+++ b/compiler/optimizing/intrinsics_x86.cc
@@ -3346,8 +3346,9 @@ void IntrinsicCodeGeneratorX86::VisitReferenceGetReferent(HInvoke* invoke) {
   if (kEmitCompilerReadBarrier) {
     // Check self->GetWeakRefAccessEnabled().
     ThreadOffset32 offset = Thread::WeakRefAccessEnabledOffset<kX86PointerSize>();
-    __ fs()->cmpl(Address::Absolute(offset), Immediate(0));
-    __ j(kEqual, slow_path->GetEntryLabel());
+    __ fs()->cmpl(Address::Absolute(offset),
+                  Immediate(enum_cast<int32_t>(WeakRefAccessState::kVisiblyEnabled)));
+    __ j(kNotEqual, slow_path->GetEntryLabel());
   }
 
   // Load the java.lang.ref.Reference class, use the output register as a temporary.
diff --git a/compiler/optimizing/intrinsics_x86_64.cc b/compiler/optimizing/intrinsics_x86_64.cc
index d5a7cb10e1..e3be98732b 100644
--- a/compiler/optimizing/intrinsics_x86_64.cc
+++ b/compiler/optimizing/intrinsics_x86_64.cc
@@ -3098,8 +3098,9 @@ void IntrinsicCodeGeneratorX86_64::VisitReferenceGetReferent(HInvoke* invoke) {
   if (kEmitCompilerReadBarrier) {
     // Check self->GetWeakRefAccessEnabled().
     ThreadOffset64 offset = Thread::WeakRefAccessEnabledOffset<kX86_64PointerSize>();
-    __ gs()->cmpl(Address::Absolute(offset, /* no_rip= */ true), Immediate(0));
-    __ j(kEqual, slow_path->GetEntryLabel());
+    __ gs()->cmpl(Address::Absolute(offset, /* no_rip= */ true),
+                  Immediate(enum_cast<int32_t>(WeakRefAccessState::kVisiblyEnabled)));
+    __ j(kNotEqual, slow_path->GetEntryLabel());
   }
 
   // Load the java.lang.ref.Reference class, use the output register as a temporary.