BufferQueue: Fix deadlock in setMaxAcquiredBufferCount

Uncovered this while testing. The deadlock happens when:

- ConsumerBase::setMaxAcquiredBufferCount locks itself
- Calls IGBC::setMaxAcquiredBufferCount, which can call
  ConsumerListener::onBuffersReleased
- Which, in ConsumerBase, will take the lock again

Instead of this, we add a callback to be called instead of the
IConsumerListener. This callback is called on the same stack, with the
lock held, so that we can resolve everything atomically.

Bug: b/393639203
Flag: EXEMPT small cleanup
Test: new test
Change-Id: Iddd8f902d1fd0aeed6aac095eaa6c0b870ffff70

1 files changed, 5 insertions, 1 deletions

diff --git a/libs/gui/ConsumerBase.cpp b/libs/gui/ConsumerBase.cpp
index 117a362661..5b89c6e17e 100644
--- a/libs/gui/ConsumerBase.cpp
+++ b/libs/gui/ConsumerBase.cpp
@@ -264,7 +264,10 @@ void ConsumerBase::onFrameReplaced(const BufferItem &item) {
 
 void ConsumerBase::onBuffersReleased() {
     Mutex::Autolock lock(mMutex);
+    onBuffersReleasedLocked();
+}
 
+void ConsumerBase::onBuffersReleasedLocked() {
     CB_LOGV("onBuffersReleased");
 
     if (mAbandoned) {
@@ -481,7 +484,8 @@ status_t ConsumerBase::setMaxAcquiredBufferCount(int maxAcquiredBuffers) {
         CB_LOGE("setMaxAcquiredBufferCount: ConsumerBase is abandoned!");
         return NO_INIT;
     }
-    return mConsumer->setMaxAcquiredBufferCount(maxAcquiredBuffers);
+    return mConsumer->setMaxAcquiredBufferCount(maxAcquiredBuffers,
+                                                {[this]() { onBuffersReleasedLocked(); }});
 }
 
 status_t ConsumerBase::setConsumerIsProtected(bool isProtected) {