Provide a fixed transform hint if the layer is in a fixed orientation 1/2

The transform hint is used to prevent allocating a buffer of a
different size when a layer is rotated. The producer can choose to
consume the hint and allocate the buffer with the same size.

Provide the graphic producer a transform hint if the layer and its
children are in an orientation different from the display's
orientation. The caller is responsible for clearing this transform
hint if the layer is no longer in a fixed orientation.

Bug: 152919661
Test: atest VulkanPreTransformTest
Test: confirm with winscope trace, buffers are allocated taking into
account the transform hint in fixed orientation scenarios
Test: go/wm-smoke

Change-Id: Iea9dcf909921802a5be5c44dd61be3274f36bbd8

1 files changed, 6 insertions, 0 deletions

diff --git a/libs/gui/LayerState.cpp b/libs/gui/LayerState.cpp
index f7158d0472..e43446ac8c 100644
--- a/libs/gui/LayerState.cpp
+++ b/libs/gui/LayerState.cpp
@@ -116,6 +116,7 @@ status_t layer_state_t::write(Parcel& output) const
     output.writeInt32(frameRateSelectionPriority);
     output.writeFloat(frameRate);
     output.writeByte(frameRateCompatibility);
+    output.writeUint32(fixedTransformHint);
     return NO_ERROR;
 }
 
@@ -198,6 +199,7 @@ status_t layer_state_t::read(const Parcel& input)
     frameRateSelectionPriority = input.readInt32();
     frameRate = input.readFloat();
     frameRateCompatibility = input.readByte();
+    fixedTransformHint = static_cast<ui::Transform::RotationFlags>(input.readUint32());
     return NO_ERROR;
 }
 
@@ -433,6 +435,10 @@ void layer_state_t::merge(const layer_state_t& other) {
         frameRate = other.frameRate;
         frameRateCompatibility = other.frameRateCompatibility;
     }
+    if (other.what & eFixedTransformHintChanged) {
+        what |= eFixedTransformHintChanged;
+        fixedTransformHint = other.fixedTransformHint;
+    }
     if ((other.what & what) != other.what) {
         ALOGE("Unmerged SurfaceComposer Transaction properties. LayerState::merge needs updating? "
               "other.what=0x%" PRIu64 " what=0x%" PRIu64,