Input Coordinate Processing in InputFlinger

This document aims to illustrate why we need to take care when converting between the discrete and continuous coordinate spaces, especially when performing rotations.

The Linux evdev protocol works over discrete integral values. The same is true for displays, which output discrete pixels. WindowManager also tracks window bounds in pixels in the rotated logical display.

However, our MotionEvent APIs report floating point axis values in a continuous space. This disparity is important to note when working in InputFlinger, which has to make sure the discrete raw coordinates are converted to the continuous space correctly in all scenarios.

Disparity between continuous and discrete coordinates during rotation

Let's consider an example of device that has a 3 x 4 screen.

Natural orientation: No rotation

If the user interacts with the highlighted pixel, the touchscreen would report the discreet coordinates (0, 2).

     ┌─────┬─────┬─────┐
     │ 0,0 │ 1,0 │ 2,0 │
     ├─────┼─────┼─────┤
     │ 0,1 │ 1,1 │ 2,1 │
     ├─────┼─────┼─────┤
     │█0,2█│ 1,2 │ 2,2 │
     ├─────┼─────┼─────┤
     │ 0,3 │ 1,3 │ 2,3 │
     └─────┴─────┴─────┘

When converted to the continuous space, the point (0, 2) corresponds to the location shown below.

     0     1     2     3
  0  ┌─────┬─────┬─────┐
     │     │     │     │
  1  ├─────┼─────┼─────┤
     │     │     │     │
  2  █─────┼─────┼─────┤
     │     │     │     │
  3  ├─────┼─────┼─────┤
     │     │     │     │
  4  └─────┴─────┴─────┘

Rotated orientation: 90-degree counter-clockwise rotation

When the device is rotated and the same place on the touchscreen is touched, the input device will still report the same coordinates of (0, 2).

In the rotated display, that now corresponds to the pixel (2, 2).

     ┌─────┬─────┬─────┬─────┐
     │ 0,0 │ 1,0 │ 2,0 │ 3,0 │
     ├─────┼─────┼─────┼─────┤
     │ 0,1 │ 1,1 │ 2,1 │ 3,1 │
     ├─────┼─────┼─────┼─────┤
     │ 0,2 │ 1,2 │█2,2█│ 3,2 │
     └─────┴─────┴─────┴─────┘

It is important to note that rotating the device 90 degrees is NOT equivalent to rotating the continuous coordinate space by 90 degrees.

The point (2, 2) now corresponds to a different location in the continuous space than before, even though the user was interacting at the same place on the touchscreen.

     0     1     2     3     4
  0  ┌─────┬─────┬─────┬─────┐
     │     │     │     │     │
  1  ├─────┼─────┼─────┼─────┤
     │     │     │     │     │
  2  ├─────┼─────█─────┼─────┤
     │     │     │     │     │
  3  └─────┴─────┴─────┴─────┘

If we were to simply (incorrectly) rotate the continuous space from before by 90 degrees, the touched point would correspond to the location (2, 3), shown below. This new point is outside the bounds of the display, since it does not correspond to any pixel at that location.

It should be impossible for a touchscreen to generate points outside the bounds of the display, because we assume that the area of the touchscreen maps directly to the area of the display. Therefore, that point is an invalid coordinate that cannot be generated by an input device.

     0     1     2     3     4
  0  ┌─────┬─────┬─────┬─────┐
     │     │     │     │     ╏
  1  ├─────┼─────┼─────┼─────┤
     │     │     │     │     ╏
  2  ├─────┼─────┼─────┼─────┤
     │     │     │     │     ╏
  3  └-----┴-----█-----┴-----┘

The same logic applies to windows as well. When performing hit tests to determine if a point in the continuous space falls inside a window's bounds, hit test must be performed in the correct orientation, since points on the right and bottom edges of the window do not fall within the window bounds.