diff options
3 files changed, 125 insertions, 37 deletions
diff --git a/core/java/android/hardware/camera2/CameraCharacteristics.java b/core/java/android/hardware/camera2/CameraCharacteristics.java index 60e4ce29ab51..d027fd997b95 100644 --- a/core/java/android/hardware/camera2/CameraCharacteristics.java +++ b/core/java/android/hardware/camera2/CameraCharacteristics.java @@ -1249,7 +1249,9 @@ public final class CameraCharacteristics extends CameraMetadata<CameraCharacteri * <p>If this device is the largest or only camera device with a given facing, then this * position will be <code>(0, 0, 0)</code>; a camera device with a lens optical center located 3 cm * from the main sensor along the +X axis (to the right from the user's perspective) will - * report <code>(0.03, 0, 0)</code>.</p> + * report <code>(0.03, 0, 0)</code>. Note that this means that, for many computer vision + * applications, the position needs to be negated to convert it to a translation from the + * camera to the origin.</p> * <p>To transform a pixel coordinates between two cameras facing the same direction, first * the source camera {@link CameraCharacteristics#LENS_DISTORTION android.lens.distortion} must be corrected for. Then the source * camera {@link CameraCharacteristics#LENS_INTRINSIC_CALIBRATION android.lens.intrinsicCalibration} needs to be applied, followed by the @@ -1261,7 +1263,8 @@ public final class CameraCharacteristics extends CameraMetadata<CameraCharacteri * <p>To compare this against a real image from the destination camera, the destination camera * image then needs to be corrected for radial distortion before comparison or sampling.</p> * <p>When {@link CameraCharacteristics#LENS_POSE_REFERENCE android.lens.poseReference} is GYROSCOPE, then this position is relative to - * the center of the primary gyroscope on the device.</p> + * the center of the primary gyroscope on the device. The axis definitions are the same as + * with PRIMARY_CAMERA.</p> * <p><b>Units</b>: Meters</p> * <p><b>Optional</b> - This value may be {@code null} on some devices.</p> * @@ -1293,13 +1296,15 @@ public final class CameraCharacteristics extends CameraMetadata<CameraCharacteri * </code></pre> * <p>which can then be combined with the camera pose rotation * <code>R</code> and translation <code>t</code> ({@link CameraCharacteristics#LENS_POSE_ROTATION android.lens.poseRotation} and - * {@link CameraCharacteristics#LENS_POSE_TRANSLATION android.lens.poseTranslation}, respective) to calculate the + * {@link CameraCharacteristics#LENS_POSE_TRANSLATION android.lens.poseTranslation}, respectively) to calculate the * complete transform from world coordinates to pixel * coordinates:</p> - * <pre><code>P = [ K 0 * [ R t - * 0 1 ] 0 1 ] + * <pre><code>P = [ K 0 * [ R -Rt + * 0 1 ] 0 1 ] * </code></pre> - * <p>and with <code>p_w</code> being a point in the world coordinate system + * <p>(Note the negation of poseTranslation when mapping from camera + * to world coordinates, and multiplication by the rotation).</p> + * <p>With <code>p_w</code> being a point in the world coordinate system * and <code>p_s</code> being a point in the camera active pixel array * coordinate system, and with the mapping including the * homogeneous division by z:</p> @@ -1321,6 +1326,13 @@ public final class CameraCharacteristics extends CameraMetadata<CameraCharacteri * activeArraySize rectangle), to determine the final pixel * coordinate of the world point for processed (non-RAW) * output buffers.</p> + * <p>For camera devices, the center of pixel <code>(x,y)</code> is located at + * coordinate <code>(x + 0.5, y + 0.5)</code>. So on a device with a + * precorrection active array of size <code>(10,10)</code>, the valid pixel + * indices go from <code>(0,0)-(9,9)</code>, and an perfectly-built camera would + * have an optical center at the exact center of the pixel grid, at + * coordinates <code>(5.0, 5.0)</code>, which is the top-left corner of pixel + * <code>(5,5)</code>.</p> * <p><b>Units</b>: * Pixels in the * {@link CameraCharacteristics#SENSOR_INFO_PRE_CORRECTION_ACTIVE_ARRAY_SIZE android.sensor.info.preCorrectionActiveArraySize} diff --git a/core/java/android/hardware/camera2/CaptureRequest.java b/core/java/android/hardware/camera2/CaptureRequest.java index aca77a5be47e..0c3fe778dda7 100644 --- a/core/java/android/hardware/camera2/CaptureRequest.java +++ b/core/java/android/hardware/camera2/CaptureRequest.java @@ -2522,7 +2522,7 @@ public final class CaptureRequest extends CameraMetadata<CaptureRequest.Key<?>> * outputs will crop horizontally (pillarbox), and 16:9 * streams will match exactly. These additional crops will * be centered within the crop region.</p> - * <p>If the coordinate system is android.sensor.info.activeArraysSize, the width and height + * <p>If the coordinate system is {@link CameraCharacteristics#SENSOR_INFO_ACTIVE_ARRAY_SIZE android.sensor.info.activeArraySize}, the width and height * of the crop region cannot be set to be smaller than * <code>floor( activeArraySize.width / {@link CameraCharacteristics#SCALER_AVAILABLE_MAX_DIGITAL_ZOOM android.scaler.availableMaxDigitalZoom} )</code> and * <code>floor( activeArraySize.height / {@link CameraCharacteristics#SCALER_AVAILABLE_MAX_DIGITAL_ZOOM android.scaler.availableMaxDigitalZoom} )</code>, respectively.</p> @@ -2863,8 +2863,14 @@ public final class CaptureRequest extends CameraMetadata<CaptureRequest.Key<?>> new Key<Integer>("android.statistics.lensShadingMapMode", int.class); /** - * <p>A control for selecting whether OIS position information is included in output - * result metadata.</p> + * <p>A control for selecting whether optical stabilization (OIS) position + * information is included in output result metadata.</p> + * <p>Since optical image stabilization generally involves motion much faster than the duration + * of individualq image exposure, multiple OIS samples can be included for a single capture + * result. For example, if the OIS reporting operates at 200 Hz, a typical camera operating + * at 30fps may have 6-7 OIS samples per capture result. This information can be combined + * with the rolling shutter skew to account for lens motion during image exposure in + * post-processing algorithms.</p> * <p><b>Possible values:</b> * <ul> * <li>{@link #STATISTICS_OIS_DATA_MODE_OFF OFF}</li> @@ -3264,14 +3270,28 @@ public final class CaptureRequest extends CameraMetadata<CaptureRequest.Key<?>> * any correction at all would slow down capture rate. Every output stream will have a * similar amount of enhancement applied.</p> * <p>The correction only applies to processed outputs such as YUV, JPEG, or DEPTH16; it is not - * applied to any RAW output. Metadata coordinates such as face rectangles or metering - * regions are also not affected by correction.</p> + * applied to any RAW output.</p> * <p>This control will be on by default on devices that support this control. Applications * disabling distortion correction need to pay extra attention with the coordinate system of * metering regions, crop region, and face rectangles. When distortion correction is OFF, * metadata coordinates follow the coordinate system of * {@link CameraCharacteristics#SENSOR_INFO_PRE_CORRECTION_ACTIVE_ARRAY_SIZE android.sensor.info.preCorrectionActiveArraySize}. When distortion is not OFF, metadata - * coordinates follow the coordinate system of {@link CameraCharacteristics#SENSOR_INFO_ACTIVE_ARRAY_SIZE android.sensor.info.activeArraySize}.</p> + * coordinates follow the coordinate system of {@link CameraCharacteristics#SENSOR_INFO_ACTIVE_ARRAY_SIZE android.sensor.info.activeArraySize}. The + * camera device will map these metadata fields to match the corrected image produced by the + * camera device, for both capture requests and results. However, this mapping is not very + * precise, since rectangles do not generally map to rectangles when corrected. Only linear + * scaling between the active array and precorrection active array coordinates is + * performed. Applications that require precise correction of metadata need to undo that + * linear scaling, and apply a more complete correction that takes into the account the app's + * own requirements.</p> + * <p>The full list of metadata that is affected in this way by distortion correction is:</p> + * <ul> + * <li>{@link CaptureRequest#CONTROL_AF_REGIONS android.control.afRegions}</li> + * <li>{@link CaptureRequest#CONTROL_AE_REGIONS android.control.aeRegions}</li> + * <li>{@link CaptureRequest#CONTROL_AWB_REGIONS android.control.awbRegions}</li> + * <li>{@link CaptureRequest#SCALER_CROP_REGION android.scaler.cropRegion}</li> + * <li>{@link CaptureResult#STATISTICS_FACES android.statistics.faces}</li> + * </ul> * <p><b>Possible values:</b> * <ul> * <li>{@link #DISTORTION_CORRECTION_MODE_OFF OFF}</li> @@ -3282,10 +3302,15 @@ public final class CaptureRequest extends CameraMetadata<CaptureRequest.Key<?>> * {@link CameraCharacteristics#DISTORTION_CORRECTION_AVAILABLE_MODES android.distortionCorrection.availableModes}</p> * <p><b>Optional</b> - This value may be {@code null} on some devices.</p> * + * @see CaptureRequest#CONTROL_AE_REGIONS + * @see CaptureRequest#CONTROL_AF_REGIONS + * @see CaptureRequest#CONTROL_AWB_REGIONS * @see CameraCharacteristics#DISTORTION_CORRECTION_AVAILABLE_MODES * @see CameraCharacteristics#LENS_DISTORTION + * @see CaptureRequest#SCALER_CROP_REGION * @see CameraCharacteristics#SENSOR_INFO_ACTIVE_ARRAY_SIZE * @see CameraCharacteristics#SENSOR_INFO_PRE_CORRECTION_ACTIVE_ARRAY_SIZE + * @see CaptureResult#STATISTICS_FACES * @see #DISTORTION_CORRECTION_MODE_OFF * @see #DISTORTION_CORRECTION_MODE_FAST * @see #DISTORTION_CORRECTION_MODE_HIGH_QUALITY diff --git a/core/java/android/hardware/camera2/CaptureResult.java b/core/java/android/hardware/camera2/CaptureResult.java index 75c27f56c0a6..845f1d3cbcba 100644 --- a/core/java/android/hardware/camera2/CaptureResult.java +++ b/core/java/android/hardware/camera2/CaptureResult.java @@ -2852,7 +2852,9 @@ public class CaptureResult extends CameraMetadata<CaptureResult.Key<?>> { * <p>If this device is the largest or only camera device with a given facing, then this * position will be <code>(0, 0, 0)</code>; a camera device with a lens optical center located 3 cm * from the main sensor along the +X axis (to the right from the user's perspective) will - * report <code>(0.03, 0, 0)</code>.</p> + * report <code>(0.03, 0, 0)</code>. Note that this means that, for many computer vision + * applications, the position needs to be negated to convert it to a translation from the + * camera to the origin.</p> * <p>To transform a pixel coordinates between two cameras facing the same direction, first * the source camera {@link CameraCharacteristics#LENS_DISTORTION android.lens.distortion} must be corrected for. Then the source * camera {@link CameraCharacteristics#LENS_INTRINSIC_CALIBRATION android.lens.intrinsicCalibration} needs to be applied, followed by the @@ -2864,7 +2866,8 @@ public class CaptureResult extends CameraMetadata<CaptureResult.Key<?>> { * <p>To compare this against a real image from the destination camera, the destination camera * image then needs to be corrected for radial distortion before comparison or sampling.</p> * <p>When {@link CameraCharacteristics#LENS_POSE_REFERENCE android.lens.poseReference} is GYROSCOPE, then this position is relative to - * the center of the primary gyroscope on the device.</p> + * the center of the primary gyroscope on the device. The axis definitions are the same as + * with PRIMARY_CAMERA.</p> * <p><b>Units</b>: Meters</p> * <p><b>Optional</b> - This value may be {@code null} on some devices.</p> * @@ -2896,13 +2899,15 @@ public class CaptureResult extends CameraMetadata<CaptureResult.Key<?>> { * </code></pre> * <p>which can then be combined with the camera pose rotation * <code>R</code> and translation <code>t</code> ({@link CameraCharacteristics#LENS_POSE_ROTATION android.lens.poseRotation} and - * {@link CameraCharacteristics#LENS_POSE_TRANSLATION android.lens.poseTranslation}, respective) to calculate the + * {@link CameraCharacteristics#LENS_POSE_TRANSLATION android.lens.poseTranslation}, respectively) to calculate the * complete transform from world coordinates to pixel * coordinates:</p> - * <pre><code>P = [ K 0 * [ R t - * 0 1 ] 0 1 ] + * <pre><code>P = [ K 0 * [ R -Rt + * 0 1 ] 0 1 ] * </code></pre> - * <p>and with <code>p_w</code> being a point in the world coordinate system + * <p>(Note the negation of poseTranslation when mapping from camera + * to world coordinates, and multiplication by the rotation).</p> + * <p>With <code>p_w</code> being a point in the world coordinate system * and <code>p_s</code> being a point in the camera active pixel array * coordinate system, and with the mapping including the * homogeneous division by z:</p> @@ -2924,6 +2929,13 @@ public class CaptureResult extends CameraMetadata<CaptureResult.Key<?>> { * activeArraySize rectangle), to determine the final pixel * coordinate of the world point for processed (non-RAW) * output buffers.</p> + * <p>For camera devices, the center of pixel <code>(x,y)</code> is located at + * coordinate <code>(x + 0.5, y + 0.5)</code>. So on a device with a + * precorrection active array of size <code>(10,10)</code>, the valid pixel + * indices go from <code>(0,0)-(9,9)</code>, and an perfectly-built camera would + * have an optical center at the exact center of the pixel grid, at + * coordinates <code>(5.0, 5.0)</code>, which is the top-left corner of pixel + * <code>(5,5)</code>.</p> * <p><b>Units</b>: * Pixels in the * {@link CameraCharacteristics#SENSOR_INFO_PRE_CORRECTION_ACTIVE_ARRAY_SIZE android.sensor.info.preCorrectionActiveArraySize} @@ -3188,7 +3200,7 @@ public class CaptureResult extends CameraMetadata<CaptureResult.Key<?>> { * outputs will crop horizontally (pillarbox), and 16:9 * streams will match exactly. These additional crops will * be centered within the crop region.</p> - * <p>If the coordinate system is android.sensor.info.activeArraysSize, the width and height + * <p>If the coordinate system is {@link CameraCharacteristics#SENSOR_INFO_ACTIVE_ARRAY_SIZE android.sensor.info.activeArraySize}, the width and height * of the crop region cannot be set to be smaller than * <code>floor( activeArraySize.width / {@link CameraCharacteristics#SCALER_AVAILABLE_MAX_DIGITAL_ZOOM android.scaler.availableMaxDigitalZoom} )</code> and * <code>floor( activeArraySize.height / {@link CameraCharacteristics#SCALER_AVAILABLE_MAX_DIGITAL_ZOOM android.scaler.availableMaxDigitalZoom} )</code>, respectively.</p> @@ -4077,8 +4089,14 @@ public class CaptureResult extends CameraMetadata<CaptureResult.Key<?>> { new Key<Integer>("android.statistics.lensShadingMapMode", int.class); /** - * <p>A control for selecting whether OIS position information is included in output - * result metadata.</p> + * <p>A control for selecting whether optical stabilization (OIS) position + * information is included in output result metadata.</p> + * <p>Since optical image stabilization generally involves motion much faster than the duration + * of individualq image exposure, multiple OIS samples can be included for a single capture + * result. For example, if the OIS reporting operates at 200 Hz, a typical camera operating + * at 30fps may have 6-7 OIS samples per capture result. This information can be combined + * with the rolling shutter skew to account for lens motion during image exposure in + * post-processing algorithms.</p> * <p><b>Possible values:</b> * <ul> * <li>{@link #STATISTICS_OIS_DATA_MODE_OFF OFF}</li> @@ -4112,11 +4130,15 @@ public class CaptureResult extends CameraMetadata<CaptureResult.Key<?>> { /** * <p>An array of shifts of OIS samples, in x direction.</p> * <p>The array contains the amount of shifts in x direction, in pixels, based on OIS samples. - * A positive value is a shift from left to right in active array coordinate system. For - * example, if the optical center is (1000, 500) in active array coordinates, a shift of - * (3, 0) puts the new optical center at (1003, 500).</p> + * A positive value is a shift from left to right in the pre-correction active array + * coordinate system. For example, if the optical center is (1000, 500) in pre-correction + * active array coordinates, a shift of (3, 0) puts the new optical center at (1003, 500).</p> * <p>The number of shifts must match the number of timestamps in * android.statistics.oisTimestamps.</p> + * <p>The OIS samples are not affected by whether lens distortion correction is enabled (on + * supporting devices). They are always reported in pre-correction active array coordinates, + * since the scaling of OIS shifts would depend on the specific spot on the sensor the shift + * is needed.</p> * <p><b>Units</b>: Pixels in active array.</p> * <p><b>Optional</b> - This value may be {@code null} on some devices.</p> * @hide @@ -4127,11 +4149,15 @@ public class CaptureResult extends CameraMetadata<CaptureResult.Key<?>> { /** * <p>An array of shifts of OIS samples, in y direction.</p> * <p>The array contains the amount of shifts in y direction, in pixels, based on OIS samples. - * A positive value is a shift from top to bottom in active array coordinate system. For - * example, if the optical center is (1000, 500) in active array coordinates, a shift of - * (0, 5) puts the new optical center at (1000, 505).</p> + * A positive value is a shift from top to bottom in pre-correction active array coordinate + * system. For example, if the optical center is (1000, 500) in active array coordinates, a + * shift of (0, 5) puts the new optical center at (1000, 505).</p> * <p>The number of shifts must match the number of timestamps in * android.statistics.oisTimestamps.</p> + * <p>The OIS samples are not affected by whether lens distortion correction is enabled (on + * supporting devices). They are always reported in pre-correction active array coordinates, + * since the scaling of OIS shifts would depend on the specific spot on the sensor the shift + * is needed.</p> * <p><b>Units</b>: Pixels in active array.</p> * <p><b>Optional</b> - This value may be {@code null} on some devices.</p> * @hide @@ -4140,15 +4166,21 @@ public class CaptureResult extends CameraMetadata<CaptureResult.Key<?>> { new Key<float[]>("android.statistics.oisYShifts", float[].class); /** - * <p>An array of OIS samples.</p> + * <p>An array of optical stabilization (OIS) position samples.</p> * <p>Each OIS sample contains the timestamp and the amount of shifts in x and y direction, * in pixels, of the OIS sample.</p> - * <p>A positive value for a shift in x direction is a shift from left to right in active array - * coordinate system. For example, if the optical center is (1000, 500) in active array - * coordinates, a shift of (3, 0) puts the new optical center at (1003, 500).</p> - * <p>A positive value for a shift in y direction is a shift from top to bottom in active array - * coordinate system. For example, if the optical center is (1000, 500) in active array - * coordinates, a shift of (0, 5) puts the new optical center at (1000, 505).</p> + * <p>A positive value for a shift in x direction is a shift from left to right in the + * pre-correction active array coordinate system. For example, if the optical center is + * (1000, 500) in pre-correction active array coordinates, a shift of (3, 0) puts the new + * optical center at (1003, 500).</p> + * <p>A positive value for a shift in y direction is a shift from top to bottom in + * pre-correction active array coordinate system. For example, if the optical center is + * (1000, 500) in active array coordinates, a shift of (0, 5) puts the new optical center at + * (1000, 505).</p> + * <p>The OIS samples are not affected by whether lens distortion correction is enabled (on + * supporting devices). They are always reported in pre-correction active array coordinates, + * since the scaling of OIS shifts would depend on the specific spot on the sensor the shift + * is needed.</p> * <p><b>Optional</b> - This value may be {@code null} on some devices.</p> */ @PublicKey @@ -4578,14 +4610,28 @@ public class CaptureResult extends CameraMetadata<CaptureResult.Key<?>> { * any correction at all would slow down capture rate. Every output stream will have a * similar amount of enhancement applied.</p> * <p>The correction only applies to processed outputs such as YUV, JPEG, or DEPTH16; it is not - * applied to any RAW output. Metadata coordinates such as face rectangles or metering - * regions are also not affected by correction.</p> + * applied to any RAW output.</p> * <p>This control will be on by default on devices that support this control. Applications * disabling distortion correction need to pay extra attention with the coordinate system of * metering regions, crop region, and face rectangles. When distortion correction is OFF, * metadata coordinates follow the coordinate system of * {@link CameraCharacteristics#SENSOR_INFO_PRE_CORRECTION_ACTIVE_ARRAY_SIZE android.sensor.info.preCorrectionActiveArraySize}. When distortion is not OFF, metadata - * coordinates follow the coordinate system of {@link CameraCharacteristics#SENSOR_INFO_ACTIVE_ARRAY_SIZE android.sensor.info.activeArraySize}.</p> + * coordinates follow the coordinate system of {@link CameraCharacteristics#SENSOR_INFO_ACTIVE_ARRAY_SIZE android.sensor.info.activeArraySize}. The + * camera device will map these metadata fields to match the corrected image produced by the + * camera device, for both capture requests and results. However, this mapping is not very + * precise, since rectangles do not generally map to rectangles when corrected. Only linear + * scaling between the active array and precorrection active array coordinates is + * performed. Applications that require precise correction of metadata need to undo that + * linear scaling, and apply a more complete correction that takes into the account the app's + * own requirements.</p> + * <p>The full list of metadata that is affected in this way by distortion correction is:</p> + * <ul> + * <li>{@link CaptureRequest#CONTROL_AF_REGIONS android.control.afRegions}</li> + * <li>{@link CaptureRequest#CONTROL_AE_REGIONS android.control.aeRegions}</li> + * <li>{@link CaptureRequest#CONTROL_AWB_REGIONS android.control.awbRegions}</li> + * <li>{@link CaptureRequest#SCALER_CROP_REGION android.scaler.cropRegion}</li> + * <li>{@link CaptureResult#STATISTICS_FACES android.statistics.faces}</li> + * </ul> * <p><b>Possible values:</b> * <ul> * <li>{@link #DISTORTION_CORRECTION_MODE_OFF OFF}</li> @@ -4596,10 +4642,15 @@ public class CaptureResult extends CameraMetadata<CaptureResult.Key<?>> { * {@link CameraCharacteristics#DISTORTION_CORRECTION_AVAILABLE_MODES android.distortionCorrection.availableModes}</p> * <p><b>Optional</b> - This value may be {@code null} on some devices.</p> * + * @see CaptureRequest#CONTROL_AE_REGIONS + * @see CaptureRequest#CONTROL_AF_REGIONS + * @see CaptureRequest#CONTROL_AWB_REGIONS * @see CameraCharacteristics#DISTORTION_CORRECTION_AVAILABLE_MODES * @see CameraCharacteristics#LENS_DISTORTION + * @see CaptureRequest#SCALER_CROP_REGION * @see CameraCharacteristics#SENSOR_INFO_ACTIVE_ARRAY_SIZE * @see CameraCharacteristics#SENSOR_INFO_PRE_CORRECTION_ACTIVE_ARRAY_SIZE + * @see CaptureResult#STATISTICS_FACES * @see #DISTORTION_CORRECTION_MODE_OFF * @see #DISTORTION_CORRECTION_MODE_FAST * @see #DISTORTION_CORRECTION_MODE_HIGH_QUALITY |