From 0dd17509a953774039c0e73beac4b0546504c774 Mon Sep 17 00:00:00 2001 From: Eino-Ville Talvala Date: Tue, 7 Jul 2015 10:43:07 -0700 Subject: Camera: Add ZERO_SHUTTER_LAG modes to noise reduction and edge enhancement This mode splits processing quality based on stream resolution, to enable ZSL operation where low-resolution (preview/recording) streams require more processing since they are immediately user-visible, while the high-resolution intermediate ZSL stream should have minimal/no processing since it will be reprocessed into final quality when necessary Bug: 22266686 Change-Id: Ib41102b66b07d61a099f021f8c6251f28c62686f --- .../android/hardware/camera2/CameraMetadata.java | 57 ++++++++++++++++++++++ .../android/hardware/camera2/CaptureRequest.java | 22 ++++++++- .../android/hardware/camera2/CaptureResult.java | 22 ++++++++- 3 files changed, 97 insertions(+), 4 deletions(-) (limited to 'core/java/android') diff --git a/core/java/android/hardware/camera2/CameraMetadata.java b/core/java/android/hardware/camera2/CameraMetadata.java index e8dbc5b..f7c6274 100644 --- a/core/java/android/hardware/camera2/CameraMetadata.java +++ b/core/java/android/hardware/camera2/CameraMetadata.java @@ -488,9 +488,13 @@ public abstract class CameraMetadata { *
  • {@link CaptureRequest#EDGE_MODE android.edge.mode}
    • * * + *
  • {@link CameraCharacteristics#NOISE_REDUCTION_AVAILABLE_NOISE_REDUCTION_MODES android.noiseReduction.availableNoiseReductionModes} and + * {@link CameraCharacteristics#EDGE_AVAILABLE_EDGE_MODES android.edge.availableEdgeModes} will both list ZERO_SHUTTER_LAG as a supported mode.
    • * * + * @see CameraCharacteristics#EDGE_AVAILABLE_EDGE_MODES * @see CaptureRequest#EDGE_MODE + * @see CameraCharacteristics#NOISE_REDUCTION_AVAILABLE_NOISE_REDUCTION_MODES * @see CaptureRequest#NOISE_REDUCTION_MODE * @see CameraCharacteristics#REPROCESS_MAX_CAPTURE_STALL * @see CameraCharacteristics#REQUEST_MAX_NUM_INPUT_STREAMS @@ -593,9 +597,13 @@ public abstract class CameraMetadata { *
  • {@link CaptureRequest#REPROCESS_EFFECTIVE_EXPOSURE_FACTOR android.reprocess.effectiveExposureFactor}
    • * * + *
  • {@link CameraCharacteristics#NOISE_REDUCTION_AVAILABLE_NOISE_REDUCTION_MODES android.noiseReduction.availableNoiseReductionModes} and + * {@link CameraCharacteristics#EDGE_AVAILABLE_EDGE_MODES android.edge.availableEdgeModes} will both list ZERO_SHUTTER_LAG as a supported mode.
    • * * + * @see CameraCharacteristics#EDGE_AVAILABLE_EDGE_MODES * @see CaptureRequest#EDGE_MODE + * @see CameraCharacteristics#NOISE_REDUCTION_AVAILABLE_NOISE_REDUCTION_MODES * @see CaptureRequest#NOISE_REDUCTION_MODE * @see CaptureRequest#REPROCESS_EFFECTIVE_EXPOSURE_FACTOR * @see CameraCharacteristics#REPROCESS_MAX_CAPTURE_STALL @@ -1999,6 +2007,30 @@ public abstract class CameraMetadata { */ public static final int EDGE_MODE_HIGH_QUALITY = 2; + /** + *

    Edge enhancement is applied at different levels for different output streams, + * based on resolution. Streams at maximum recording resolution (see {@link android.hardware.camera2.CameraDevice#createCaptureSession }) or below have + * edge enhancement applied, while higher-resolution streams have no edge enhancement + * applied. The level of edge enhancement for low-resolution streams is tuned so that + * frame rate is not impacted, and the quality is equal to or better than FAST (since it + * is only applied to lower-resolution outputs, quality may improve from FAST).

    + *

    This mode is intended to be used by applications operating in a zero-shutter-lag mode + * with YUV or PRIVATE reprocessing, where the application continuously captures + * high-resolution intermediate buffers into a circular buffer, from which a final image is + * produced via reprocessing when a user takes a picture. For such a use case, the + * high-resolution buffers must not have edge enhancement applied to maximize efficiency of + * preview and to avoid double-applying enhancement when reprocessed, while low-resolution + * buffers (used for recording or preview, generally) need edge enhancement applied for + * reasonable preview quality.

    + *

    This mode is guaranteed to be supported by devices that support either the + * YUV_REPROCESSING or PRIVATE_REPROCESSING capabilities + * ({@link CameraCharacteristics#REQUEST_AVAILABLE_CAPABILITIES android.request.availableCapabilities} lists either of those capabilities).

    + * + * @see CameraCharacteristics#REQUEST_AVAILABLE_CAPABILITIES + * @see CaptureRequest#EDGE_MODE + */ + public static final int EDGE_MODE_ZERO_SHUTTER_LAG = 3; + // // Enumeration values for CaptureRequest#FLASH_MODE // @@ -2104,6 +2136,31 @@ public abstract class CameraMetadata { */ public static final int NOISE_REDUCTION_MODE_MINIMAL = 3; + /** + *

    Noise reduction is applied at different levels for different output streams, + * based on resolution. Streams at maximum recording resolution (see {@link android.hardware.camera2.CameraDevice#createCaptureSession }) or below have noise + * reduction applied, while higher-resolution streams have MINIMAL (if supported) or no + * noise reduction applied (if MINIMAL is not supported.) The degree of noise reduction + * for low-resolution streams is tuned so that frame rate is not impacted, and the quality + * is equal to or better than FAST (since it is only applied to lower-resolution outputs, + * quality may improve from FAST).

    + *

    This mode is intended to be used by applications operating in a zero-shutter-lag mode + * with YUV or PRIVATE reprocessing, where the application continuously captures + * high-resolution intermediate buffers into a circular buffer, from which a final image is + * produced via reprocessing when a user takes a picture. For such a use case, the + * high-resolution buffers must not have noise reduction applied to maximize efficiency of + * preview and to avoid over-applying noise filtering when reprocessing, while + * low-resolution buffers (used for recording or preview, generally) need noise reduction + * applied for reasonable preview quality.

    + *

    This mode is guaranteed to be supported by devices that support either the + * YUV_REPROCESSING or PRIVATE_REPROCESSING capabilities + * ({@link CameraCharacteristics#REQUEST_AVAILABLE_CAPABILITIES android.request.availableCapabilities} lists either of those capabilities).

    + * + * @see CameraCharacteristics#REQUEST_AVAILABLE_CAPABILITIES + * @see CaptureRequest#NOISE_REDUCTION_MODE + */ + public static final int NOISE_REDUCTION_MODE_ZERO_SHUTTER_LAG = 4; + // // Enumeration values for CaptureRequest#SENSOR_TEST_PATTERN_MODE // diff --git a/core/java/android/hardware/camera2/CaptureRequest.java b/core/java/android/hardware/camera2/CaptureRequest.java index 33cc962..f201fe9 100644 --- a/core/java/android/hardware/camera2/CaptureRequest.java +++ b/core/java/android/hardware/camera2/CaptureRequest.java @@ -1569,7 +1569,14 @@ public final class CaptureRequest extends CameraMetadata> * will be applied. HIGH_QUALITY mode indicates that the * camera device will use the highest-quality enhancement algorithms, * even if it slows down capture rate. FAST means the camera device will - * not slow down capture rate when applying edge enhancement.

    + * not slow down capture rate when applying edge enhancement. Every output stream will + * have a similar amount of enhancement applied.

    + *

    ZERO_SHUTTER_LAG is meant to be used by applications that maintain a continuous circular + * buffer of high-resolution images during preview and reprocess image(s) from that buffer + * into a final capture when triggered by the user. In this mode, the camera device applies + * edge enhancement to low-resolution streams (below maximum recording resolution) to + * maximize preview quality, but does not apply edge enhancement to high-resolution streams, + * since those will be reprocessed later if necessary.

    *

    For YUV_REPROCESSING, these FAST/HIGH_QUALITY modes both mean that the camera * device will apply FAST/HIGH_QUALITY YUV-domain edge enhancement, respectively. * The camera device may adjust its internal noise reduction parameters for best @@ -1579,6 +1586,7 @@ public final class CaptureRequest extends CameraMetadata> *

  • {@link #EDGE_MODE_OFF OFF}
    • *
  • {@link #EDGE_MODE_FAST FAST}
    • *
  • {@link #EDGE_MODE_HIGH_QUALITY HIGH_QUALITY}
    • + *
  • {@link #EDGE_MODE_ZERO_SHUTTER_LAG ZERO_SHUTTER_LAG}
    • *

    *

    Available values for this device:
    * {@link CameraCharacteristics#EDGE_AVAILABLE_EDGE_MODES android.edge.availableEdgeModes}

    @@ -1593,6 +1601,7 @@ public final class CaptureRequest extends CameraMetadata> * @see #EDGE_MODE_OFF * @see #EDGE_MODE_FAST * @see #EDGE_MODE_HIGH_QUALITY + * @see #EDGE_MODE_ZERO_SHUTTER_LAG */ @PublicKey public static final Key EDGE_MODE = @@ -1979,7 +1988,14 @@ public final class CaptureRequest extends CameraMetadata> * will be applied. HIGH_QUALITY mode indicates that the camera device * will use the highest-quality noise filtering algorithms, * even if it slows down capture rate. FAST means the camera device will not - * slow down capture rate when applying noise filtering.

    + * slow down capture rate when applying noise filtering. Every output stream will + * have a similar amount of enhancement applied.

    + *

    ZERO_SHUTTER_LAG is meant to be used by applications that maintain a continuous circular + * buffer of high-resolution images during preview and reprocess image(s) from that buffer + * into a final capture when triggered by the user. In this mode, the camera device applies + * noise reduction to low-resolution streams (below maximum recording resolution) to maximize + * preview quality, but does not apply noise reduction to high-resolution streams, since + * those will be reprocessed later if necessary.

    *

    For YUV_REPROCESSING, these FAST/HIGH_QUALITY modes both mean that the camera device * will apply FAST/HIGH_QUALITY YUV domain noise reduction, respectively. The camera device * may adjust the noise reduction parameters for best image quality based on the @@ -1990,6 +2006,7 @@ public final class CaptureRequest extends CameraMetadata> *

  • {@link #NOISE_REDUCTION_MODE_FAST FAST}
    • *
  • {@link #NOISE_REDUCTION_MODE_HIGH_QUALITY HIGH_QUALITY}
    • *
  • {@link #NOISE_REDUCTION_MODE_MINIMAL MINIMAL}
    • + *
  • {@link #NOISE_REDUCTION_MODE_ZERO_SHUTTER_LAG ZERO_SHUTTER_LAG}
    • *

    *

    Available values for this device:
    * {@link CameraCharacteristics#NOISE_REDUCTION_AVAILABLE_NOISE_REDUCTION_MODES android.noiseReduction.availableNoiseReductionModes}

    @@ -2005,6 +2022,7 @@ public final class CaptureRequest extends CameraMetadata> * @see #NOISE_REDUCTION_MODE_FAST * @see #NOISE_REDUCTION_MODE_HIGH_QUALITY * @see #NOISE_REDUCTION_MODE_MINIMAL + * @see #NOISE_REDUCTION_MODE_ZERO_SHUTTER_LAG */ @PublicKey public static final Key NOISE_REDUCTION_MODE = diff --git a/core/java/android/hardware/camera2/CaptureResult.java b/core/java/android/hardware/camera2/CaptureResult.java index 9dee045..0f80051 100644 --- a/core/java/android/hardware/camera2/CaptureResult.java +++ b/core/java/android/hardware/camera2/CaptureResult.java @@ -2081,7 +2081,14 @@ public class CaptureResult extends CameraMetadata> { * will be applied. HIGH_QUALITY mode indicates that the * camera device will use the highest-quality enhancement algorithms, * even if it slows down capture rate. FAST means the camera device will - * not slow down capture rate when applying edge enhancement.

    + * not slow down capture rate when applying edge enhancement. Every output stream will + * have a similar amount of enhancement applied.

    + *

    ZERO_SHUTTER_LAG is meant to be used by applications that maintain a continuous circular + * buffer of high-resolution images during preview and reprocess image(s) from that buffer + * into a final capture when triggered by the user. In this mode, the camera device applies + * edge enhancement to low-resolution streams (below maximum recording resolution) to + * maximize preview quality, but does not apply edge enhancement to high-resolution streams, + * since those will be reprocessed later if necessary.

    *

    For YUV_REPROCESSING, these FAST/HIGH_QUALITY modes both mean that the camera * device will apply FAST/HIGH_QUALITY YUV-domain edge enhancement, respectively. * The camera device may adjust its internal noise reduction parameters for best @@ -2091,6 +2098,7 @@ public class CaptureResult extends CameraMetadata> { *

  • {@link #EDGE_MODE_OFF OFF}
    • *
  • {@link #EDGE_MODE_FAST FAST}
    • *
  • {@link #EDGE_MODE_HIGH_QUALITY HIGH_QUALITY}
    • + *
  • {@link #EDGE_MODE_ZERO_SHUTTER_LAG ZERO_SHUTTER_LAG}
    • *

    *

    Available values for this device:
    * {@link CameraCharacteristics#EDGE_AVAILABLE_EDGE_MODES android.edge.availableEdgeModes}

    @@ -2105,6 +2113,7 @@ public class CaptureResult extends CameraMetadata> { * @see #EDGE_MODE_OFF * @see #EDGE_MODE_FAST * @see #EDGE_MODE_HIGH_QUALITY + * @see #EDGE_MODE_ZERO_SHUTTER_LAG */ @PublicKey public static final Key EDGE_MODE = @@ -2763,7 +2772,14 @@ public class CaptureResult extends CameraMetadata> { * will be applied. HIGH_QUALITY mode indicates that the camera device * will use the highest-quality noise filtering algorithms, * even if it slows down capture rate. FAST means the camera device will not - * slow down capture rate when applying noise filtering.

    + * slow down capture rate when applying noise filtering. Every output stream will + * have a similar amount of enhancement applied.

    + *

    ZERO_SHUTTER_LAG is meant to be used by applications that maintain a continuous circular + * buffer of high-resolution images during preview and reprocess image(s) from that buffer + * into a final capture when triggered by the user. In this mode, the camera device applies + * noise reduction to low-resolution streams (below maximum recording resolution) to maximize + * preview quality, but does not apply noise reduction to high-resolution streams, since + * those will be reprocessed later if necessary.

    *

    For YUV_REPROCESSING, these FAST/HIGH_QUALITY modes both mean that the camera device * will apply FAST/HIGH_QUALITY YUV domain noise reduction, respectively. The camera device * may adjust the noise reduction parameters for best image quality based on the @@ -2774,6 +2790,7 @@ public class CaptureResult extends CameraMetadata> { *

  • {@link #NOISE_REDUCTION_MODE_FAST FAST}
    • *
  • {@link #NOISE_REDUCTION_MODE_HIGH_QUALITY HIGH_QUALITY}
    • *
  • {@link #NOISE_REDUCTION_MODE_MINIMAL MINIMAL}
    • + *
  • {@link #NOISE_REDUCTION_MODE_ZERO_SHUTTER_LAG ZERO_SHUTTER_LAG}
    • *

    *

    Available values for this device:
    * {@link CameraCharacteristics#NOISE_REDUCTION_AVAILABLE_NOISE_REDUCTION_MODES android.noiseReduction.availableNoiseReductionModes}

    @@ -2789,6 +2806,7 @@ public class CaptureResult extends CameraMetadata> { * @see #NOISE_REDUCTION_MODE_FAST * @see #NOISE_REDUCTION_MODE_HIGH_QUALITY * @see #NOISE_REDUCTION_MODE_MINIMAL + * @see #NOISE_REDUCTION_MODE_ZERO_SHUTTER_LAG */ @PublicKey public static final Key NOISE_REDUCTION_MODE = -- cgit v1.1