From fab663ebfdbb7c0c8e12e27dd82b0c9331f14184 Mon Sep 17 00:00:00 2001 From: Zhijun He Date: Tue, 26 May 2015 19:46:31 -0700 Subject: camera_metadata: update high speed video recording API Update high speed video recording API to support batch mode recording. Bug: 21442271 Change-Id: I62de6067c75fcfbe6320dd987cce65843d810021 --- .../hardware/camera2/CameraCharacteristics.java | 58 +++++++++---- .../android/hardware/camera2/CameraMetadata.java | 99 ++++++++++++++++++++++ 2 files changed, 139 insertions(+), 18 deletions(-) (limited to 'core/java') diff --git a/core/java/android/hardware/camera2/CameraCharacteristics.java b/core/java/android/hardware/camera2/CameraCharacteristics.java index d5867a9..85e8827 100644 --- a/core/java/android/hardware/camera2/CameraCharacteristics.java +++ b/core/java/android/hardware/camera2/CameraCharacteristics.java @@ -626,35 +626,54 @@ public final class CameraCharacteristics extends CameraMetadata("android.control.maxRegionsAf", int.class); /** - *

List of available high speed video size and fps range configurations - * supported by the camera device, in the format of (width, height, fps_min, fps_max).

- *

When HIGH_SPEED_VIDEO is supported in {@link CameraCharacteristics#CONTROL_AVAILABLE_SCENE_MODES android.control.availableSceneModes}, this metadata - * will list the supported high speed video size and fps range configurations. All the sizes - * listed in this configuration will be a subset of the sizes reported by {@link android.hardware.camera2.params.StreamConfigurationMap#getOutputSizes } for processed - * non-stalling formats.

- *

For the high speed video use case, where the application will set - * {@link CaptureRequest#CONTROL_SCENE_MODE android.control.sceneMode} to HIGH_SPEED_VIDEO in capture requests, the application must + *

List of available high speed video size, fps range and max batch size configurations + * supported by the camera device, in the format of (width, height, fps_min, fps_max, batch_size_max).

+ *

When CONSTRAINED_HIGH_SPEED_VIDEO is supported in android.control.availableCapabilities, + * this metadata will list the supported high speed video size, fps range and max batch size + * configurations. All the sizes listed in this configuration will be a subset of the sizes + * reported by {@link android.hardware.camera2.params.StreamConfigurationMap#getOutputSizes } + * for processed non-stalling formats.

+ *

For the high speed video use case, the application must * select the video size and fps range from this metadata to configure the recording and * preview streams and setup the recording requests. For example, if the application intends * to do high speed recording, it can select the maximum size reported by this metadata to * configure output streams. Once the size is selected, application can filter this metadata * by selected size and get the supported fps ranges, and use these fps ranges to setup the * recording requests. Note that for the use case of multiple output streams, application - * must select one unique size from this metadata to use. Otherwise a request error might - * occur.

- *

For normal video recording use case, where some application will NOT set - * {@link CaptureRequest#CONTROL_SCENE_MODE android.control.sceneMode} to HIGH_SPEED_VIDEO in capture requests, the fps ranges - * reported in this metadata must not be used to setup capture requests, or it will cause - * request error.

+ * must select one unique size from this metadata to use (e.g., preview and recording streams + * must have the same size). Otherwise, the high speed capture session creation will fail.

+ *

The min and max fps will be multiple times of 30fps.

+ *

High speed video streaming extends significant performance pressue to camera hardware, + * to achieve efficient high speed streaming, the camera device may have to aggregate + * multiple frames together and send to camera device for processing where the request + * controls are same for all the frames in this batch. Max batch size indicates + * the max possible number of frames the camera device will group together for this high + * speed stream configuration. This max batch size will be used to generate a high speed + * recording request list by + * {@link android.hardware.camera2.CameraDevice#createConstrainedHighSpeedRequestList }. + * The max batch size for each configuration will satisfy below conditions:

+ * + *

The camera device doesn't have to support batch mode to achieve high speed video recording, + * in such case, batch_size_max will be reported as 1 in each configuration entry.

+ *

This fps ranges in this configuration list can only be used to create requests + * that are submitted to a high speed camera capture session created by + * {@link android.hardware.camera2.CameraDevice#createConstrainedHighSpeedCaptureSession }. + * The fps ranges reported in this metadata must not be used to setup capture requests for + * normal capture session, or it will cause request error.

*

Range of valid values:

- *

For each configuration, the fps_max >= 60fps.

+ *

For each configuration, the fps_max >= 120fps.

*

Optional - This value may be {@code null} on some devices.

*

Limited capability - * Present on all camera devices that report being at least {@link CameraCharacteristics#INFO_SUPPORTED_HARDWARE_LEVEL_LIMITED HARDWARE_LEVEL_LIMITED} devices in the * {@link CameraCharacteristics#INFO_SUPPORTED_HARDWARE_LEVEL android.info.supportedHardwareLevel} key

* - * @see CameraCharacteristics#CONTROL_AVAILABLE_SCENE_MODES - * @see CaptureRequest#CONTROL_SCENE_MODE * @see CameraCharacteristics#INFO_SUPPORTED_HARDWARE_LEVEL * @hide */ @@ -1308,7 +1327,8 @@ public final class CameraCharacteristics extends CameraMetadata *

A pipeline depth of X stages is equivalent to a pipeline latency of * X frame intervals.

- *

This value will be 8 or less.

+ *

This value will normally be 8 or less, however, for high speed capture session, + * the max pipeline depth will be up to 8 x size of high speed capture request list.

*

This key is available on all devices.

* * @see CaptureResult#REQUEST_PIPELINE_DEPTH @@ -1371,6 +1391,7 @@ public final class CameraCharacteristics extends CameraMetadata{@link #REQUEST_AVAILABLE_CAPABILITIES_BURST_CAPTURE BURST_CAPTURE} *
  • {@link #REQUEST_AVAILABLE_CAPABILITIES_YUV_REPROCESSING YUV_REPROCESSING}
    • *
  • {@link #REQUEST_AVAILABLE_CAPABILITIES_DEPTH_OUTPUT DEPTH_OUTPUT}
    • + *
  • {@link #REQUEST_AVAILABLE_CAPABILITIES_CONSTRAINED_HIGH_SPEED_VIDEO CONSTRAINED_HIGH_SPEED_VIDEO}
    • *

    *

    This key is available on all devices.

    * @@ -1384,6 +1405,7 @@ public final class CameraCharacteristics extends CameraMetadata REQUEST_AVAILABLE_CAPABILITIES = diff --git a/core/java/android/hardware/camera2/CameraMetadata.java b/core/java/android/hardware/camera2/CameraMetadata.java index ac29f80..f8db6d9 100644 --- a/core/java/android/hardware/camera2/CameraMetadata.java +++ b/core/java/android/hardware/camera2/CameraMetadata.java @@ -648,6 +648,100 @@ public abstract class CameraMetadata { */ public static final int REQUEST_AVAILABLE_CAPABILITIES_DEPTH_OUTPUT = 8; + /** + *

    The device supports constrained high speed video recording (frame rate >=120fps) + * use case. The camera device will support high speed capture session created by + * {@link android.hardware.camera2.CameraDevice#createConstrainedHighSpeedCaptureSession }, which + * only accepts high speed request list created by + * {@link android.hardware.camera2.CameraDevice#createConstrainedHighSpeedRequestList }.

    + *

    A camera device can still support high speed video streaming by advertising the high speed + * FPS ranges in {@link CameraCharacteristics#CONTROL_AE_AVAILABLE_TARGET_FPS_RANGES android.control.aeAvailableTargetFpsRanges}. For this case, all normal + * capture request per frame control and synchronization requirements will apply to + * the high speed fps ranges, the same as all other fps ranges. This capability describes + * the capability of a specialized operating mode with many limitations (see below), which + * is only targeted at high speed video recording.

    + *

    The supported high speed video sizes and fps ranges are specified in + * {@link android.hardware.camera2.params.StreamConfigurationMap#getHighSpeedVideoFpsRanges }. + * To get desired output frame rates, the application is only allowed to select video size + * and FPS range combinations provided by + * {@link android.hardware.camera2.params.StreamConfigurationMap#getHighSpeedVideoSizes }. + * The fps range can be controlled via {@link CaptureRequest#CONTROL_AE_TARGET_FPS_RANGE android.control.aeTargetFpsRange}.

    + *

    In this capability, the camera device will override aeMode, awbMode, and afMode to + * ON, ON, and CONTINUOUS_VIDEO, respectively. All post-processing block mode + * controls will be overridden to be FAST. Therefore, no manual control of capture + * and post-processing parameters is possible. All other controls operate the + * same as when {@link CaptureRequest#CONTROL_MODE android.control.mode} == AUTO. This means that all other + * android.control.* fields continue to work, such as

    + *
      + *
    • {@link CaptureRequest#CONTROL_AE_TARGET_FPS_RANGE android.control.aeTargetFpsRange}
      • + *
    • {@link CaptureRequest#CONTROL_AE_EXPOSURE_COMPENSATION android.control.aeExposureCompensation}
      • + *
    • {@link CaptureRequest#CONTROL_AE_LOCK android.control.aeLock}
      • + *
    • {@link CaptureRequest#CONTROL_AWB_LOCK android.control.awbLock}
      • + *
    • {@link CaptureRequest#CONTROL_EFFECT_MODE android.control.effectMode}
      • + *
    • {@link CaptureRequest#CONTROL_AE_REGIONS android.control.aeRegions}
      • + *
    • {@link CaptureRequest#CONTROL_AF_REGIONS android.control.afRegions}
      • + *
    • {@link CaptureRequest#CONTROL_AWB_REGIONS android.control.awbRegions}
      • + *
    • {@link CaptureRequest#CONTROL_AF_TRIGGER android.control.afTrigger}
      • + *
    • {@link CaptureRequest#CONTROL_AE_PRECAPTURE_TRIGGER android.control.aePrecaptureTrigger}
      • + *
    + *

    Outside of android.control.*, the following controls will work:

    + *
      + *
    • {@link CaptureRequest#FLASH_MODE android.flash.mode} (TORCH mode only, automatic flash for still capture will not + * work since aeMode is ON)
      • + *
    • {@link CaptureRequest#LENS_OPTICAL_STABILIZATION_MODE android.lens.opticalStabilizationMode} (if it is supported)
      • + *
    • {@link CaptureRequest#SCALER_CROP_REGION android.scaler.cropRegion}
      • + *
    • {@link CaptureRequest#STATISTICS_FACE_DETECT_MODE android.statistics.faceDetectMode} (if it is supported)
      • + *
    + *

    For high speed recording use case, the actual maximum supported frame rate may + * be lower than what camera can output, depending on the destination Surfaces for + * the image data. For example, if the destination surface is from video encoder, + * the application need check if the video encoder is capable of supporting the + * high frame rate for a given video size, or it will end up with lower recording + * frame rate. If the destination surface is from preview window, the actual preview frame + * rate will be bounded by the screen refresh rate.

    + *

    The camera device will only support up to 2 high speed simultaneous output surfaces + * (preview and recording surfaces) + * in this mode. Above controls will be effective only if all of below conditions are true:

    + *
      + *
    • The application creates a camera capture session with no more than 2 surfaces via + * {@link android.hardware.camera2.CameraDevice#createConstrainedHighSpeedCaptureSession }. The + * targeted surfaces must be preview surface (either from + * {@link android.view.SurfaceView } or {@link android.graphics.SurfaceTexture }) or + * recording surface(either from {@link android.media.MediaRecorder#getSurface } or + * {@link android.media.MediaCodec#createInputSurface }).
      • + *
    • The stream sizes are selected from the sizes reported by + * {@link android.hardware.camera2.params.StreamConfigurationMap#getHighSpeedVideoSizes }.
      • + *
    • The FPS ranges are selected from + * {@link android.hardware.camera2.params.StreamConfigurationMap#getHighSpeedVideoFpsRanges }.
      • + *
    + *

    When above conditions are NOT satistied, the + * {@link android.hardware.camera2.CameraDevice#createConstrainedHighSpeedCaptureSession } + * and {@link android.hardware.camera2.CameraDevice#createConstrainedHighSpeedRequestList } will fail.

    + *

    Switching to a FPS range that has different maximum FPS may trigger some camera device + * reconfigurations, which may introduce extra latency. It is recommended that + * the application avoids unnecessary maximum target FPS changes as much as possible + * during high speed streaming.

    + * + * @see CameraCharacteristics#CONTROL_AE_AVAILABLE_TARGET_FPS_RANGES + * @see CaptureRequest#CONTROL_AE_EXPOSURE_COMPENSATION + * @see CaptureRequest#CONTROL_AE_LOCK + * @see CaptureRequest#CONTROL_AE_PRECAPTURE_TRIGGER + * @see CaptureRequest#CONTROL_AE_REGIONS + * @see CaptureRequest#CONTROL_AE_TARGET_FPS_RANGE + * @see CaptureRequest#CONTROL_AF_REGIONS + * @see CaptureRequest#CONTROL_AF_TRIGGER + * @see CaptureRequest#CONTROL_AWB_LOCK + * @see CaptureRequest#CONTROL_AWB_REGIONS + * @see CaptureRequest#CONTROL_EFFECT_MODE + * @see CaptureRequest#CONTROL_MODE + * @see CaptureRequest#FLASH_MODE + * @see CaptureRequest#LENS_OPTICAL_STABILIZATION_MODE + * @see CaptureRequest#SCALER_CROP_REGION + * @see CaptureRequest#STATISTICS_FACE_DETECT_MODE + * @see CameraCharacteristics#REQUEST_AVAILABLE_CAPABILITIES + */ + public static final int REQUEST_AVAILABLE_CAPABILITIES_CONSTRAINED_HIGH_SPEED_VIDEO = 9; + // // Enumeration values for CameraCharacteristics#SCALER_CROPPING_TYPE // @@ -1725,6 +1819,10 @@ public abstract class CameraMetadata { public static final int CONTROL_SCENE_MODE_BARCODE = 16; /** + *

    This is deprecated, please use + * {@link android.hardware.camera2.CameraDevice#createConstrainedHighSpeedCaptureSession } + * and {@link android.hardware.camera2.CameraDevice#createConstrainedHighSpeedRequestList } + * for high speed video recording.

    *

    Optimized for high speed video recording (frame rate >=60fps) use case.

    *

    The supported high speed video sizes and fps ranges are specified in * android.control.availableHighSpeedVideoConfigurations. To get desired @@ -1799,6 +1897,7 @@ public abstract class CameraMetadata { * @see CaptureRequest#SCALER_CROP_REGION * @see CaptureRequest#STATISTICS_FACE_DETECT_MODE * @see CaptureRequest#CONTROL_SCENE_MODE + * @deprecated Please refer to this API documentation to find the alternatives */ public static final int CONTROL_SCENE_MODE_HIGH_SPEED_VIDEO = 17; -- cgit v1.1