aboutsummaryrefslogtreecommitdiffstatshomepage
path: root/Documentation/admin-guide/media/ipu3.rst
diff options
context:
space:
mode:
Diffstat (limited to 'Documentation/admin-guide/media/ipu3.rst')
-rw-r--r--Documentation/admin-guide/media/ipu3.rst197
1 files changed, 103 insertions, 94 deletions
diff --git a/Documentation/admin-guide/media/ipu3.rst b/Documentation/admin-guide/media/ipu3.rst
index 9361c34f123e..83b3cd03b35c 100644
--- a/Documentation/admin-guide/media/ipu3.rst
+++ b/Documentation/admin-guide/media/ipu3.rst
@@ -51,10 +51,11 @@ to userspace as a V4L2 sub-device node and has two pads:
.. tabularcolumns:: |p{0.8cm}|p{4.0cm}|p{4.0cm}|
.. flat-table::
+ :header-rows: 1
- * - pad
- - direction
- - purpose
+ * - Pad
+ - Direction
+ - Purpose
* - 0
- sink
@@ -86,44 +87,44 @@ raw Bayer format that is specific to IPU3.
Let us take the example of ov5670 sensor connected to CSI2 port 0, for a
2592x1944 image capture.
-Using the media contorller APIs, the ov5670 sensor is configured to send
+Using the media controller APIs, the ov5670 sensor is configured to send
frames in packed raw Bayer format to IPU3 CSI2 receiver.
-# This example assumes /dev/media0 as the CIO2 media device
-
-export MDEV=/dev/media0
-
-# and that ov5670 sensor is connected to i2c bus 10 with address 0x36
-
-export SDEV=$(media-ctl -d $MDEV -e "ov5670 10-0036")
+.. code-block:: none
-# Establish the link for the media devices using media-ctl [#f3]_
-media-ctl -d $MDEV -l "ov5670:0 -> ipu3-csi2 0:0[1]"
+ # This example assumes /dev/media0 as the CIO2 media device
+ export MDEV=/dev/media0
-# Set the format for the media devices
-media-ctl -d $MDEV -V "ov5670:0 [fmt:SGRBG10/2592x1944]"
+ # and that ov5670 sensor is connected to i2c bus 10 with address 0x36
+ export SDEV=$(media-ctl -d $MDEV -e "ov5670 10-0036")
-media-ctl -d $MDEV -V "ipu3-csi2 0:0 [fmt:SGRBG10/2592x1944]"
+ # Establish the link for the media devices using media-ctl [#f3]_
+ media-ctl -d $MDEV -l "ov5670:0 -> ipu3-csi2 0:0[1]"
-media-ctl -d $MDEV -V "ipu3-csi2 0:1 [fmt:SGRBG10/2592x1944]"
+ # Set the format for the media devices
+ media-ctl -d $MDEV -V "ov5670:0 [fmt:SGRBG10/2592x1944]"
+ media-ctl -d $MDEV -V "ipu3-csi2 0:0 [fmt:SGRBG10/2592x1944]"
+ media-ctl -d $MDEV -V "ipu3-csi2 0:1 [fmt:SGRBG10/2592x1944]"
Once the media pipeline is configured, desired sensor specific settings
(such as exposure and gain settings) can be set, using the yavta tool.
e.g
-yavta -w 0x009e0903 444 $SDEV
-
-yavta -w 0x009e0913 1024 $SDEV
+.. code-block:: none
-yavta -w 0x009e0911 2046 $SDEV
+ yavta -w 0x009e0903 444 $SDEV
+ yavta -w 0x009e0913 1024 $SDEV
+ yavta -w 0x009e0911 2046 $SDEV
Once the desired sensor settings are set, frame captures can be done as below.
e.g
-yavta --data-prefix -u -c10 -n5 -I -s2592x1944 --file=/tmp/frame-#.bin \
- -f IPU3_SGRBG10 $(media-ctl -d $MDEV -e "ipu3-cio2 0")
+.. code-block:: none
+
+ yavta --data-prefix -u -c10 -n5 -I -s2592x1944 --file=/tmp/frame-#.bin \
+ -f IPU3_SGRBG10 $(media-ctl -d $MDEV -e "ipu3-cio2 0")
With the above command, 10 frames are captured at 2592x1944 resolution, with
sGRBG10 format and output as IPU3_SGRBG10 format.
@@ -148,10 +149,11 @@ Each pipe has two sink pads and three source pads for the following purpose:
.. tabularcolumns:: |p{0.8cm}|p{4.0cm}|p{4.0cm}|
.. flat-table::
+ :header-rows: 1
- * - pad
- - direction
- - purpose
+ * - Pad
+ - Direction
+ - Purpose
* - 0
- sink
@@ -234,22 +236,23 @@ The IPU3 ImgU pipelines can be configured using the Media Controller, defined at
Running mode and firmware binary selection
------------------------------------------
-ImgU works based on firmware, currently the ImgU firmware support run 2 pipes in
-time-sharing with single input frame data. Each pipe can run at certain mode -
-"VIDEO" or "STILL", "VIDEO" mode is commonly used for video frames capture, and
-"STILL" is used for still frame capture. However, you can also select "VIDEO" to
-capture still frames if you want to capture images with less system load and
-power. For "STILL" mode, ImgU will try to use smaller BDS factor and output
-larger bayer frame for further YUV processing than "VIDEO" mode to get high
-quality images. Besides, "STILL" mode need XNR3 to do noise reduction, hence
-"STILL" mode will need more power and memory bandwidth than "VIDEO" mode. TNR
-will be enabled in "VIDEO" mode and bypassed by "STILL" mode. ImgU is running at
-“VIDEO” mode by default, the user can use v4l2 control V4L2_CID_INTEL_IPU3_MODE
-(currently defined in drivers/staging/media/ipu3/include/intel-ipu3.h) to query
-and set the running mode. For user, there is no difference for buffer queueing
-between the "VIDEO" and "STILL" mode, mandatory input and main output node
-should be enabled and buffers need be queued, the statistics and the view-finder
-queues are optional.
+ImgU works based on firmware, currently the ImgU firmware support run 2 pipes
+in time-sharing with single input frame data. Each pipe can run at certain mode
+- "VIDEO" or "STILL", "VIDEO" mode is commonly used for video frames capture,
+and "STILL" is used for still frame capture. However, you can also select
+"VIDEO" to capture still frames if you want to capture images with less system
+load and power. For "STILL" mode, ImgU will try to use smaller BDS factor and
+output larger bayer frame for further YUV processing than "VIDEO" mode to get
+high quality images. Besides, "STILL" mode need XNR3 to do noise reduction,
+hence "STILL" mode will need more power and memory bandwidth than "VIDEO" mode.
+TNR will be enabled in "VIDEO" mode and bypassed by "STILL" mode. ImgU is
+running at "VIDEO" mode by default, the user can use v4l2 control
+V4L2_CID_INTEL_IPU3_MODE (currently defined in
+drivers/staging/media/ipu3/include/uapi/intel-ipu3.h) to query and set the
+running mode. For user, there is no difference for buffer queueing between the
+"VIDEO" and "STILL" mode, mandatory input and main output node should be
+enabled and buffers need be queued, the statistics and the view-finder queues
+are optional.
The firmware binary will be selected according to current running mode, such log
"using binary if_to_osys_striped " or "using binary if_to_osys_primary_striped"
@@ -269,21 +272,21 @@ all the video nodes setup correctly.
Let us take "ipu3-imgu 0" subdev as an example.
-media-ctl -d $MDEV -r
-
-media-ctl -d $MDEV -l "ipu3-imgu 0 input":0 -> "ipu3-imgu 0":0[1]
-
-media-ctl -d $MDEV -l "ipu3-imgu 0":2 -> "ipu3-imgu 0 output":0[1]
-
-media-ctl -d $MDEV -l "ipu3-imgu 0":3 -> "ipu3-imgu 0 viewfinder":0[1]
+.. code-block:: none
-media-ctl -d $MDEV -l "ipu3-imgu 0":4 -> "ipu3-imgu 0 3a stat":0[1]
+ media-ctl -d $MDEV -r
+ media-ctl -d $MDEV -l "ipu3-imgu 0 input":0 -> "ipu3-imgu 0":0[1]
+ media-ctl -d $MDEV -l "ipu3-imgu 0":2 -> "ipu3-imgu 0 output":0[1]
+ media-ctl -d $MDEV -l "ipu3-imgu 0":3 -> "ipu3-imgu 0 viewfinder":0[1]
+ media-ctl -d $MDEV -l "ipu3-imgu 0":4 -> "ipu3-imgu 0 3a stat":0[1]
Also the pipe mode of the corresponding V4L2 subdev should be set as desired
(e.g 0 for video mode or 1 for still mode) through the control id 0x009819a1 as
below.
-yavta -w "0x009819A1 1" /dev/v4l-subdev7
+.. code-block:: none
+
+ yavta -w "0x009819A1 1" /dev/v4l-subdev7
Certain hardware blocks in ImgU pipeline can change the frame resolution by
cropping or scaling, these hardware blocks include Input Feeder(IF), Bayer Down
@@ -313,8 +316,8 @@ configuration steps of 0.03125 (1/32).
**Geometric Distortion Correction**
-Geometric Distortion Correction is used to performe correction of distortions
-and image filtering. It needs some extra filter and envelop padding pixels to
+Geometric Distortion Correction is used to perform correction of distortions
+and image filtering. It needs some extra filter and envelope padding pixels to
work, so the input resolution of GDC should be larger than the output
resolution.
@@ -371,30 +374,32 @@ v4l2n command can be used. This helps process the raw Bayer frames and produces
the desired results for the main output image and the viewfinder output, in NV12
format.
-v4l2n --pipe=4 --load=/tmp/frame-#.bin --open=/dev/video4
---fmt=type:VIDEO_OUTPUT_MPLANE,width=2592,height=1944,pixelformat=0X47337069
---reqbufs=type:VIDEO_OUTPUT_MPLANE,count:1 --pipe=1 --output=/tmp/frames.out
---open=/dev/video5
---fmt=type:VIDEO_CAPTURE_MPLANE,width=2560,height=1920,pixelformat=NV12
---reqbufs=type:VIDEO_CAPTURE_MPLANE,count:1 --pipe=2 --output=/tmp/frames.vf
---open=/dev/video6
---fmt=type:VIDEO_CAPTURE_MPLANE,width=2560,height=1920,pixelformat=NV12
---reqbufs=type:VIDEO_CAPTURE_MPLANE,count:1 --pipe=3 --open=/dev/video7
---output=/tmp/frames.3A --fmt=type:META_CAPTURE,?
---reqbufs=count:1,type:META_CAPTURE --pipe=1,2,3,4 --stream=5
+.. code-block:: none
+
+ v4l2n --pipe=4 --load=/tmp/frame-#.bin --open=/dev/video4
+ --fmt=type:VIDEO_OUTPUT_MPLANE,width=2592,height=1944,pixelformat=0X47337069 \
+ --reqbufs=type:VIDEO_OUTPUT_MPLANE,count:1 --pipe=1 \
+ --output=/tmp/frames.out --open=/dev/video5 \
+ --fmt=type:VIDEO_CAPTURE_MPLANE,width=2560,height=1920,pixelformat=NV12 \
+ --reqbufs=type:VIDEO_CAPTURE_MPLANE,count:1 --pipe=2 \
+ --output=/tmp/frames.vf --open=/dev/video6 \
+ --fmt=type:VIDEO_CAPTURE_MPLANE,width=2560,height=1920,pixelformat=NV12 \
+ --reqbufs=type:VIDEO_CAPTURE_MPLANE,count:1 --pipe=3 --open=/dev/video7 \
+ --output=/tmp/frames.3A --fmt=type:META_CAPTURE,? \
+ --reqbufs=count:1,type:META_CAPTURE --pipe=1,2,3,4 --stream=5
You can also use yavta [#f2]_ command to do same thing as above:
.. code-block:: none
- yavta --data-prefix -Bcapture-mplane -c10 -n5 -I -s2592x1944 \
- --file=frame-#.out-f NV12 /dev/video5 & \
- yavta --data-prefix -Bcapture-mplane -c10 -n5 -I -s2592x1944 \
- --file=frame-#.vf -f NV12 /dev/video6 & \
- yavta --data-prefix -Bmeta-capture -c10 -n5 -I \
- --file=frame-#.3a /dev/video7 & \
- yavta --data-prefix -Boutput-mplane -c10 -n5 -I -s2592x1944 \
- --file=/tmp/frame-in.cio2 -f IPU3_SGRBG10 /dev/video4
+ yavta --data-prefix -Bcapture-mplane -c10 -n5 -I -s2592x1944 \
+ --file=frame-#.out-f NV12 /dev/video5 & \
+ yavta --data-prefix -Bcapture-mplane -c10 -n5 -I -s2592x1944 \
+ --file=frame-#.vf -f NV12 /dev/video6 & \
+ yavta --data-prefix -Bmeta-capture -c10 -n5 -I \
+ --file=frame-#.3a /dev/video7 & \
+ yavta --data-prefix -Boutput-mplane -c10 -n5 -I -s2592x1944 \
+ --file=/tmp/frame-in.cio2 -f IPU3_SGRBG10 /dev/video4
where /dev/video4, /dev/video5, /dev/video6 and /dev/video7 devices point to
input, output, viewfinder and 3A statistics video nodes respectively.
@@ -408,7 +413,9 @@ as below.
Main output frames
~~~~~~~~~~~~~~~~~~
-raw2pnm -x2560 -y1920 -fNV12 /tmp/frames.out /tmp/frames.out.ppm
+.. code-block:: none
+
+ raw2pnm -x2560 -y1920 -fNV12 /tmp/frames.out /tmp/frames.out.ppm
where 2560x1920 is output resolution, NV12 is the video format, followed
by input frame and output PNM file.
@@ -416,7 +423,9 @@ by input frame and output PNM file.
Viewfinder output frames
~~~~~~~~~~~~~~~~~~~~~~~~
-raw2pnm -x2560 -y1920 -fNV12 /tmp/frames.vf /tmp/frames.vf.ppm
+.. code-block:: none
+
+ raw2pnm -x2560 -y1920 -fNV12 /tmp/frames.vf /tmp/frames.vf.ppm
where 2560x1920 is output resolution, NV12 is the video format, followed
by input frame and output PNM file.
@@ -482,63 +491,63 @@ Name Description
Optical Black Correction Optical Black Correction block subtracts a pre-defined
value from the respective pixel values to obtain better
image quality.
- Defined in :c:type:`ipu3_uapi_obgrid_param`.
+ Defined in struct ipu3_uapi_obgrid_param.
Linearization This algo block uses linearization parameters to
address non-linearity sensor effects. The Lookup table
table is defined in
- :c:type:`ipu3_uapi_isp_lin_vmem_params`.
+ struct ipu3_uapi_isp_lin_vmem_params.
SHD Lens shading correction is used to correct spatial
non-uniformity of the pixel response due to optical
lens shading. This is done by applying a different gain
for each pixel. The gain, black level etc are
- configured in :c:type:`ipu3_uapi_shd_config_static`.
+ configured in struct ipu3_uapi_shd_config_static.
BNR Bayer noise reduction block removes image noise by
applying a bilateral filter.
- See :c:type:`ipu3_uapi_bnr_static_config` for details.
+ See struct ipu3_uapi_bnr_static_config for details.
ANR Advanced Noise Reduction is a block based algorithm
that performs noise reduction in the Bayer domain. The
convolution matrix etc can be found in
- :c:type:`ipu3_uapi_anr_config`.
+ struct ipu3_uapi_anr_config.
DM Demosaicing converts raw sensor data in Bayer format
into RGB (Red, Green, Blue) presentation. Then add
outputs of estimation of Y channel for following stream
processing by Firmware. The struct is defined as
- :c:type:`ipu3_uapi_dm_config`.
+ struct ipu3_uapi_dm_config.
Color Correction Color Correction algo transforms sensor specific color
space to the standard "sRGB" color space. This is done
by applying 3x3 matrix defined in
- :c:type:`ipu3_uapi_ccm_mat_config`.
-Gamma correction Gamma correction :c:type:`ipu3_uapi_gamma_config` is a
+ struct ipu3_uapi_ccm_mat_config.
+Gamma correction Gamma correction struct ipu3_uapi_gamma_config is a
basic non-linear tone mapping correction that is
applied per pixel for each pixel component.
CSC Color space conversion transforms each pixel from the
RGB primary presentation to YUV (Y: brightness,
UV: Luminance) presentation. This is done by applying
a 3x3 matrix defined in
- :c:type:`ipu3_uapi_csc_mat_config`
+ struct ipu3_uapi_csc_mat_config
CDS Chroma down sampling
After the CSC is performed, the Chroma Down Sampling
is applied for a UV plane down sampling by a factor
of 2 in each direction for YUV 4:2:0 using a 4x2
- configurable filter :c:type:`ipu3_uapi_cds_params`.
+ configurable filter struct ipu3_uapi_cds_params.
CHNR Chroma noise reduction
This block processes only the chrominance pixels and
performs noise reduction by cleaning the high
frequency noise.
- See struct :c:type:`ipu3_uapi_yuvp1_chnr_config`.
+ See struct struct ipu3_uapi_yuvp1_chnr_config.
TCC Total color correction as defined in struct
- :c:type:`ipu3_uapi_yuvp2_tcc_static_config`.
+ struct ipu3_uapi_yuvp2_tcc_static_config.
XNR3 eXtreme Noise Reduction V3 is the third revision of
noise reduction algorithm used to improve image
quality. This removes the low frequency noise in the
captured image. Two related structs are being defined,
- :c:type:`ipu3_uapi_isp_xnr3_params` for ISP data memory
- and :c:type:`ipu3_uapi_isp_xnr3_vmem_params` for vector
+ struct ipu3_uapi_isp_xnr3_params for ISP data memory
+ and struct ipu3_uapi_isp_xnr3_vmem_params for vector
memory.
TNR Temporal Noise Reduction block compares successive
frames in time to remove anomalies / noise in pixel
- values. :c:type:`ipu3_uapi_isp_tnr3_vmem_params` and
- :c:type:`ipu3_uapi_isp_tnr3_params` are defined for ISP
+ values. struct ipu3_uapi_isp_tnr3_vmem_params and
+ struct ipu3_uapi_isp_tnr3_params are defined for ISP
vector and data memory respectively.
======================== =======================================================
@@ -570,9 +579,9 @@ processor, while many others will use a set of fixed hardware blocks also
called accelerator cluster (ACC) to crunch pixel data and produce statistics.
ACC parameters of individual algorithms, as defined by
-:c:type:`ipu3_uapi_acc_param`, can be chosen to be applied by the user
-space through struct :c:type:`ipu3_uapi_flags` embedded in
-:c:type:`ipu3_uapi_params` structure. For parameters that are configured as
+struct ipu3_uapi_acc_param, can be chosen to be applied by the user
+space through struct struct ipu3_uapi_flags embedded in
+struct ipu3_uapi_params structure. For parameters that are configured as
not enabled by the user space, the corresponding structs are ignored by the
driver, in which case the existing configuration of the algorithm will be
preserved.
@@ -580,7 +589,7 @@ preserved.
References
==========
-.. [#f5] drivers/staging/media/ipu3/include/intel-ipu3.h
+.. [#f5] drivers/staging/media/ipu3/include/uapi/intel-ipu3.h
.. [#f1] https://github.com/intel/nvt