diff options
Diffstat (limited to 'Documentation/driver-api/media/v4l2-subdev.rst')
| -rw-r--r-- | Documentation/driver-api/media/v4l2-subdev.rst | 268 |
1 files changed, 210 insertions, 58 deletions
diff --git a/Documentation/driver-api/media/v4l2-subdev.rst b/Documentation/driver-api/media/v4l2-subdev.rst index bc7e1fc40a9d..13aec460e802 100644 --- a/Documentation/driver-api/media/v4l2-subdev.rst +++ b/Documentation/driver-api/media/v4l2-subdev.rst @@ -34,7 +34,7 @@ provides host private data for that purpose that can be accessed with From the bridge driver perspective, you load the sub-device module and somehow obtain the :c:type:`v4l2_subdev` pointer. For i2c devices this is easy: you call ``i2c_get_clientdata()``. For other buses something similar needs to be done. -Helper functions exists for sub-devices on an I2C bus that do most of this +Helper functions exist for sub-devices on an I2C bus that do most of this tricky work for you. Each :c:type:`v4l2_subdev` contains function pointers that sub-device drivers @@ -110,7 +110,7 @@ pads: err = media_entity_pads_init(&sd->entity, npads, pads); The pads array must have been previously initialized. There is no need to -manually set the struct :c:type:`media_entity` function and name fields, but the +manually set the struct media_entity function and name fields, but the revision field must be initialized if needed. A reference to the entity will be automatically acquired/released when the @@ -122,15 +122,12 @@ Don't forget to cleanup the media entity before the sub-device is destroyed: media_entity_cleanup(&sd->entity); -If the subdev driver intends to process video and integrate with the media -framework, it must implement format related functionality using -:c:type:`v4l2_subdev_pad_ops` instead of :c:type:`v4l2_subdev_video_ops`. - -In that case, the subdev driver may set the link_validate field to provide -its own link validation function. The link validation function is called for -every link in the pipeline where both of the ends of the links are V4L2 -sub-devices. The driver is still responsible for validating the correctness -of the format configuration between sub-devices and video nodes. +If a sub-device driver implements sink pads, the subdev driver may set the +link_validate field in :c:type:`v4l2_subdev_pad_ops` to provide its own link +validation function. For every link in the pipeline, the link_validate pad +operation of the sink end of the link is called. In both cases the driver is +still responsible for validating the correctness of the format configuration +between sub-devices and video nodes. If link_validate op is not set, the default function :c:func:`v4l2_subdev_link_validate_default` is used instead. This function @@ -138,6 +135,9 @@ ensures that width, height and the media bus pixel code are equal on both source and sink of the link. Subdev drivers are also free to use this function to perform the checks mentioned above in addition to their own checks. +Subdev registration +~~~~~~~~~~~~~~~~~~~ + There are currently two ways to register subdevices with the V4L2 core. The first (traditional) possibility is to have subdevices registered by bridge drivers. This can be done when the bridge driver has the complete information @@ -157,7 +157,10 @@ below. Using one or the other registration method only affects the probing process, the run-time bridge-subdevice interaction is in both cases the same. -In the synchronous case a device (bridge) driver needs to register the +Registering synchronous sub-devices +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +In the **synchronous** case a device (bridge) driver needs to register the :c:type:`v4l2_subdev` with the v4l2_device: :c:func:`v4l2_device_register_subdev <v4l2_device_register_subdev>` @@ -175,11 +178,125 @@ You can unregister a sub-device using: :c:func:`v4l2_device_unregister_subdev <v4l2_device_unregister_subdev>` (:c:type:`sd <v4l2_subdev>`). - Afterwards the subdev module can be unloaded and :c:type:`sd <v4l2_subdev>`->dev == ``NULL``. -You can call an ops function either directly: +.. _media-registering-async-subdevs: + +Registering asynchronous sub-devices +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +In the **asynchronous** case subdevice probing can be invoked independently of +the bridge driver availability. The subdevice driver then has to verify whether +all the requirements for a successful probing are satisfied. This can include a +check for a master clock availability. If any of the conditions aren't satisfied +the driver might decide to return ``-EPROBE_DEFER`` to request further reprobing +attempts. Once all conditions are met the subdevice shall be registered using +the :c:func:`v4l2_async_register_subdev` function. Unregistration is +performed using the :c:func:`v4l2_async_unregister_subdev` call. Subdevices +registered this way are stored in a global list of subdevices, ready to be +picked up by bridge drivers. + +Drivers must complete all initialization of the sub-device before +registering it using :c:func:`v4l2_async_register_subdev`, including +enabling runtime PM. This is because the sub-device becomes accessible +as soon as it gets registered. + +Asynchronous sub-device notifiers +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Bridge drivers in turn have to register a notifier object. This is performed +using the :c:func:`v4l2_async_nf_register` call. To unregister the notifier the +driver has to call :c:func:`v4l2_async_nf_unregister`. Before releasing memory +of an unregister notifier, it must be cleaned up by calling +:c:func:`v4l2_async_nf_cleanup`. + +Before registering the notifier, bridge drivers must do two things: first, the +notifier must be initialized using the :c:func:`v4l2_async_nf_init`. Second, +bridge drivers can then begin to form a list of async connection descriptors +that the bridge device needs for its +operation. :c:func:`v4l2_async_nf_add_fwnode`, +:c:func:`v4l2_async_nf_add_fwnode_remote` and :c:func:`v4l2_async_nf_add_i2c` + +Async connection descriptors describe connections to external sub-devices the +drivers for which are not yet probed. Based on an async connection, a media data +or ancillary link may be created when the related sub-device becomes +available. There may be one or more async connections to a given sub-device but +this is not known at the time of adding the connections to the notifier. Async +connections are bound as matching async sub-devices are found, one by one. + +Asynchronous sub-device notifier for sub-devices +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +A driver that registers an asynchronous sub-device may also register an +asynchronous notifier. This is called an asynchronous sub-device notifier and the +process is similar to that of a bridge driver apart from that the notifier is +initialised using :c:func:`v4l2_async_subdev_nf_init` instead. A sub-device +notifier may complete only after the V4L2 device becomes available, i.e. there's +a path via async sub-devices and notifiers to a notifier that is not an +asynchronous sub-device notifier. + +Asynchronous sub-device registration helper for camera sensor drivers +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +:c:func:`v4l2_async_register_subdev_sensor` is a helper function for sensor +drivers registering their own async connection, but it also registers a notifier +and further registers async connections for lens and flash devices found in +firmware. The notifier for the sub-device is unregistered and cleaned up with +the async sub-device, using :c:func:`v4l2_async_unregister_subdev`. + +Asynchronous sub-device notifier example +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +These functions allocate an async connection descriptor which is of type struct +:c:type:`v4l2_async_connection` embedded in a driver-specific struct. The &struct +:c:type:`v4l2_async_connection` shall be the first member of this struct: + +.. code-block:: c + + struct my_async_connection { + struct v4l2_async_connection asc; + ... + }; + + struct my_async_connection *my_asc; + struct fwnode_handle *ep; + + ... + + my_asc = v4l2_async_nf_add_fwnode_remote(¬ifier, ep, + struct my_async_connection); + fwnode_handle_put(ep); + + if (IS_ERR(my_asc)) + return PTR_ERR(my_asc); + +Asynchronous sub-device notifier callbacks +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The V4L2 core will then use these connection descriptors to match asynchronously +registered subdevices to them. If a match is detected the ``.bound()`` notifier +callback is called. After all connections have been bound the .complete() +callback is called. When a connection is removed from the system the +``.unbind()`` method is called. All three callbacks are optional. + +Drivers can store any type of custom data in their driver-specific +:c:type:`v4l2_async_connection` wrapper. If any of that data requires special +handling when the structure is freed, drivers must implement the ``.destroy()`` +notifier callback. The framework will call it right before freeing the +:c:type:`v4l2_async_connection`. + +Calling subdev operations +~~~~~~~~~~~~~~~~~~~~~~~~~ + +The advantage of using :c:type:`v4l2_subdev` is that it is a generic struct and +does not contain any knowledge about the underlying hardware. So a driver might +contain several subdevs that use an I2C bus, but also a subdev that is +controlled through GPIO pins. This distinction is only relevant when setting +up the device, but once the subdev is registered it is completely transparent. + +Once the subdev has been registered you can call an ops function either +directly: .. code-block:: c @@ -191,7 +308,7 @@ but it is better and easier to use this macro: err = v4l2_subdev_call(sd, core, g_std, &norm); -The macro will to the right ``NULL`` pointer checks and returns ``-ENODEV`` +The macro will do the right ``NULL`` pointer checks and returns ``-ENODEV`` if :c:type:`sd <v4l2_subdev>` is ``NULL``, ``-ENOIOCTLCMD`` if either :c:type:`sd <v4l2_subdev>`->core or :c:type:`sd <v4l2_subdev>`->core->g_std is ``NULL``, or the actual result of the :c:type:`sd <v4l2_subdev>`->ops->core->g_std ops. @@ -232,46 +349,6 @@ it can call ``v4l2_subdev_notify(sd, notification, arg)``. This macro checks whether there is a ``notify()`` callback defined and returns ``-ENODEV`` if not. Otherwise the result of the ``notify()`` call is returned. -The advantage of using :c:type:`v4l2_subdev` is that it is a generic struct and -does not contain any knowledge about the underlying hardware. So a driver might -contain several subdevs that use an I2C bus, but also a subdev that is -controlled through GPIO pins. This distinction is only relevant when setting -up the device, but once the subdev is registered it is completely transparent. - -In the asynchronous case subdevice probing can be invoked independently of the -bridge driver availability. The subdevice driver then has to verify whether all -the requirements for a successful probing are satisfied. This can include a -check for a master clock availability. If any of the conditions aren't satisfied -the driver might decide to return ``-EPROBE_DEFER`` to request further reprobing -attempts. Once all conditions are met the subdevice shall be registered using -the :c:func:`v4l2_async_register_subdev` function. Unregistration is -performed using the :c:func:`v4l2_async_unregister_subdev` call. Subdevices -registered this way are stored in a global list of subdevices, ready to be -picked up by bridge drivers. - -Bridge drivers in turn have to register a notifier object. This is -performed using the :c:func:`v4l2_async_notifier_register` call. To -unregister the notifier the driver has to call -:c:func:`v4l2_async_notifier_unregister`. The former of the two functions -takes two arguments: a pointer to struct :c:type:`v4l2_device` and a -pointer to struct :c:type:`v4l2_async_notifier`. - -Before registering the notifier, bridge drivers must do two things: -first, the notifier must be initialized using the -:c:func:`v4l2_async_notifier_init`. Second, bridge drivers can then -begin to form a list of subdevice descriptors that the bridge device -needs for its operation. Subdevice descriptors are added to the notifier -using the :c:func:`v4l2_async_notifier_add_subdev` call. This function -takes two arguments: a pointer to struct :c:type:`v4l2_async_notifier`, -and a pointer to the subdevice descripter, which is of type struct -:c:type:`v4l2_async_subdev`. - -The V4L2 core will then use these descriptors to match asynchronously -registered subdevices to them. If a match is detected the ``.bound()`` -notifier callback is called. After all subdevices have been located the -.complete() callback is called. When a subdevice is removed from the -system the .unbind() method is called. All three callbacks are optional. - V4L2 sub-device userspace API ----------------------------- @@ -281,7 +358,7 @@ response to video node operations. This hides the complexity of the underlying hardware from applications. For complex devices, finer-grained control of the device than what the video nodes offer may be required. In those cases, bridge drivers that implement :ref:`the media controller API <media_controller>` may -opt for making the subdevice operations directly accessible from userpace. +opt for making the subdevice operations directly accessible from userspace. Device nodes named ``v4l-subdev``\ *X* can be created in ``/dev`` to access sub-devices directly. If a sub-device supports direct userspace configuration @@ -484,9 +561,84 @@ The :c:func:`v4l2_i2c_new_subdev` function will call :c:type:`i2c_board_info` structure using the ``client_type`` and the ``addr`` to fill it. +Centrally managed subdev active state +------------------------------------- + +Traditionally V4L2 subdev drivers maintained internal state for the active +device configuration. This is often implemented as e.g. an array of struct +v4l2_mbus_framefmt, one entry for each pad, and similarly for crop and compose +rectangles. + +In addition to the active configuration, each subdev file handle has a struct +v4l2_subdev_state, managed by the V4L2 core, which contains the try +configuration. + +To simplify the subdev drivers the V4L2 subdev API now optionally supports a +centrally managed active configuration represented by +:c:type:`v4l2_subdev_state`. One instance of state, which contains the active +device configuration, is stored in the sub-device itself as part of +the :c:type:`v4l2_subdev` structure, while the core associates a try state to +each open file handle, to store the try configuration related to that file +handle. + +Sub-device drivers can opt-in and use state to manage their active configuration +by initializing the subdevice state with a call to v4l2_subdev_init_finalize() +before registering the sub-device. They must also call v4l2_subdev_cleanup() +to release all the allocated resources before unregistering the sub-device. +The core automatically allocates and initializes a state for each open file +handle to store the try configurations and frees it when closing the file +handle. + +V4L2 sub-device operations that use both the :ref:`ACTIVE and TRY formats +<v4l2-subdev-format-whence>` receive the correct state to operate on through +the 'state' parameter. The state must be locked and unlocked by the +caller by calling :c:func:`v4l2_subdev_lock_state()` and +:c:func:`v4l2_subdev_unlock_state()`. The caller can do so by calling the subdev +operation through the :c:func:`v4l2_subdev_call_state_active()` macro. + +Operations that do not receive a state parameter implicitly operate on the +subdevice active state, which drivers can exclusively access by +calling :c:func:`v4l2_subdev_lock_and_get_active_state()`. The sub-device active +state must equally be released by calling :c:func:`v4l2_subdev_unlock_state()`. + +Drivers must never manually access the state stored in the :c:type:`v4l2_subdev` +or in the file handle without going through the designated helpers. + +While the V4L2 core passes the correct try or active state to the subdevice +operations, many existing device drivers pass a NULL state when calling +operations with :c:func:`v4l2_subdev_call()`. This legacy construct causes +issues with subdevice drivers that let the V4L2 core manage the active state, +as they expect to receive the appropriate state as a parameter. To help the +conversion of subdevice drivers to a managed active state without having to +convert all callers at the same time, an additional wrapper layer has been +added to v4l2_subdev_call(), which handles the NULL case by getting and locking +the callee's active state with :c:func:`v4l2_subdev_lock_and_get_active_state()`, +and unlocking the state after the call. + +The whole subdev state is in reality split into three parts: the +v4l2_subdev_state, subdev controls and subdev driver's internal state. In the +future these parts should be combined into a single state. For the time being +we need a way to handle the locking for these parts. This can be accomplished +by sharing a lock. The v4l2_ctrl_handler already supports this via its 'lock' +pointer and the same model is used with states. The driver can do the following +before calling v4l2_subdev_init_finalize(): + +.. code-block:: c + + sd->ctrl_handler->lock = &priv->mutex; + sd->state_lock = &priv->mutex; + +This shares the driver's private mutex between the controls and the states. + +Streams, multiplexed media pads and internal routing +---------------------------------------------------- + +A subdevice driver can implement support for multiplexed streams by setting +the V4L2_SUBDEV_FL_STREAMS subdev flag and implementing support for +centrally managed subdev active state, routing and stream based +configuration. + V4L2 sub-device functions and data structures --------------------------------------------- .. kernel-doc:: include/media/v4l2-subdev.h - -.. kernel-doc:: include/media/v4l2-async.h |