docs: Improve documentation on timekeepers

- Add section on X4x0 devices, explaining that X440 has 2 timekeepers
- Expand docs on mb_controller::timekeeper
- Provide more context on multi_usrp time-related calls

3 files changed, 112 insertions, 12 deletions

diff --git a/host/docs/usrp_x4xx.dox b/host/docs/usrp_x4xx.dox
index 065e89662..59be4db89 100644
--- a/host/docs/usrp_x4xx.dox
+++ b/host/docs/usrp_x4xx.dox
@@ -968,6 +968,24 @@ std::cout << usrp->get_master_clock_rate_range().start() << std::endl; // Prints
 std::cout << usrp->get_master_clock_rate_range().stop() << std::endl; // Prints 245.76e6
 ~~~
 
+\subsection x4xx_usage_timedcmds Timed Commands
+
+The USRP X4x0 series can execute \ref page_timedcmds the same way as other USRPs,
+with the following restrictions:
+
+- Timed tuning is not supported (to be accurate, timed tuning of the RFSoC NCO
+  is not supported)
+- The USRP X440 only (this does not apply to the USRP X410!) has two timekeepers
+  instead of the usual single timekeeper.
+  - Timekeeper 0 manages the time for daughterboard 0, timekeeper 1 manages the
+    time for daughterboard 1.
+  - When a UHD session is initialized, timekeepers will be configured to run in
+    lockstep. By default, the time for both daughterboards is thus synchronized.
+  - When using multi_usrp API calls, e.g. uhd::usrp::multi_usrp::set_time_now(),
+    UHD will set the time on all available timekeepers. However, this means that
+    it is possible to desynchronize the time between daughterboards.
+  - When using the motherboard controller directly, attention must be paid to
+    not only access timekeeper zero
 
 \subsection x4xx_usage_gps GPS
 
diff --git a/host/include/uhd/rfnoc/mb_controller.hpp b/host/include/uhd/rfnoc/mb_controller.hpp
index d4d54b5aa..6c64d0da1 100644
--- a/host/include/uhd/rfnoc/mb_controller.hpp
+++ b/host/include/uhd/rfnoc/mb_controller.hpp
@@ -44,8 +44,17 @@ public:
      *************************************************************************/
     /*! Interface to interact with timekeepers
      *
-     * Timekeepers are objects separate from RFNoC blocks. This class is meant
-     * to be subclassed by motherboards implementing it.
+     * A timekeeper is an entity within a USRPs FPGA to track the time. For
+     * example, the execution of timed commands requires the existence of a
+     * timekeeper.
+     *
+     * Timekeepers are objects separate from RFNoC blocks, but RFNoC blocks can
+     * have access to the time provided by timekeepers in order to execute
+     * commands at a certain time (e.g., the radio blocks use this to be able to
+     * assign a timestamp to samples). Note that most other RFNoC blocks do not
+     * require access to a timekeeper to execute timed commands, as they will
+     * execute commands relative to the incoming data.
+     *
      */
     class UHD_API timekeeper
     {
@@ -64,12 +73,28 @@ public:
          * value. Calling this on two synchronized clocks sequentially will
          * definitely return two different values.
          *
+         * When using the RFNoC API, radio blocks also provide API calls
+         * (uhd::rfnoc::radio_control::get_time_now() and
+         * uhd::rfnoc::radio_control::get_ticks_now()) which directly returns the
+         * current time from the radio block itself. This has the advantage that
+         * it is not necessary to know which timekeeper is providing the time to
+         * which radio block when reading the current time, and generally has a
+         * lower latency than executing this call.
+         *
          * \returns the current time
          */
         uhd::time_spec_t get_time_now(void);
 
         /*! Return the current time as a tick count
          *
+         * When using the RFNoC API, radio blocks also provide API calls
+         * (uhd::rfnoc::radio_control::get_time_now() and
+         * uhd::rfnoc::radio_control::get_ticks_now()) which directly returns the
+         * current time from the radio block itself. This has the advantage that
+         * it is not necessary to know which timekeeper is providing the time to
+         * which radio block when reading the current time, and generally has a
+         * lower latency than executing this call.
+         *
          * See also get_time_now().
          *
          * \returns the current time
@@ -92,18 +117,40 @@ public:
         virtual uint64_t get_ticks_last_pps() = 0;
 
         /*! Set the time "now" from a time spec
+         *
+         * This will convert \p time into a tick count value and call
+         * set_ticks_now().
          */
         void set_time_now(const uhd::time_spec_t& time);
 
         /*! Set the ticks "now"
+         *
+         * This will set the tick count on the remote device's timekeeper as
+         * soon as possible. Note that there is some amount of lag between
+         * executing this call and when the device's time will be updated, e.g.,
+         * due to network latency.
          */
         virtual void set_ticks_now(const uint64_t ticks) = 0;
 
         /*! Set the time at next PPS from a time spec
+         *
+         * This will convert \p time into a tick count value and use that to
+         * call set_ticks_next_pps().
          */
         void set_time_next_pps(const uhd::time_spec_t& time);
 
         /*! Set the ticks at next PPS
+         *
+         * This will instruct the remote device to set its tick count when the
+         * next PPS edge is detected. Use this to set multiple devices to the
+         * same time (assuming they are synchronized in time and frequency).
+         *
+         * To guarantee that devices are synchronized in time it is recommended
+         * to wait for a PPS edge before calling this command. Otherwise, it
+         * could happen that due to network latency or other reasons, this
+         * command reaches different devices on different sides of the same PPS
+         * edge, causing devices to be unsynchronized in time by exactly one
+         * second.
          */
         virtual void set_ticks_next_pps(const uint64_t ticks) = 0;
 
@@ -138,10 +185,23 @@ public:
 
     //! Returns the number of timekeepers, which equals the number of timebases
     // on this device.
+    //
+    // Most USRPs have one timekeeper. Custom FPGA images can implement multiple
+    // timekeepers for various purposes. The USRP X440 has a minimum of two
+    // timekeepers, one for each daughterboard.
     size_t get_num_timekeepers() const;
 
     //! Return a reference to the \p tk_idx-th timekeeper on this motherboard
     //
+    // For most USRPs, timekeeper index 0 is used to access the main timekeeper.
+    // On the USRP X440, timekeeper index 0 used to keep time for daughterboard
+    // 0, and timekeeper index 1 is used to keep time for daughterboard 1.
+    //
+    // Custom FPGA images can implement multiple timekeepers.
+    //
+    // To make sure that \p tk_idx is a valid value, get_num_timekeepers() can
+    // be used to query the number of timekeepers available.
+    //
     // \throws uhd::index_error if \p tk_idx is not valid
     timekeeper::sptr get_timekeeper(const size_t tk_idx) const;
 
diff --git a/host/include/uhd/usrp/multi_usrp.hpp b/host/include/uhd/usrp/multi_usrp.hpp
index eeddef1ce..05575c44e 100644
--- a/host/include/uhd/usrp/multi_usrp.hpp
+++ b/host/include/uhd/usrp/multi_usrp.hpp
@@ -256,13 +256,23 @@ public:
      */
     virtual time_spec_t get_time_last_pps(size_t mboard = 0) = 0;
 
-    /*!
-     * Sets the time registers on the usrp immediately.
+    /*! Sets the time registers on the USRP immediately.
+     *
+     * This will set the tick count on the remote device's timekeeper as soon
+     * as possible. Note that there is some amount of lag between executing
+     * this call and when the device's time will be updated, e.g., due to
+     * network latency. This call is therefore unsuitable for setting the time
+     * on multiple devices synchronously.  Please use the set_time_next_pps() or
+     * set_time_unknown_pps() calls with a PPS signal for this use case.
+     *
+     * The one exception is when using a pair of USRP2/N200/N210 with a MIMO
+     * cable: If only one MIMO master is present in your configuration,
+     * set_time_now is safe to use because the slave's time automatically
+     * follows the master's time.
      *
-     * If only one MIMO master is present in your configuration, set_time_now is
-     * safe to use because the slave's time automatically follows the master's time.
-     * Otherwise, this call cannot set the time synchronously across multiple devices.
-     * Please use the set_time_next_pps or set_time_unknown_pps calls with a PPS signal.
+     * On RFNoC devices, this will call
+     * uhd::rfnoc::mb_controller::timekeeper::set_time_now() on all available timekeepers
+     * (usually there is one timekeeper per device, with the exception of the USRP X440).
      *
      * \param time_spec the time to latch into the usrp device
      * \param mboard the motherboard index 0 to M-1
@@ -270,8 +280,8 @@ public:
     virtual void set_time_now(
         const time_spec_t& time_spec, size_t mboard = ALL_MBOARDS) = 0;
 
-    /*!
-     * Set the time registers on the usrp at the next pps tick.
+    /*! Set the time registers on the USRP at the next PPS tick.
+     *
      * The values will not be latched in until the pulse occurs.
      * It is recommended that the user sleep(1) after calling to ensure
      * that the time registers will be in a known state prior to use.
@@ -279,14 +289,26 @@ public:
      * Note: Because this call sets the time on the "next" pps,
      * the seconds in the time spec should be current seconds + 1.
      *
+     * Note: Make sure to not call this shortly before the next PPS edge. This
+     * should be called with plenty of time before the next PPS edge (at least
+     * 50 ms) to ensure that all devices will execute this command on the same
+     * PPS edge. If not, devices could be unsynchronized in time by exactly one
+     * second. If in doubt, use set_time_unknown_pps() which will take care of
+     * this issue (but will also take longer to execute).
+     *
+     * On RFNoC devices, this will call
+     * uhd::rfnoc::mb_controller::timekeeper::set_time_next_pps() on all available
+     * timekeepers (usually there is one timekeeper per device, with the exception of the
+     * USRP X440).
+     *
      * \param time_spec the time to latch into the usrp device
      * \param mboard the motherboard index 0 to M-1
      */
     virtual void set_time_next_pps(
         const time_spec_t& time_spec, size_t mboard = ALL_MBOARDS) = 0;
 
-    /*!
-     * Synchronize the times across all motherboards in this configuration.
+    /*! Synchronize the times across all motherboards in this configuration.
+     *
      * Use this method to sync the times when the edge of the PPS is unknown.
      *
      * Ex: Host machine is not attached to serial port of GPSDO