System Setup
===============

This chapter covers the hardware and software setup required by Flexiv RDK. 

Hardware setup
--------------

Hardware connection of the robotic system is described in the provided quick start guide (printed or electronic). 
Please refer to the guide for initial hardware setup.

.. _UI_tablet:

Connect the UI tablet
---------------------------

.. important::
   When using RDK, the *Flexiv Elements* UI tablet is still needed for taking care of auxiliary operations and peripherals.

Operations that are managed by the UI tablet:

1. Switch ON/OFF the RDK server module on the robot.
2. Show detailed error messages of the robot.
3. Configure end-effector tools.
4. Enable free drive (floating) of the robot when needed, see :ref:`freedrive`.
5. Confirm Auto/Manual mode switch, see :ref:`resume_RDK`.

The UI tablet can be connected to the robot using either of the 2 ways:

1. Wired connection: use the provided Ethernet cable to connect to any of the LAN ports on the robot's control box. Or
2. Wireless connection: connect to the control box's WiFi hotspot from the UI tablet, the hotspot name starts with "CX" and is password-free.

.. note::
   Most RDK users choose wireless connection for the UI tablet.

When the UI tablet is successfully connected to the robot, you should be able to enable the robot from the tablet and see the sensor data.

System requirements
---------------------------

Minimum requirements for **real-time** access to the robot:

+--------------------+-------------------------------------------------+
| **Workstation PC** |                                                 |
+--------------------+-------------------------------------------------+
| Operating system   | Ubuntu 20.04/22.04 (real-time kernel optional*) |
+--------------------+-------------------------------------------------+
| CPU platform       | x86_64 or arm64                                 |
+--------------------+-------------------------------------------------+
| CPU performance    | Depends on computation load of user program     |
+--------------------+-------------------------------------------------+
| **Network**        |                                                 |
+--------------------+-------------------------------------------------+
| Nodes              | 1 workstation PC and 1 or more robot server(s)  |
+--------------------+-------------------------------------------------+
| Connection         | Wired LAN                                       |
+--------------------+-------------------------------------------------+
| Latency            | Round trip < 1 ms                               |
+--------------------+-------------------------------------------------+
| Bandwidth          | Send/receive both ways > 1.0 MiB/s              |
+--------------------+-------------------------------------------------+

\*\ See details in section :ref:`rt_kernel`.

Minimum requirements for **non-real-time** access to the robot:

+--------------------+------------------------------------------------+
| **Workstation PC**                                                  |
+--------------------+------------------------------------------------+
| Operating system   | All supported OS                               |
+--------------------+------------------------------------------------+
| CPU platform       | x86_64 or arm64                                |
+--------------------+------------------------------------------------+
| CPU performance    | Depends on computation load of user program    |
+--------------------+------------------------------------------------+
| **Network**                                                         |
+--------------------+------------------------------------------------+
| Nodes              | 1 workstation PC and 1 or more robot server(s) |
+--------------------+------------------------------------------------+
| Connection         | Wired or wireless LAN                          |
+--------------------+------------------------------------------------+
| Latency            | Round trip < 1000 ms                           |
+--------------------+------------------------------------------------+
| Bandwidth          | Send/receive both ways > 1.0 MiB/s             |
+--------------------+------------------------------------------------+

.. _rt_kernel:

Real-time kernel setup (optional, Linux only)
-------------------------------------------------

.. important::
   Not all use cases require patching the Linux real-time kernel, please read below to determine if you need it.


Should I patch the real-time kernel?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Yes
^^^

Linux real-time kernel is recommended if you require **hard real-time** performance, which means:

* No missed cycle.
* Minimal and stable whole-loop latency (from sending out command to receiving feedback: ~2ms).
* Deterministic program execution with stable cycle time.

But do note that:

* The patching process is tedious and may take hours.
* Successful patching is not guaranteed since some computers may be incompatible with the real-time kernel.

.. note::
   Ubuntu 22.04 users can take advantage of its native real-time kernel support and skip the following manual patching process. To activate the native real-time kernel, follow `this tutorial <https://ubuntu.com/blog/real-time-ubuntu-released>`_.


No
^^^

You can skip the patching and use the generic kernel shipped natively with Linux installation if you only require **soft real-time** performance, which means:

* A small number of missed cycles can be tolerated.
* Relatively greater and less stable whole-loop latency (from sending out command to receiving feedback: 4~10ms).
* Less deterministic program execution with less stable cycle time.

Or, you only need **non-real-time** access to the robot.


Prepare files
~~~~~~~~~~~~~~

In order to control the robot using RDK, the controller program on the workstation PC must run as a *real-time process* under a 
real-time kernel. We suggest the *PREEMPT_RT* kernel for this purpose due to its balance between ease of use and real-time performance.

.. note::
   There has been reports on issues with NVIDIA graphics card drivers when used with *PREEMPT_RT* kernel. Please refer to 
   NVIDIA's support forums for installing a driver patch.

Install dependencies::

   sudo apt install -y git build-essential cmake ncurses-dev xz-utils libssl-dev bc flex libelf-dev bison curl gnupg2


A standard Ubuntu installation comes with a generic kernel, you can check the version of your kernel using ``uname -r``. 
It should appear as ``x.y.z`` where x, y and z are the major, minor and patch digits. In the following, we use Ubuntu 20.04 
machine with a ``5.11.4`` kernel as example.

Real-time kernel patches are only available for selected versions. Please visit `<https://mirrors.edge.kernel.org/pub/linux/kernel/v5.x/>`_ 
to make sure the kernel source named ``linux-5.11.4.tar.xz`` is available. Then visit `<https://mirrors.edge.kernel.org/pub/linux/kernel/projects/rt/>`_ 
to make sure the real-time patch corresponding to the kernel source you are going to use is also available. The version of 
the kernel source should be the same as the version indicated on the patch. For example, we need to make sure ``patch-5.11.4-rt11.patch.xz`` is 
available.

When decided which version to use, download both the source and patch files using::

   curl -LO https://mirrors.edge.kernel.org/pub/linux/kernel/v5.x/linux-5.11.4.tar.xz
   curl -LO https://mirrors.edge.kernel.org/pub/linux/kernel/v5.x/linux-5.11.4.tar.sign
   curl -LO https://mirrors.edge.kernel.org/pub/linux/kernel/projects/rt/5.11/patch-5.11.4-rt11.patch.xz
   curl -LO https://mirrors.edge.kernel.org/pub/linux/kernel/projects/rt/5.11/patch-5.11.4-rt11.patch.sign  

Unzip the xz packages::

   xz -dk linux-5.11.4.tar.xz
   xz -dk patch-5.11.4-rt11.patch.xz


Verify downloaded files (optional)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

To verify the signature of kernel sources, run the following::

   gpg2 --locate-keys torvalds@kernel.org gregkh@kernel.org
   gpg2 --verify linux-5.11.4.tar.sign

To verify the patches, please visit `the Linux Foundation Wiki <https://wiki.linuxfoundation.org/realtime/preempt_rt_versions>`_  
to search for a key of the author, then run the following::

   gpg2 --verify patch-5.11.4-rt11.patch.sign


Compile and install real-time kernel
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Before compiling the sources, we have to apply the patches::

   tar xf linux-5.11.4.tar
   cd linux-5.11.4
   patch -p1 < ../patch-5.11.4-rt11.patch

Next, we configure our kernel::

   make oldconfig

A series of questions will pop up, you can select the default option. When asked for the Preemption Model, choose the ``Fully Preemptible Kernel``::

   Preemption Model
   1. No Forced Preemption (Server) (PREEMPT_NONE)
   > 2. Voluntary Kernel Preemption (Desktop) (PREEMPT_VOLUNTARY)
   3. Preemptible Kernel (Low-Latency Desktop) (PREEMPT)
   4. Fully Preemptible Kernel (Real-Time) (PREEMPT_RT) (NEW)
   choice[1-4?]: 4

For all the other questions, press enter to select the default option. Note you can long-press enter to fast-forward all questions with the default option. 
When done, a file called ".config" will be generated. Before we can compile the kernel, open the ".config" file, find and set the following configuration parameters::

   CONFIG_SYSTEM_TRUSTED_KEYS = ""
   CONFIG_SYSTEM_REVOCATION_KEYS = ""
   
Now, we are ready to compile our new kernel::

   make -j `getconf _NPROCESSORS_ONLN` deb-pkg

After the build is finished check the debian packages::

   ls ../*deb
   
Then we install all compiled kernel debian packages::

   sudo dpkg -i ../*.deb


Set user privileges to use real-time scheduling
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

To be able to schedule threads with user privileges, we will need to change the user's limits by modifying ``/etc/security/limits.conf``. 
To do so, we create a group named *realtime* and add the user operating the robot to this group::

   sudo groupadd realtime
   sudo usermod -aG realtime $(whoami)

Next, make sure ``/etc/security/limits.conf`` contains the following::

   @realtime soft rtprio 99
   @realtime soft priority 99
   @realtime soft memlock 102400
   @realtime hard rtprio 99
   @realtime hard priority 99
   @realtime hard memlock 102400


Set GRUB to boot with real-time kernel
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

We will use grub-customizer to configure grub. To install for Ubuntu 20.04::

   sudo apt install grub-customizer -y

To install for Ubuntu 18.04::

   sudo add-apt-repository ppa:danielrichter2007/grub-customizer -y
   sudo apt update -y
   sudo apt install grub-customizer -y

Simply follow the instructions in this `It's FOSS tutorial <https://itsfoss.com/grub-customizer-ubuntu/>`_ to update your grub settings.

Now reboot the system and select the ``PREEMPT_RT`` kernel. Once booted, run ``uname -a`` command in Terminal. If everything is done correctly, 
the command will return a string with ``PREEMPT_RT`` keyword.


Disable CPU speed scaling (optional) 
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Modern CPUs are able to dynamically scale their frequency depending on operating conditions. In some cases, this can lead to uneven processing speed 
that in turn interrupts execution. These fluctuations in processing speed can affect the overall performance of the real-time processes.

To disable speed scaling, we will use ``cpufrequtils`` to disable power saving mode::

   sudo apt install cpufrequtils

Running ``cpufreq-info`` shows you the available cpufreq governors. The standard setting is ``performance, powersave``. 
We will switch all governers to ``performance``::

   sudo systemctl disable ondemand
   sudo systemctl enable cpufrequtils
   sudo sh -c 'echo "GOVERNOR=performance" > /etc/default/cpufrequtils'
   sudo systemctl daemon-reload && sudo systemctl restart cpufrequtils