Cloud-based teleoperation technologies for robotics have gained increasing importance due to their ability to bridge the gap between human operators and remote robotic systems, fulfilling various needs across industries.
However, implementing effective cloud-based teleoperation systems comes with several challenges. A human operator, or a cloud-based algorithm for that matter, must have a continuous, real-time stream of the robot’s cameras and other sensors to make split-second decisions that may be required. These streams may need to be shared across multiple clients in a scalable manner. The operator’s decisions must be delivered with minimal delays to the robot and in a reliable manner. Incoming and outgoing streams must be secure to some degree. On top of that, the robot may roam from one network to another and lead to varying degrees of signal strengths, loads, and latencies.
In cloud-based teleoperation, choosing a protocol is a critical consideration. While peer-to-peer protocols with end-to-end encryption are ideal for secure one-on-one communication, they face challenges in efficiently sharing a robot’s dynamic camera streams with multiple users in a scalable manner. Attempting to extend peer-to-peer solutions by opening multiple channels may strain the robot’s performance. Traditional video-chat protocols, which are typically optimized for somewhat static views (i.e., one person in front of a static background), are ill-suited for the dynamic nature of the camera streams of a mobile robot, making it essential to explore alternative protocols. The question then arises: which protocol can best meet our needs?
GRPC, short for Google Remote Procedure Call, has emerged as a powerful tool for building efficient, cross-platform, and language-agnostic APIs. Originally designed for intercommunication between microservices and may not be the immediate choice for a teleoperation protocol. WebRTC, on the other hand, was designed for real-time communication and comes to the forefront when the need for peer-to-peer interaction and low latency is in need.
There are many additional protocols out there. Some of which one would not consider immediately as an option for teleoperation, but still may hold some advantages over other prominent protocols in some cases.
Let’s explore some of the main challenges and how we may choose to address them.
- Data Integrity and Reliability:
One of the standout features of GRPC is its reliance on TCP, a reliable transport protocol. This ensures that messages sent between the robot and operator are delivered in the order they were sent. In critical teleoperation scenarios, where precision and safety are paramount, this attribute becomes invaluable. The guaranteed delivery of instructions is particularly crucial when handling hazardous materials or situations where even minor errors can have significant consequences.
- Security Measures:
GRPC integrates seamlessly with Transport Layer Security (TLS), which provides strong encryption and authentication mechanisms. This ensures that sensitive data, including video feeds and control commands, are protected during transit. When the stakes are high, gRPC offers a reassuring level of security that meets the stringent requirements of applications like remote surgery or disaster response.
- Peer-to-Peer Communication:
In cloud-based teleoperation systems where the robot’s camera feeds must be instantly accessible to the operator for split-second decisions, WebRTC excels. Its architecture is inherently peer-to-peer, allowing for direct and swift data exchange between the robot and the operator, reducing latency, and providing real-time control.
- UDP for Low Latency:
WebRTC uses UDP, which, unlike TCP, does not impose strict ordering and reliability constraints. While this can result in occasional packet loss, it is beneficial in cases where minimizing latency is a priority. In situations where an operator needs to guide a robot through a remote location with unstable or varying network conditions, low-latency communication becomes critical.
The Unified Solution: Selecting Between different protocols ‘on the fly’:
A cloud-based teleoperation system for robots is dynamic by nature, and no single protocol can cater to all scenarios and challenges. To achieve the highest level of adaptability, a well-engineered system should be capable of switching between different protocols (such as gRPC and WebRTC) based on immediate requirements.
Intelligent Decision-Making:
An intelligent system can evaluate the current network conditions, the criticality of the task, and other relevant factors to determine the most suitable protocol. For instance, if the robot operates in a stable network environment with complex precision tasks, it might opt for gRPC. However, if the robot enters an area with a higher risk of latency and requires quick responses, the system can seamlessly transition to WebRTC.
Resilience to Network Changes:
Cloud-based teleoperation systems need to account for the robot’s movements and potential shifts between networks, such as Wi-Fi to cellular. In such cases, the ability to dynamically switch between protocols can maintain the quality of service and allow the teleoperator to adapt to the changing circumstances seamlessly.
Support of several protocols in cloud-based teleoperation systems for robotics is crucial for addressing the diverse and ever-changing challenges of remote control. These protocols can offer distinct advantages that, when employed together and dynamically, can ensure optimal performance, safety, and adaptability in the most demanding environments.
Fig 1: Real-time teleoperation from anywhere in the world using Cogniteam’s platform.
Cogniteam’s Cloud Platform empowers users to craft their custom Teleoperation dashboards. This adaptable platform allows for the convenient assignment of buttons tailored to specific tasks, whether it’s triggering an emergency stop, initiating a return to the docking station, or activating custom code. Furthermore, the platform offers the versatility of employing various joysticks to manually control different robot actuators and manipulators. Users can confidently select between different protocols to align with their precise requirements, ensuring an optimal teleoperation experience.
Cogniteam’s Cloud Platform empowers users to craft their custom Teleoperation dashboards. This adaptable platform allows for the convenient assignment of buttons tailored to specific tasks, whether it’s triggering an emergency stop, initiating a return to the docking station, or activating custom code. Furthermore, the platform offers the versatility of employing various joysticks to manually control different robot actuators and manipulators. Users can confidently select between different protocols to align with their precise requirements, ensuring an optimal teleoperation experience.
Fig 2: Teleoperation Settings Window
The user can choose on the fly between WebRTC and gRPC and select the desired streams, their qualities, and other properties. Cogniteam’s Platform supports the integration of the emerging protocols of the future.
Dr. Eliahu Khalastchi,
Research Scientist
