TELEPRESENCE

Teleoperation: Controlling a robot from remote

The KI.FABRIK Teleoperation in Production Demonstrator uses a visual and bilateral haptic feedback that enables the human expert to remotely teach new skills to a robot, to support assembly tasks, and to conduct quality checks by acting together with the robot on the shop floor. This approach is known from other areas, i.e. surgery, (aero) space, but not yet used in manufacturing.

The Teleoperation avatar in KI.FABRIK contains exciting new features that facilitates its use for cutting-edge research in different fields related to industrial assembly, some of these being Human-Robot collaboration, Digital Twining, Model Mediated Teleoperation or Machine Learning.

Digital Twin-Mediated Teleoperation

In bilateral teleoperation, haptic feedback is crucial in providing the operator with a sense of touch of the remote environment. The haptic feedback allows the operator to effect changes from distance much more accurately. However, the delay in the communication channel degrades the true perception of the remote side, making it harder or even impossible to achieve precise tasks.

Model-Mediated Teleoperation (MMT) addresses these issues by using a virtual representation of the environment locally, that allows the system to calculate the haptic feedback, instead of having to transmit it through the network. In KI.FABRIK, a very precise Digital Twin of the teleoperation system is used for this purpose. Precise object and environment 3D tracking using cameras is constantly at work to provide the most realistic and comfortable experience for the user.

Automating Virtual Fixture Design

With the rise of teleoperated manipulators, it is expected that tools that facilitate the interaction of humans with the remote environment will be more prevalent in industry, leading to a greater availability of them. This is the case with Virtual Fixtures, which are collections of abstract sensory information overlaid on top of reflected sensory feedback from a remote environment. Other ever more prevalent tools are Digital Twins, virtual representations of systems that facilitate bidirectional communication between the real and the virtual worlds.

Being more likely than ever that factories have both an implemented Digital Twin of a system and Virtual Fixtures for a particular manipulation system, we developed a method to leverage both tools.

In KI.FABRIK we use state-of-the-art Reinforcement Learning algorithms to construct an optimal Virtual Fixture to aid the human user in a telemanipulation task, tackling the cumbersome problems of reward function and Virtual Fixture design simulataneously.

Teleoperated Learning from Demonstration

Programming autonomous behavior in machines and robots traditionally requires a specific set of skills and knowledge. On the other hand, human experts can demonstrate the desired task even if they do not know how to program the necessary behavior in a machine or robot. The purpose of Learning from Demonstration (LfD) is to efficiently learn a desired behavior by imitating the teacher. LfD is considered a key technology for applications in manufacturing, elder care, and the service industry. These applications require efficient, intuitive ways to teach robots the motions they need to perform.

In recent times there has been a renewed interest in robot teleoperation, since it allows workers to accomplish their tasks remotely from home office or anywhere in the world. Unfortunately, network conditions play a significant role in the stability of teleoperated systems, and factors such as delays or reduced bandwidth can be decisive in the successful completion of even the simplest tasks. In KI.FABRIK we are developing methods to teach insertion skills from teleoperated demonstrations that combine visual and haptic information. Both streams of data are decoupled, which allows for easier provision of Quality of Service under adverse network conditions.

Ergonomic Design of a Teleoperation Workstation

In modern production systems, the skills, adaptability, and creativity of human operators remain crucial components. This becomes even more apparent as the number of variations in products increases and their lifecycles shorten, making automation of processes more challenging. Thus, it’s essential to not only optimize the technical systems and their software components, but to consider the human perspective when designing the interaction between humans and robots for teleoperation tasks. Our research focuses on how different interfaces between humans and machines affect users and overall performance. This includes technologies such as augmented, virtual, and mixed reality for visual feedback, as well as various input devices to ensure an effective and efficient human-machine system.

Contact

Diego Fernández

KI.FABRIK Research and Development (AI.Portal, PaaS)

Theresa Prinz

KI.FABRIK Research and Development (Central.AI, AI.Portal)