Shared Haptic Virtual Environments can be realized using a client-server architecture. In this architecture, each client maintains a local copy of the virtual environment (VE). A centralized physics simulation running on a server calculates the object states based on haptic device position information received from the clients. The object states are sent back to the clients to update the local copies of the VE, which are used to render interaction forces displayed to the user through a haptic device.

Communication delay leads to delayed object state updates and increased force feedback rendered at the clients. In this work, we analyze the effect of communication delay on the magnitude of the rendered forces at the clients for cooperative multi-user interactions with rigid objects. The analysis reveals guidelines on the tolerable communication delay. If this delay is exceeded, the increased force magnitude becomes haptically perceivable.We propose an adaptive force rendering scheme to compensate for this effect, which dynamically changes the stiffness used in the force rendering at the clients. Our experimental results, including a subjective user study, verify the applicability of the analysis and the proposed scheme to compensate the effect of time-varying communication delay in a multi-user SHVE.

Publications:

C. Schuwerk, X. Xu, R. Chaudhari, E. Steinbach,

Compensating the Effect of Communication Delay in Client-Server-Based Shared Haptic Virtual Environments,
ACM Transactions on Applied Perception, vol. 13, no. 1, pp. 1-22, December 2015.

In this work, we study Model-Mediated Teleoperation (MMT) for objects which can be moved and rotated on a planar surface. In MMT, a simple object model is employed on the master side to approximate the remote environment. The haptic feedback is rendered locally on the master side based on the model whose parameters are estimated on the slave side. Since the haptic control loop is running locally without noticeable delay, system stability can be guaranteed in the presence of arbitrary communication delays. The main challenges for MMT are environment modeling and online parameter identification. This work has two contributions. Firstly, we describe how to model movable objects with approximately known model geometry for MMT systems. The object dynamics including environment damping as well as the friction are modeled in real time during teleoperation. Secondly, we propose a hybrid deadband-based and time-triggered updating scheme to adaptively reduce the required packet rate while transmitting the estimated model parameters from the slave to the master. According to our experiments, the movable object can be accurately modeled in real time. In addition, compared to the conventional deadbandbased updating scheme, our proposed hybrid updating scheme shows improvements in both model accuracy and packet rate reduction.

Publications:

X. Xu, S. Chen, E. Steinbach, 
Model-mediated Teleoperation for Movable Objects: Dynamics Modeling and Packet Rate Reduction, 
14th IEEE International Symposium on Haptic Audio-Visual Environments and Games, Ottawa, Canada, October 2015.

We propose a perceptual haptic data reduction approach for teleoperation systems which uses the time domain passivity approach (TDPA) as their control architecture for dealing with time-varying delay. Our goal is to reduce the packet rate over the communication network while preserving system stability in the presence of time-varying and unknown communication delays. Compared to the existing wave variable-based (WV-based) haptic data reduction approaches, our proposed scheme leads to smaller distortion in the force signals and is flexible and robust to time-varying delays. Experiments show that our proposed approach can reduce the packet rate by up to 80%, without introducing significant distortion. In addition, the proposed approach outperforms the existing WV-based approaches in both packet rate reduction and subjective preference for the tested delays. 

Publications:

X. Xu, C. Schuwerk, E. Steinbach, 
Passivity-based Model Update for Model-mediated Teleoperation,
6th IEEE International Workshop on Hot Topics in 3D - Hot3D, Proc. of IEEE International Conference on Multimedia & Expo, Turin, Italy, June 2015.

In this work we describe a client-server architecture for haptic interaction with simulated deformable objects. The computationally expensive object deformation is computed on the server at a low temporal update rate and transmitted to the clients. There, an intermediate representation of the deformable object is used to locally render haptic force feedback displayed to the user at the required rate of 1 kHz. Based on a one-dimensional deformable object, we analyze the transparency of this multi-rate architecture for a single client interaction. The delay introduced by the deformation simulation and the client-server communication leads to increased rendered forces at the clients compared to a reference scenario without delay. We propose a method that adaptively adjusts the stiffness used in the local force rendering at the client to compensate for this. The evaluation shows that the proposed method successfully compensates the effect of delay in the tested delay range of up to 100 ms. 

Publications:

C. Schuwerk, X. Xu, W. Freund, E. Steinbach,
Transparency Analysis of Client-Server-based Multi-rate Haptic Interaction with Deformable Objects,
IEEE World Haptics Conference, Chicago, USA, June 2015.

While stroking a rigid tool over an object surface, vibrations induced on the tool, which represent the interaction between the tool and the surface texture, can be measured by means of an accelerometer. Such acceleration signals can be used to recognize or to classify object surface textures. The temporal and spectral properties of the acquired signals, however, heavily depend on different parameters like the applied force on the surface or the lateral velocity during the exploration. Robust features that are invariant against such scan-time parameters are currently lacking, but would enable texture classification and recognition using uncontrolled human exploratory movements. In this work, we introduce a haptic texture database which allows for a systematic analysis of feature candidates. The publicly available database includes recorded accelerations measured during controlled and well-defined texture scans, as well as uncontrolled human free hand texture explorations for 43 different textures. As a preliminary feature analysis, we test and compare six well-established features from audio and speech recognition together with a Gaussian Mixture Model-based classifier on our recorded free hand signals. Among the tested features, best results are achieved using Mel-Frequency Cepstral Coefficients (MFCCs), leading to a texture recognition accuracy of 80.2%. Please visit our LMT Haptic Texture Database website for more details.

Publications:

M. Strese, J. Lee, C. Schuwerk, Q. Han, H. Kim, E. Steinbach,
A Haptic Texture Database for Tool-mediated Texture Recognition and Classification,
Proc. of IEEE Int. Symposium on Haptic Audio-Visual Environments and Games (HAVE), Dallas, Texas, USA, October 2014.

Mobile systems interacting with objects in unstructured environments require both haptic and visual sensors to acquire sufficient scene knowledge for tasks such as navigation and manipulation. Typically, separate sensors and processing systems are used for the two modalities. We propose to acquire haptic and visual measurements simultaneously, providing naturally coherent data. For this, compression of a passive, deformable foam rod mountedon the actuator is measured visually by a low-cost camera, yielding a 1D stress function sampled along the contour of the rod. The same camera observes the nearby scene to detect objects and their reactions to manipulation. The system is passively compliant and the complexity of the sensor subsystems is reduced. Furthermore, we present an integrated approach for navigation and manipulation on mobile platforms which integrates haptic data from the sensor. A high-level planning graph represents both the structure of a visually acquired map, as well as manipulable obstacles. Paths within this graph represent high-level navigation and manipulation tasks, e.g. pushing of obstacles. A cost-optimal task plan is generated using standard pathfinding techniques. The approach is implemented and validated on a mobile robotic platform. Obtained forces are compared to a reference, showing high accuracy within the medium sensor range. A real-world experiment is presented which uses the sensor for haptic exploration of obstacles in anoffice environment. Substantially faster task plans can be found in cluttered scenes compared to purely visual navigation.

Publications:

N. Alt, E. Steinbach,
Navigation and Manipulation Planning using a Visuo-haptic Sensor on a Mobile Platform,
IEEE Transactions on Instrumentation and Measurement, vol. 63, no. 11, 2014.
 

Shared Haptic Virtual Environments (SHVEs) are often realized using a client-server architecture. At the server, a physics simulation engine calculates the object states based on the position information received from the clients. At the clients, the object state information, received from the server, is used to update the local copy of the virtual environment. Forces displayed to the user through a haptic device are computed locally at the client based on the interactions with the objects. Communication delay leads to delayed object state updates and increased interaction forces rendered at the clients. Users perceive this as increased object weight. In this work, we systematically analyze the loss of transparency caused by communication delay and propose a novel adaptive force rendering scheme to compensate for it. The proposed scheme reduces the objects’ stiffness at the clients, based on delay, device velocity and motion constraints of the objects, only if necessary to achieve perceptual transparency. Simulations and subjective evaluations show that the effect of increased weight of objects is successfully compensated for the tested delay range of up to 150 ms. At the same time, if the object is unmovable, the perception of interaction with a rigid object is preserved. 

Publications:

C. Schuwerk, R. Chaudhari, E. Steinbach,
Delay Compensation in Shared Haptic Virtual Environments,
IEEE Haptics Symposium (HAPTICS), Houston, USA, Februar 2014.

We introduced three different uses of linear regression for improving the prediction of samples in haptic communication and compared this technique to the commonly employed zero-order and first-order linear predictors. We coupled the prediction techniques with (error resilient) perceptual data reduction approaches and evaluated their robustness when losses in the network are present. Experimental results showed that the proposed prediction technique improved haptic data reduction while keeping lower signal distortion compared to the traditional prediction methods when facing adverse network conditions.

Publications:

F. Brandi, E. Steinbach,
Prediction Techniques for Haptic Communication and their Vulnerability to Packet Losses,
Proc. of IEEE Int. Symposium on Haptic Audio-Visual Environments and Games (HAVE), Istanbul, Turkey, Oktober 2013.

Communication architectures conceived for Shared Haptic Virtual Environments (SHVEs) are based on either the client-server or the peer-to-peer paradigms. High-rate rendering and communication between collaborating users quickly leads to performance bottlenecks, if the virtual environment’s size and complexity or the number of collaborating users increases. We propose a decentralized communication architecture for SHVEs, which exploits areas of interest (AoI) of a user to dynamically form smaller communication groups. High-rate information ex- change following the client-server paradigm is used within these groups to support satisfactory haptic collaboration and consistency. A single group member is selected to simulate the object states and to relay this state to the other group members. Peer-to-peer inter-group communication is also used, based on spatial proximity, to further reduce the overall communication load. We implemented a prototype based on our proposed architecture and present some first evaluation results. They serve as a proof-of-concept and show the effectiveness of dynamic group maintenance in conjunction with additional traffic control schemes.

Publications:

C. Schuwerk, N. Chaudhari, E. Steinbach,
An Area-of-Interest based Communication Architecture for Shared Haptic Virtual Environments,
Proc. of IEEE Int. Symposium on Haptic Audio-Visual Environments and Games (HAVE), Istanbul, Turkey, Oktober 2013.

In this work we study the position and velocity signal reconstructions using predictive coding when packets are lost during telemanipulation sessions and classify the high-level haptic artifacts perceived by the users. The usage of packet-switched networks for bilateral telemanipulation systems is challenging due to several adversities such as low transmission rates, packet delays, jitter and losses. The previously proposed deadband-based haptic data reduction approaches se- lectively decrease the high transmission rate of the force-feedback and position/velocity samples on account of human perception limitations. Recently, an error-resilient perceptual haptic data reduction approach was proposed to address the packet losses in the feedback channel. However, the impact of faltered transmission on the forward channel and its subjective influence on the user are still an open issue and thus are treated in this paper. 

Publications:

F. Brandi, B. Cizmeci, E. Steinbach,
On the Perceptual Artifacts Introduced by Packet Losses on the Forward Channel of Haptic Telemanipulation Sessions,
Proc. of EuroHaptics Conference, Tampere, Finland, Juni 2012.

In this work we study traffic control for server to client communication in distributed haptic virtual environments (VE). We adopt a client-server architecture where the server manages the state consistency of the distributed VE, while haptic feedback is computed locally at each client. The update rate of network traffic from the server to the client is dynamically adapted by exploiting characteristics and limitations of human haptic perception. With this, an excellent trade- off between network communication efficiency and perceptually robust rendering of haptic feedback is achieved. Subjective tests with two users collaboratively manipulating a common object show a packet rate reduction of up to 99% from the server to the clients without deteriorating haptic feedback quality.

Publications:

C. Schuwerk, R. Chaudhari, E. Steinbach,
Perceptually Robust Traffic Control in Distributed Haptic Virtual Environments,
Proc. of EuroHaptics Conference, Tampere, Finland, Juni 2012.

For realtime teleoperation with haptic feedback, network-induced artifacts like delay, packet loss and lossy coding of haptic data deteriorate the operability of the system. We provide a systematic quantification of the limits within which, these network-induced degradations are tolerable for human task performance in executing a given haptic task. These limits are conservative in the sense that we do not account for any stabilizing approaches from control engineering to counter the impact of the network-induced degradations. Through experimental evaluation with a pursuit tracking task, we show that task performance is most affected by delay on the network and a strong lossy coding scheme. With statistical analysis we show that task performance is significantly decreased for one-way delay higher than 14 ms and strong lossy coding with a deadband parameter higher than 27 %. For the given task, packet loss is found not to affect task performance significantly.

Publications:

R. Chaudhari, C. Schuwerk, V. Nitsch, E. Steinbach, B. Faerber,
Opening the Haptic Loop: Network Degradation Limits for Haptic Task Performance,
IEEE International Symposium on Haptic Audio Visual Environments and Games, Nanchang, Jiangxi, China, Oktober 2011.