New paper published on The International Journal of Robotics Research: “Cutaneous haptic feedback to ensure the stability of robotic teleoperation systems”

Cutaneous haptic feedback can be used to enhance the performance of robotic teleoperation systems while guaranteeing their safety. Delivering ungrounded cutaneous cues to the human operator conveys in fact information about the forces exerted at the slave side and does not affect the stability of the control loop.

In this work we analyze the feasibility, effectiveness, and implications of providing solely cutaneous feedback in robotic teleoperation.

Screenshot from 2015-10-20 09:57:36

We carried out two peg-in-hole experiments, both in a virtual environment and in a real (teleoperated) environment. Two novel 3-degree-of-freedom fingertip cutaneous displays deliver a suitable amount of cutaneous feedback at the thumb and index fingers. Results assessed the feasibility and effectiveness of the proposed approach.

Pacchierotti2015-IJRR

Cutaneous feedback was outperformed by full haptic feedback provided by grounded haptic interfaces, but it outperformed conditions providing no force feedback at all. Moreover, cutaneous feedback always kept the system stable, even in the presence of destabilizing factors such as communication delays and hard contacts.

Video: https://youtu.be/mQ8AYmNUBFo
PDF: http://sirslab.dii.unisi.it/papers/2015/Pacchierotti.IJRR.2015.Subtraction.pdf

C. Pacchierotti, L. Meli, F. Chinello, M. Malvezzi, D. Prattichizzo. Cutaneous haptic feedback to ensure the stability of robotic teleoperation systems. International Journal of Robotics Research, 2015. doi: 10.1177/0278364915603135.pdf

Human Guidance with Wearable Haptics: the research so far in our SIRSLab

This post summarises our contributions to the research on human guidance with wearable devices. One of our recent work “Evaluation of a predictive approach in steering the human locomotion via haptic feedback”, [1], has been recently accepted for publication in the proceedings of 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2015). In this work, we consider the human as an unicycle robot, following the work by Arechavaleta [2], and we exploit a path following control [3] for generating appropriate haptic (vibrotactile) cues to be applied on the user. The task here is to follow some ideal lines, with no knowledge of them. This is just one examples of the application of the wearable haptics – in this case, with vibrotactile feedack – for human guidance.

This paper is an evolution of one of our previous works [4], in which we were guiding the human considering her/him inside a mixed human-robot formation, where the role of leader and follower was, in some sense, blended. Here, vibrotactile cues were used to tune the position of the human user, so that she/he was maintaining a rigid formation w.r.t. to the robot, while it was moving toward a target.

We exploited also the idea of considering the human as leader of a mixed human-robot team [5, 6]. In these cases, the haptic feedback was used to notify the user about violations of formation constraints, so that she/he could modify her/his pace and maintain the formation.

Most of our results have received funding from the European Union Seventh Framework Programme FP7/2007-2013 under grant agreement n. 601165 of the project “WEARHAP – WEARable HAPtics for humans and robots” and under grant agreement n. 288917 of the project “DALi – Devices for Assisted Living”, and from the European Union’s Horizon 2020 research and innovation programme – Societal Challenge 1 (DG CONNECT/H) under grant agreement n. 643644 of the project “ACANTO: A CyberphysicAl social NeTwOrk using robot friends”.

Publications/Videos/pdf also available on our website (sirslab.dii.unisi.it)

[1] M. Aggravi, S. Scheggi, and D. Prattichizzo – Evaluation of a predictive approach in steering the human locomotion via haptic feedback – IROS, 2015 [pdf]

[2] G. Arechavaleta, J.-P. Laumond, H. Hicheur, and A. Berthoz – On the nonholonomic nature of human locomotion – Autonomous Robots, 2008

[3] C. Canudas De Wit, G. Bastin, and B. Siciliano – Theory of robot control – Chapter 9 Nonlinear Feedback Control

[4] S. Scheggi, M. Aggravi, F. Morbidi, and D. Prattichizzo – Cooperative human-robot haptic navigation – ICRA, 2014, [pdf]

[5] S. Scheggi, F. Morbidi, and D. Prattichizzo. Human-robot formation control via visual and vibrotactile haptic feedback – IEEE Trans. on Haptics, 2014, [pdf]

[6] S. Scheggi, F. Chinello, D. Prattichizzo. Vibrotactile haptic feedback for human-robot interaction in leader-follower tasks. In Proc. ACM Int. Conf. on PErvasive Technologies Related to Assistive Environments, PETRA ’12, Pages 1-4, 2012, [pdf]

New paper accepted for publication in the Proceedings of the 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2015)

In this paper, we present a haptic guidance policy to steer the user along predefined paths and we evaluate a predictive approach to compensate actuation delays that humans have when they are guided along a given trajectory via sensory stimuli.

The proposed navigation policy exploits the nonholonomic nature of human locomotion in goal directed paths, which leads to a very simple guidance mechanism. The proposed method has been evaluated in a real scenario where seven human subjects were asked to walk along a set of predefined paths via vibrotactile cues. Their poses as well as the related distances from the path have been recorded using an accurate optical tracking system.

Results revealed that an average error of 0.24 m is achieved by using the proposed haptic policy, and that the predictive approach does not bring significant improvements to the path following problem for what concerns the distance error. On the contrary, the predictive approach achieved a definitely lower activation time of the haptic interfaces.

M. Aggravi, S. Scheggi, D. Prattichizzo. Evaluation of a predictive approach in steering the human locomotion via haptic feedback. In Proc. IEEE/RSJ International Conference Intelligent Robots and Systems. In Press, 2015.

New paper published on the IEEE Transactions on Biomedical Engineering: “Cutaneous Feedback of Fingertip Deformation and Vibration for Palpation in Robotic Surgery”

Despite its expected clinical benefits, current teleoperated surgical robots do not provide the surgeon with haptic feedback largely because grounded forces can destabilize the system’s closed-loop controller.

This article presents an alternative approach that enables the surgeon to feel fingertip contact deformations and vibrations while guaranteeing the teleoperator’s stability.

We implemented our cutaneous feedback solution on an Intuitive Surgical da Vinci Standard robot by mounting a SynTouch BioTac tactile sensor to the distal end of a surgical instrument and a custom cutaneous display to the corresponding master controller. As the user probes the remote environment, the contact deformations, DC pressure, and AC pressure (vibrations) sensed by the BioTac are directly mapped to input commands for the cutaneous device’s motors using a model-free algorithm based on look-up tables. The cutaneous display continually moves, tilts, and vibrates a flat plate at the operator’s fingertip to optimally reproduce the tactile sensations experienced by the BioTac.

We tested the proposed approach by having eighteen subjects use the augmented da Vinci robot to palpate a heart model with no haptic feedback, only deformation feedback, and deformation plus vibration feedback. Fingertip deformation feedback significantly improved palpation performance by reducing the task completion time, the pressure exerted on the heart model, and the subject’s absolute error in detecting the orientation of the embedded plastic stick. Vibration feedback significantly improved palpation performance only for the seven subjects who dragged the BioTac across the model, rather than pressing straight into it.

system

C. Pacchierotti, D. Prattichizzo, K. J. Kuchenbecker. Cutaneous feedback of fingertip deformation and vibration for palpation in robotic surgery. IEEE Transactions on Biomedical Engineering. In Press, 2015. pdf

Teleoperation of steerable flexible needles by combining kinesthetic and vibratory feedback

A new journal article has been accepted for publication in the IEEE Transactions on Haptics!

Reference:
C. Pacchierotti, M. Abayazid, S. Misra, D. Prattichizzo. “Teleoperation of steerable flexible needles by combining kinesthetic and vibratory feedback”. IEEE Transactions on Haptics, 2015.

PaAbMiPr-TOH15
Abstract:
Needle insertion in soft-tissue is a minimally invasive surgical procedure that demands high accuracy. In this respect, robotic systems with autonomous control algorithms have been exploited as the main tool to achieve high accuracy and reliability. However, for reasons of safety and responsibility, autonomous robotic control is often not desirable. Therefore, it is necessary to focus also on techniques enabling clinicians to directly control the motion of the surgical tools. In this work we address that challenge and present a novel teleoperated robotic system able to steer flexible needles.

The proposed system tracks the position of the needle using an ultrasound imaging system and computes needle’s ideal position and orientation to reach a given target. The master haptic interface then provides the clinician with mixed kinesthetic-vibratory navigation cues to guide the needle toward the computed ideal position and orientation.
Twenty participants carried out an experiment of teleoperated needle insertion into a soft-tissue phantom, considering four different experimental conditions. Participants were provided with either mixed kinesthetic-vibratory feedback or mixed kinestheticvisual feedback. Moreover, we considered two different ways of computing ideal position and orientation of the needle: with or without set-points. Vibratory feedback was found more effective than visual feedback in conveying navigation cues, with a mean targeting error of 0.72 mm when using set-points, and of 1.10 mm without set-points.

Full-text PDF:
http://sirslab.dii.unisi.it/papers/2015/Pacchierotti.TOH.2015.Haptics.Fin.pdf

VIDEO:

Wearable Haptics and Hand Tracking @ SIGGRAPH 2014

Our lab is heading to Vancouver to present the paper

Wearable Haptics and Hand Tracking via an RGB-D Camera
for Immersive Tactile Experiences
.

to the upcoming SIGGRAPH 2014 conference.

Screenshot from 2014-08-08 10:52:52
Enjoy the video!

One of our journal papers listed as the most popular TOH paper in IEEEXplore

The journal paper “Towards Wearability in Fingertip Haptics: A 3-DoF Wearable Device for Cutaneous Force Feedback” by D. Prattichizzo, F. Chinello, C. Pacchierotti, and M. Malvezzi is now listed in IEEEXplore as the most popular IEEE Transactions on Haptics paper, with over 400 downloads!

 

IEEE Xplore: Haptics, IEEE Transactions on - (Home) - Google Chrome_003IEEE Xplore Abstract (Metrics) - Towards Wearability in Fingertip Haptics: A 3-DoF Wearable Device for Cutaneous Force Feedback - Google Chrome_004

You can download the paper from here.