Francesco presenting his work at the Haptics Symposium in Philadelphia #haptics2016

Paper on “THE” Project published on Physics of Life Reviews

An article summarising the results of the four years EU project “The Hand Embodied – THE “ has been published on the prestigious journal Physics of Life Reviews. Here the link to the article. As University of Siena, we have contributed to the modelling of hand synergies and we have studied a systematic way to transfer human hand skills onto robotic hands. These results can be found here.

Abstract of the paper
phyLif

The term ‘synergy’ – from the Greek synergia – means ‘working together’. The concept of multiple elements working together towards a common goal has been extensively used in neuroscience to develop theoretical frameworks, experimental approaches, and analytical techniques to understand neural control of movement, and for applications for neuro- rehabilitation. In the past decade, roboticists have successfully applied the framework of synergies to create novel design and control concepts for artificial hands, i.e., robotic hands and prostheses. At the same time, robotic research on the sensorimotor integration underlying the control and sensing of artificial hands has inspired new research approaches in neuroscience, and has provided useful instruments for novel experiments.

The ambitious goal of integrating expertise and research approaches in robotics and neuroscience to study the properties and applications of the concept of synergies is generating a number of multidisciplinary cooperative projects, among which the recently finished 4-year European project “The Hand Embodied” (THE). This paper reviews the main insights provided by this framework. Specifically, we provide an overview of neuroscientific bases of hand synergies and introduce how robotics has leveraged the insights from neuroscience for innovative design in hardware and controllers for biomedical engineering applications, including myoelectric hand prostheses, devices for haptics research, and wearable sensing of human hand kinematics. The review also emphasizes how this multidisciplinary collaboration has generated new ways to conceptualize a synergy-based approach for robotics, and provides guidelines and principles for analyzing human behavior and synthesizing artificial robotic systems based on a theory of synergies.

New paper published on Medical & Biological Engineering & Computing “Hand–tool–tissue interaction forces in neurosurgery for haptic rendering”

Haptics provides sensory stimuli that represent the interaction with a virtual or tele-manipulated object, and it is considered a valuable navigation and manipulation tool during tele-operated surgical procedures. Haptic feedback can be provided to the user via cutaneous information and kinesthetic feedback.

 

deviceSensory subtraction removes the kinesthetic component of the haptic feedback, having only the cutaneous component provided to the user. Such a technique guarantees a stable haptic feedback loop, while it keeps the transparency of the tele-operation system high, which means that the system faithfully replicates and render back the user’s directives.

 

figure1This work focuses on checking whether the interaction forces during a bench model neurosurgery operation can lie in the solely cutaneous perception of the human finger pads. If this assumption is found true, it would be possible to exploit sensory subtraction techniques for providing surgeons with feedback from neurosurgery. We measured the forces exerted to surgical tools by three neurosurgeons performing typical actions on a brain phantom, using contact force sensors, whilst the forces exerted by the tools to the phantom tissue were recorded using a load cell placed under the brain phantom box. The measured surgeon-tool contact forces were 0.01 – 3.49 N for the thumb and 0.01 – 6.6 N for index and middle finger, whereas the measured tool- tissue interaction forces were from six to eleven times smaller than the contact forces, i.e., 0.01 – 0.59 N.

 

Fingerprint_detail_on_male_finger_smallThe measurements for the contact forces fit the range of the cutaneous sensitivity for the human finger pad, thus, we can say that, in a tele-operated robotic neurosurgery scenario, it would possible to render forces at the fingertip level by conveying haptic cues solely through the cutaneous channel of the surgeon’s finger pads. This approach would allow high transparency and high stability of the haptic feedback loop in a tele-operation system.

 

PDF: http://sirslab.dii.unisi.it/papers/2015/Aggravi.MBEC.2015.Surgeons.pdf

M. Aggravi, E. De Momi, F. DiMeco, F. Cardinale, G. Casaceli, M. Riva, G. Ferrigno, D. Prattichizzo, D.
“Hand-Tool-Tissue Interaction Forces in Neurosurgery for Haptic Rendering.”
Medical & Biological Engineering and Computing, Springer, 2015.
DOI: 10.1007/s11517-015-1439-8

Silicon Valley 2016 – Intuitive Surgical Technology Research Grant Symposium

Silicon Valley 2016, Intuitive Surgical Technology Research Grant Symposium http://www.intuitivesurgical.com/

12507134_10208629291997965_7814823607378336411_nHave a look to the photo album of the event.
– The idea we had in our lab #sirslab about using the cutaneous-only (no kinesthetic) haptic feedback in robot-assisted surgery, like the da Vinci system, was a great idea and it got the prestigious Intuitive Surgical Research Grant in 2015 (the only Italians) with a collaborative project with University of Pennsylvania (K.J. Kuchenbecker). Claudio Pacchierotti and I have been in California to present the results of the research based on our idea. We gave talks to surgeons and engineers of Intuitive Surgical in Santa Clara in the Silicon Vally and it was amazing. A lot of great conversations, ideas, and comments. We are coming back to Italy with more energy :-).

Screen Shot 2016-01-10 at 01.41.03
If you want to know more about the idea of cutaneous feedback in surgical robotics have a look to this paper

– L. Meli, C. Pacchierotti, D. Prattichizzo. Sensory subtraction in robot-assisted surgery: fingertip skin deformation feedback to ensure safety and improve transparency in bimanual haptic interaction. IEEE Transactions on Biomedical Engineering, 61(4):1318-1327, 2014

and to the paper where the idea has been implemented in the da Vinci System in

– C. Pacchierotti, D. Prattichizzo, K. J. Kuchenbecker. Cutaneous feedback of fingertip deformation and vibration for palpation in robotic surgery. IEEE Transactions on Biomedical Engineering. In Press, 2015

_DP

 

 

New paper published on IEEE Transactions on Haptics: “Digital Handwriting with a Finger or a Stylus: a Biomechanical Comparison”

The human hand is highly versatile and easily adaptable to a variety of manipulation tasks, exposing flexible solutions to the needs of control. In daily life, humans beings are, apparently without effort, able to generate complex and elegant movements of the hand and fingers, such as typing on keyboards, playing a musical instrument, or writing. 

In this paper we focus on the analysis of human hand movements during handwriting tasks, a subject which has been studied for many decades.

Screen shot 2015-12-15 at 1.04.49 PM

We present for the first time, at the best of our knowledge, a methodological approach based on the biomechanics of the human hand to compare two different input methods, i.e., the finger and the stylus, in digital handwriting tasks.

Performance of two input methods is evaluated and compared in terms of manipulability indexes in the task space, i.e., the ratio between a measure of performance (displacement, velocity, force in the task space) and a measure of effort in the input/joint space.

Screen shot 2015-12-15 at 1.14.02 PMBeside the mathematical analysis based on a biomechanical model of the hand, two experiments are presented, in which subjects were asked to write on a touchscreen using either their index finger, or a stylus.

The results (both analytical and experimental) assess that writing with the finger is more suitable for performing large, but not very accurate motions, while writing with the stylus leads to a higher precision and more isotropic motion performance.

PDF: http://sirslab.dii.unisi.it/papers/2015/Prattichizzo.TOH.2015.Handwriting.pdf

D. Prattichizzo, L. Meli, and M. Malvezzi.
“Digital Handwriting with a Finger or a Stylus: a Biomechanical Comparison.”
IEEE Transactions on Haptics, 2015.
DOI: 10.1109/TOH.2015.2434812

The Robotic Sixth Finger: a wearable extra limb to compensate hand function in chronic post stroke patient

blog
Fig. 1. The Robotic Sixth Finger concept. The device is worn like a bracelet and pops up when needed.

This post summarises our research on wearable extra fingers. We started to investigate how to enhance the capability of the human hand by means of wearable robots in 2011 [1]. The goal was to integrate the human hand with an additional robotic finger as represented in Fig. 1. We firstly investigate the potentials of extra-finger in healthy subjects. Such devices could give humans the possibility to manipulate objects in a more efficient way, enhancing our hand grasping dexterity/ability. The first prototype has been presented in [2] together with several examples of the extra-finger applications. Together with the design issues related to portability and wearability of the devices, another critical aspect was integrating the motion of the extra–fingers with that of the human hand. In [3], we presented a mapping algorithm able to transfer to the extra–fingers a part or the whole motion of the human hand. A commercial dataglove was used to measure the hand configuration during a grasping task. A video is available here. Although this control approach guarantees a reliable tracking of the human hand, there was two main drawbacks to be solved. First, the user lacked a feedback of the robotic finger status and could only perceive the force

device-1_red
Fig. 2. The Robotic Sixth Finger together with the vibrotactile interface ring.

exerted by the device mediated by the grasped object. The second problem was related to the approaching phase of the grasp. In fact, the algorithm presented in [3] considers the motion of the whole hand to compute the motion of the extra finger, thus limiting the possibility of the user to make fine adjustments to adapt the finger shape to that of the grasped object. In [4] we addressed these issues by introducing a vibrotactile interface that can be worn as a ring. The human user receives information through the vibrotactile interface about the robotic finger status in terms of contact/no contact with the grasped object and in terms of force exerted by the device. Regarding the grasp approaching phase, we introduced a new control strategy that enables the finger to autonomously adapt to the shape of the grasped object.

Setup_icorr_robotic
Fig. 3. The Robotic Sixth Finger for hand grasping compensation in chronic stroke patients.

The experience gained with healthy subjects was fundamental for the development of Robotic Sixth Finger for compensating hand function in chronic stroke patients. We proposed to use a robotic the Robotic Sixth Finger together with the paretic hand/arm, to constrain the motion of the object. The device can be worn on the user’s forearm by means of an elastic band. The systems acts like a two-finger gripper, where one finger is represented by the Robotic Sixth Finger, while the other by the patient’s paretic limb. The patient can regulate the finger flexion/extension through a wearable switch embedded in a ring worn on the healthy hand. Two possible predefined motions can be chosen to obtain either a precision or a power grasp. In addition to the switch, the proposed ring interface also embeds a vibrotactile motor able to provide the patient with information about the force exerted by the device. The preliminary results with patients are presented in [5] and a video is available here.

Related publications

[1] O. A. Atassi, “Design of a robotic sixth finger for grasping enhancement,” Master’s thesis, Universita` degli Studi di Siena (advisor: Domenico Prattichizzo), December 2011.

[2] D. Prattichizzo, M. Malvezzi, I. Hussain, G. SalviettiThe Sixth-Finger: a Modular Extra-Finger to Enhance Human Hand Capabilities. In Proc. IEEE Int. Symp. in Robot and Human Interactive Communication, Pages 993-998, Edinburgh, United Kingdom, August 2014.

[3] D. Prattichizzo, G. Salvietti, F. Chinello, M. MalvezziAn Object-based Mapping Algorithm to Control Wearable Robotic Extra-Fingers. In Proc. IEEE/ASME Int. Conf. on Advanced Intelligent Mechatronics, Pages 1563-1568, Besançon, France, July, 2014.

[4] I. Hussain, L. Meli, C. Pacchierotti, G. Salvietti, D. PrattichizzoVibrotactile haptic fedback for intuitive control of robotic extra fingers. In Proc. IEEE World Haptics Conference (WHC), Chicago, IL, June, 2015.

[5] I. Hussain, G. Salvietti, L. Meli, C. Pacchierotti, D. PrattichizzoUsing the robotic sixth finger and vibrotactile feedback for grasp compensation in chronic stroke patients. In Proc. IEEE/RAS-EMBS International Conference on Rehabilitation Robotics (ICORR), Singapore, Republic of Singapore, 2015. [Finalist for the Best Student Paper Award]

[6] D. Prattichizzo. The interplay between humans and robots in grasping. In Proc. International Symposium on Robotic Research, Sestri Levante, Italy, September, 2015

[7] I. Hussain, G. Salvietti, M. Malvezzi and D. Prattichizzo. Design guidelines for a wearable robotic extra-finger. In proc. IEEE Int. Forum on Research and Technology for Society and Industry, Turin, Italy September, 2015

Human Guidance with Wearable Haptics: the research so far in our SIRSLab

This post summarises our contributions to the research on human guidance with wearable devices. One of our recent work “Evaluation of a predictive approach in steering the human locomotion via haptic feedback”, [1], has been recently accepted for publication in the proceedings of 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2015). In this work, we consider the human as an unicycle robot, following the work by Arechavaleta [2], and we exploit a path following control [3] for generating appropriate haptic (vibrotactile) cues to be applied on the user. The task here is to follow some ideal lines, with no knowledge of them. This is just one examples of the application of the wearable haptics – in this case, with vibrotactile feedack – for human guidance.

This paper is an evolution of one of our previous works [4], in which we were guiding the human considering her/him inside a mixed human-robot formation, where the role of leader and follower was, in some sense, blended. Here, vibrotactile cues were used to tune the position of the human user, so that she/he was maintaining a rigid formation w.r.t. to the robot, while it was moving toward a target.

We exploited also the idea of considering the human as leader of a mixed human-robot team [5, 6]. In these cases, the haptic feedback was used to notify the user about violations of formation constraints, so that she/he could modify her/his pace and maintain the formation.

Most of our results have received funding from the European Union Seventh Framework Programme FP7/2007-2013 under grant agreement n. 601165 of the project “WEARHAP – WEARable HAPtics for humans and robots” and under grant agreement n. 288917 of the project “DALi – Devices for Assisted Living”, and from the European Union’s Horizon 2020 research and innovation programme – Societal Challenge 1 (DG CONNECT/H) under grant agreement n. 643644 of the project “ACANTO: A CyberphysicAl social NeTwOrk using robot friends”.

Publications/Videos/pdf also available on our website (sirslab.dii.unisi.it)

[1] M. Aggravi, S. Scheggi, and D. Prattichizzo – Evaluation of a predictive approach in steering the human locomotion via haptic feedback – IROS, 2015 [pdf]

[2] G. Arechavaleta, J.-P. Laumond, H. Hicheur, and A. Berthoz – On the nonholonomic nature of human locomotion – Autonomous Robots, 2008

[3] C. Canudas De Wit, G. Bastin, and B. Siciliano – Theory of robot control – Chapter 9 Nonlinear Feedback Control

[4] S. Scheggi, M. Aggravi, F. Morbidi, and D. Prattichizzo – Cooperative human-robot haptic navigation – ICRA, 2014, [pdf]

[5] S. Scheggi, F. Morbidi, and D. Prattichizzo. Human-robot formation control via visual and vibrotactile haptic feedback – IEEE Trans. on Haptics, 2014, [pdf]

[6] S. Scheggi, F. Chinello, D. Prattichizzo. Vibrotactile haptic feedback for human-robot interaction in leader-follower tasks. In Proc. ACM Int. Conf. on PErvasive Technologies Related to Assistive Environments, PETRA ’12, Pages 1-4, 2012, [pdf]