Exploring gaming technology. So amputees can learn to touch again.

When a group of researchers from the different fields of games design, engineering and healthcare met at a conference, it sparked a project which is changing the lives of amputees, with the potential to improve the way the NHS trains patients to use prosthetic limbs forever.

The Imagine Project is a conference at Sheffield Hallam University that brings together academics and researchers from different areas to come up with new ideas of how they could work collaboratively.

Games developer Ivan Phelan attended one of these conferences in 2013. He had been working with a piece of virtual reality (VR) hardware called Oculus Rift, and he was excited about its potential. 'It was amazing to me at the time,' he says, 'and I wanted to tell everybody about it.'

Some of the people he spoke to started talking about how virtual reality could be used with prosthetics, and soon they had formed a group to look into it. The idea was to harness the immersive experience provided by Oculus Rift to help amputees learn how to use prosthetic limbs.

Modern electric prosthetic limbs are very expensive – up to £30,000 each. Patients are given hours of training before they get them, but it can be difficult to take to a prosthetic and many go unused.

The project team realised that virtual reality offered patients an opportunity to 'see' the limb and adjust to using it. This would make it more likely that they would take to their bionic prosthetics – which bring life-changing benefits to amputees.

Kevin Everson has been wearing a prosthetic hand for 30 years, but when he was given a Bebionic hand he found he could do things he couldn’t do before, like cut his own steak, or stand up while riding his bike uphill.

'Things that everyday people take for granted make a real difference to me,' he says. 'The day I got the new hand I made a cup of coffee, and instead of putting the coffee jar under my armpit, I was able to grip it with my prosthetic hand for the first time. That meant a lot, because it took me back to how I was before I lost my hand.'

The project starts in earnest

The group met after the conference to fine-tune their plans. 'There was a lot of excitement,' Ivan recalls.

Everyone brought something different to the project – there was a physiotherapist, a health psychologist, a games designer, a materials engineer, an industrial designer, an analytical scientist and a 3D artist.

The team combined the Oculus Rift virtual reality with a Myo armband, which is worn on the end of the amputated limb. Using the same sensors for electrical activity as much of the wearable technology available on the market, the Myo can read the muscle activity of a user’s arm. It translates these messages into the virtual arm shown on the Oculus Rift headset.

It means the patient wearing the headset can see a virtual arm and move it around, grabbing things with the hand and generally practising how to move their muscles to control a prosthetic arm.

The big test

After spending time developing the technology and using themselves as test patients, Ivan and his team were ready to move on to the next stage – testing their design with an amputee.

They joined up with an NHS trust and patients at the Northern General hospital in Sheffield. Maddy Arden, a health psychologist working on the project, remembers, 'We were extremely nervous. We'd gone through months of preparation to get to this point and we had no idea if it would work or not.'

The project was a revelation. The first test patient exclaimed 'Brilliant!' as he used it.

'Using real-life examples by setting it in a kitchen really helped them use the prosthetic,' says Ivan. 'Our patient wanted to pick up an apple, and it was the first time he had been able to use the muscle movements in his stump to control a prosthetic limb. After taking part he wanted to change his cosmetic prosthetic limb for a Myo-electric one.'

Kevin Everson took part in the initial tests. 'It was totally immersive,' he says. 'Once I'd got used to the headset, opening and closing the virtual hand was exactly the same as using my prosthetic hand.

'The electrodes used the same muscle mass as my Bebionic hand, so I can see how it would be great training for amputees learning to use one of these prosthetics.'

Potential for the future

This research could change the way amputees train for their prosthetics, bringing a range of benefits both for the patient and for the NHS.

As well as making the training for prosthetics more effective, it could speed the process up. Ivan explains, 'Patients often have to travel for up to an hour and a half to get to the hospital, for several training sessions. With this technology, you could be able to use it at home instead.'

This research is at the cutting edge of technology, and with Oculus Rift virtual reality being developed and improved all the time – and becoming more affordable – the potential for the future is immense.

'The NHS contacts we're working with can see the potential. It’s now about working together with the NHS to build a big trial, with the hope of rolling it out nationally in the future,' says Ivan.

Physiotherapist Carol Garcia adds, 'We're leading the world at this research. We're getting approaches from burns patients and people with phantom limb pain. We were quite surprised no one had taken this technology in this direction before.'

This project could only have come about thanks to Sheffield Hallam University's commitment to bringing researchers together who would never normally work with each other.

Ivan says, 'Looking back at when I first started working with virtual reality, I probably wouldn’t have made the connection to healthcare. I'd have been walking around the room with the headset on thinking "this is really cool", and wanting to show people. But I wouldn’t have made the link to prosthetics without the opportunity provided by the Imagine Project.

'It's about being creative and open-minded and collaborative, and giving time to think about new ideas.'

‘Our patient picked up an apple, and it was the first time he had been able to use the muscle movements in his arm to control a prosthetic hand.’ Ivan Phelan
Research associate

The team who worked on this project included

  • Ivan Phelan
    Cultural Communication and Computing Research Institute, Sheffield Hallam University
  • Denise Eaton
    Centre for Science Education, Sheffield Institute of Education, Sheffield Hallam University
  • Professor Maddy Arden
    Professor of health psychology, Sheffield Hallam University
  • Professor Andrew Alderson
    Materials and Engineering Research Institute, Sheffield Hallam University
  • Carol Garcia
    Senior physiotherapy lecturer, Sheffield Hallam University
  • Heath Reed
    Art and Design Research Centre, Sheffield Hallam University
  • Dr Catherine Duckett
    Biomedical Research Centre, Sheffield Hallam University
  • Christine Le Maitre
    Biomedical Research Centre, Sheffield Hallam University
  • Steve Florence
    Technical officer, Sheffield Hallam University