In 2019, Jacob George, an assistant professor in the University of Utah Department of Electrical and Computer Engineering and the Department of Physical Medicine and Rehabilitation, and fellow researchers at the U made waves with the “LUKE Arm.” Named after the robotic hand given to Luke Skywalker in The Empire Strikes Back, the team’s prosthetic arm has the ability to feel objects by transmitting the appropriate signals to the brain. Not only can the arm feel, but it can be controlled by the user’s thoughts.

Members of the campus and broader community are invited to attend a lecture from George scheduled for Wednesday, Sept. 13 at 11 a.m. in the Gould Auditorium on the first level of the U’s J. Willard Marriott Library.

As the featured speaker for The William R. and Erlyn J. Gould Distinguished Lecture on Technology and the Quality of Life, George will highlight this technology and the advances that have taken place since their breakthrough technology was first announced—including translation into bionic exoskeletons to aid individuals with paralysis and into smartwatches that could allow anyone to seamlessly control smart devices and augmented reality interfaces by thought.

Our hands represent a large part of what we can do and who we are. For individuals suffering from limb loss or paralysis, loss of hand function often results in physical disability and non-autonomy. At the University of Utah, George’s team uses electrical interfaces with the arm nerves and muscles to create a bidirectional link to an individual’s nervous system. Using this brain-computer-interface technology, George and colleagues have enabled amputees to control a dexterous prosthetic arm entirely by thought. The advanced prosthetic arm was also endowed with a natural sense of touch, allowing users to have a more delicate touch and manipulate fragile objects without breaking them.

“It might sound like science fiction, but it’s real,” said George. “We control our hands with our thoughts all the time. When we think about moving our hands, information starts in the brain as a series of electrical impulses. Those electrical signals then travel down the spinal cord to the peripheral nerves where they cause the muscles to contract, which then causes the hand to move.”

By tapping into those electrical signals, George’s team can determine what an individual is trying to do with their hands. “Even after an amputation, or after a stroke where the hand is too weak to physically move, we can record the electrical signals, amplify them on a computer and then use machine learning and artificial intelligence to decode what a person is trying to do,” continues George.

But George’s technology does more than just allow individuals to control devices with their thoughts. The technology also provides users with a natural sense of touch. “When our hands make contact with objects, mechanical forces are converted into electrical signals that propagate up the nerves to the brain, where the brain interprets them as touch,” explains George. “If we do our jobs right, and recreate those electrical signals exactly, then the brain isn’t able to tell the difference between real sensations and our ‘artificial’ sensations.”

George is now extending this technology into the commercial market, targeting applications in virtual and augmented reality. George notes that interactions with digital objects in virtual and augmented reality suffer from the same problems as prosthetic hands. “Without a sense of touch, it’s difficult to perform dexterous tasks like manipulating delicate objects or picking up objects without looking at them. But more importantly, without the ability to feel, interactions with objects are lifeless and you can no longer feel the comforting touch of a loved one.”

George’s team is now integrating the technology behind the “LUKE Arm” into a wristband to provide users with a more elegant form of touch in augmented reality. Their technology is unique in that it provides a sense of touch on the hands while keeping the hands free for daily tasks. George has a prototype of this technology in the form of a wristband, and one day envisions it being integrated into common smartwatches like the Apple Watch. For these contributions to science and technology, George was recognized as one of North America’s brightest young leaders and entrepreneurs and named one of Forbes “30 under 30” in 2022.

In another line of research, George is using the “LUKE Arm” technology to develop a similar wristband to not only add a sense of touch, but also to detect a user’s intent. This technology could enable individuals to control and interact with digital objects in virtual or augmented reality without the need for instrumented gloves or external cameras. Individuals could also use this technology to interact with smart-home devices simply with their thoughts. The idea of telepathically controlling devices has captured a lot of interest; George’s team has secured research funding to pursue the technology from the Veteran’s Affairs, the University of Utah PIVOT Center, and even Meta Reality Labs.

Says George, “we envision a world, an inclusive world, in which everyone can seamlessly interact with the technology around them, regardless of their physical capabilities.”

George also has ambitions to use the technology to restore mobility and dexterity to individuals who have experienced a stroke or spinal cord injury. George is working on bringing this approach to upper-limb exoskeletons in partnership with Myomo Inc., the lead manufacturer of upper-limb orthotic devices for stroke patients, and with support from the prestigious NIH Director’s Early Independence Award. George is the only individual at the University of Utah, and the entire Intermountain West region, to have received this prestigious award from the National Institutes of Health. 

Dr. George’s work at the University of Utah builds upon the success of many University of Utah faculty prior to him. The realization of Luke Skywalker’s bionic arm at the University of Utah dates back to as early as 1989, when bioengineering professor Kenneth Horch and colleagues developed one of the first technologies for directly interfacing with human peripheral nerves. George says the future of this technology is best summarized by prior research participant, Keven Walgamott, who lost his arm 17 years prior to the research study. After using the technology, Walgamott summarized his experience by stating, “Soon it will be just like Luke Skywalker, and then everyone will want one.” George’s team is now preparing for large-scale clinical trials to demonstrate the safety and efficacy of the prostheses in patients’ own homes.

Access downloadable images here.