Flashback to the Google Glass craze of 2013. People were vying to get an invite onto the “Glass Explorers” list in order to buy the prototype that would bring us one step closer to actualizing the scenes in the movie Minority Report of Tom Cruise interacting with a holographic computer projection in front of him.
Exelus is a French MedTech company that has been developing Nomadeec, a HoloLens telemedicine app, for the past three years that resembles the Minority Report system. Exelus’s telemedicine platform is one of the closest, most impressive demonstrations of a mixed-reality technology I have seen for medical/healthcare use. Having had the opportunity to view multiple Microsoft Hololens demos at various conferences, as well as to chat with Microsoft Hololens developers, it has been exciting to track the applications developed to date and to experience the Exelus’s Nomadeec app demo at this year’s CES conference in Las Vegas.
With the recent explosion of virtual reality (VR) and augmented reality (AR) in the tech world, these terms, as well as that of “mixed reality” (MR) can get a bit confusing, especially as the distinction between AR and MR can get a little fuzzy. The simple breakdown is as follows. Virtual reality immerses the user in a fully artificial computer-simulated environment, augmented reality overlays virtual objects onto the real-world environment, and mixed reality takes things one step further and not only overlays, but also anchors virtual objects to the real world.
The most advanced VR experiences provide freedom of movement, and users can use special hand-held controllers to enhance VR experiences (think Oculus Rift or HTC Vive with controllers). In AR, users can see and interact with the real world while digital content is superimposed onto the real world environment (think Pokémon Go). MR can start with either the real world or virtual world, wherein either a user remains in the real-world environment while digital content is added to it, or a user is fully immersed in the virtual environment while the real world is blocked out.
Nomadeec is an exciting Hololens mixed-reality application that not only showcases the potentials of MR in medicine and healthcare, but also a provides a proof-of-concept demonstration that these AR headset technologies can actually be used in practical real world field applications. Co-founded by an emergency medicine physician, Dr. Louis Rouxel, and engineer/UX expert, Xavier Maurin, Exelus has created a telemedicine platform that is an integrated, connected, and mobile solution for clinical examination performed by field practitioners. For the first time, EMTs and paramedics can seamlessly respond hands-free to incidents with the ability to diagnose and interact remotely using hand gestures and the onboard capabilities of the HoloLens hardware and software.
When you fire up the Nomadeec application after donning the HoloLens, a patient’s electronic health record (EHR) can be accessed. The EHR data is displayed in multiple thoughtfully categorized windows, as you would see on a computer screen. Using recognized hand gestures by the HoloLens, the user can then position and re-position these windows spatially in the real world around them (in the sky above, next to you, behind you, etc.) and toggle between records from different dates. Telemedicine involves virtually connecting you with a doctor miles away who diagnoses a patient and oversees treatment in real-time. Nomadeec uses the HoloLens to feed information to a remotely located doctor through video, and this doctor (also wearing a HoloLens) is then able to provide directions via video chat, voice, or holographic markers created that can be seen by both users.
To provide some context, let us imagine a paramedic out in the field dealing with an obscure animal bite that they have never seen before. Using Nomadeec, this paramedic can connect with a physician possessing the necessary expertise who can show the paramedic how to treat the wound by providing visual cues after taking a look at the patient in a real-time video feed. The paramedic can use voice or hand gestures to control the app’s interface to look at the real-time heart rate of the patient, medication allergies the patient has, blood type, and other potentially relevant information in the patient’s EHR. A holographic human body display can even be superimposed onto the patient, or displayed next to the patient that points out the critical body areas to examine. Videos of how to conduct the specific physical test or exam can be displayed with real-time feedback from the remote physician based on what is seen. This allows for fast, accurate on-site triage of a patient.
Technologically, the Microsoft HoloLens is a standalone headset with portable computing power. Nomadeec is compatible with a wide array of wireless diagnostic devices and can support 10 or more connected Bluetooth devices at a time, and features mobile connectivity (WiFi, 3G/4G, satellite) for tele-transmission and tele-consultation. As you can imagine, there are many use cases for this technology. The virtual projections look aesthetically pleasing, a vast range of customizable assessment modules to display information in the most impactful forms, and most importantly – Nomadeec actually works as a telemedicine platform that applies MR to patient management, and is not just a futuristic concept.
We had a chance to catch up with the two co-founders of Exelus: Chief Medical Director Dr. Louis Rouxel and Chief Executive Officer Xavier Maurin.
Alice Ferng, Medgadget: How was the idea for this telemedicine platform conceived? What was the inspiration behind creating this technology?
Xavier Maurin, Chief Executive Officer of Exelus: Our story began in 2013, when Dr. Louis Rouxel, the emergency physician, and myself, the UX design expert decided to start the project upon Louis’s needs and vision. Indeed, in his day-to-day practice, Louis failed from being able to share an integrated and secure view of the field scene with an remote expert if needed- especially in case of emergency or expertise- including patient medical data, circumstances, etc. He used his mobile phone to send pictures or videos, and some connected medical devices for vital parameters, but all that information was sent separately and unsecurely. So Louis came up with the idea of a mobile integrated platform, featuring on the field practitioner side, a mobile application used on tablet with connected medical devices, and on the remote doctor side, a secure web application allowing to get the whole patient information at a glance. Exelus, the company, was created late 2014 and it has been dedicated to research and development of the Nomadeec platform for 3 years.
Medgadget: Why was the Microsoft HoloLens headset chosen? Was there any technological advantage that made this ideal?
Mr. Maurin: Technically, the first and main benefit is the “hands-free” experience. That is to say, the ability to interact with the apps to fulfill the patient assessment, and communicate with the remote expert, even with blood on your gloves or in the sterile environment. This is a real revolution in outpatient assessment.This device should definitely replace tablets or smartphones in a short time, even inside the hospital.
Another very important thing is keeping the eye-contact with the patient through the surgical mask, and being able to communicate and reassure the patient in the meantime.
We believe that this kind of tool is making the difference to provide better and faster medical decision, and to improve patient care and patchway, especially in medical desert and critical area.
Clearly, Microsoft HoloLens is the only technology that is really mature today. Also, we are friends with the founders of Asobo Studio, a French company based in Bordeaux (like us), that was the first video game studio to develop on HoloLens from the early stage of the technology. They are among the best in the world, and we wanted to co-develop with their holographic division “Holoforge“ to get the “best of breed” in term of technology AND user experience. Microsoft has also been a very helpful and value-added partner. We are working with European and U.S. teams from both sides: the medical and HoloLens divisions, and they are all very supportive.
With Nomadeec, HoloLens acts as a heads-up display for monitoring a patient’s vital signs, as the platform can connect to numerous types of medical devices, such as ECGs, thermometers, stethoscopes, and ultrasound devices via Bluetooth. And when practitioners need a second opinion from a medical professional, the Nomadeec system can connect them to those medical experts via video conference.
Medgadget: How has adoption of Nomadeec among the medical and healthcare communities been going? Who have you partnered with to use this technology and validate it in the field?
Dr. Louis Rouxel, Chief Medical Director of Exelus: Our current emergency teleHealth platform (working on the tablet) is used by 25% of the emergency medical services (EMS) in France, which means around 500 field practitioners are able to communicate with around 100 EMS physicians. Our next generation platform featuring Microsoft HoloLens technology has been tested by EMTs, paramedics and Chief Medical Officers, though the commercial development has not started yet. We are currently running through the FDA approval process, which means that we will be able to sell our product by the end of this year. We are also in a consistent process of user-centric development, which means that we keep on collaborating with healthcare professionals permanently. As an example, Dr. Rafael Grossman, from the Massachusetts Institute of Technology is currently testing Nomadeec on HoloLens.
Medgadget: What are some of the most difficult challenges you have encountered so far with this kind of mixed-reality application?
Dr. Rouxel: The main challenge is to drive the change… as in any story of innovation. But here, since we are talking about a human’s life and medical care, the challenges are of course greater. Since our company’s DNA is clearly medical, we always pay a lot of attention to developing useful and meaningful innovation. Even if we are a small company at the moment, we have conducted medical studies with healthcare professionals to prove the benefit of our solution.
In 2018, some research labs are running bigger studies with Nomadeec, and we will be able to demonstrate the telemedicine impact this way.
Medgadget: Is Nomadeec currently limited by the biggest challenges in telemedicine related to internet connectivity? Does this limit field use of Nomadeec to locations with stable connections such as hospitals or homes?
Mr. Maurin: These are relevant concerns and we’re working with them in mind because one of our co-founders is an emergency physician who uses Nomadeec in his daily work. Nomadeec has been designed through a pure user-centric approach, in collaboration with emergency and prehospital practitioners. What matters is the benefit in use of the telemedicine. Our medical preliminary studies shows that the triage and the orientation of the patient’s care changed in 25% of the cases thanks to the use of Nomadeec.
The stake is to improve the field-hospital communication. We offer the ideal solution for seamless communication between on-duty staff (Paramedics, EMT, nurses, care assistants, night watchmen, etc.) and medical care coordination services in an primary care and distant-care context. The first outpatient assessment is proceed and teletransmit in less than 10 minutes in average ( no longer than current existing process) according to the feedbacks of our current users. We develop our applications considering various scenarios from the greatest performance (with full 4G connectivity) to no connectivity.
The field practitioner is not network dependent to fulfill the patient assessment since all medical devices are communicating via Bluetooth with our mobile app. We have optimized compression algorithms to allow data transmission via 3G or Edge only. 4G would be required if the assessment features numerous images or if a videoconferencing is required. But sharing a comprehensive assessment with a remote physician, including scores, pictures of the scene or prescriptions, ECG traces, etc. is already big progress and can change the patient evaluation and treatment, as we have demonstrated.
Talking about security, we have developed a rugged architecture based on a secured hub. Data are always encrypted (AES 256 bits) and transmitted according to secure protocols (SSL). NOMADEEC meets all security requirements health data transmission and storage. Video-conferencing implements standard and secure protocols. Also data, services, and web applications are available to health practitioners via a server certified for medical data hosting according to EU regulation and HIPAA-compliant for North America. This guarantees security, 24/7 access, and long-term storage.
All medical devices that are part of NOMADEEC are Bluetooth connected and made by world-class manufacturers to offer adapted, professional medical devices. They all meet CE and or FDA mark requirements (Class IIA).
Medgadget: Are there additional HoloLens applications that Exelus is working on?
Mr. Maurin: Not for the moment but we are working on further development to improve the user experience and care features on the current one… stay tuned :)
Medgadget: Are there plans to integrate other sensors and wearable devices?
Dr. Rouxel: Yes. Our platform has been designed with an open view, and we definitely seek the best and most meaningful (medically speaking) connected medical devices. Among our forthcoming projects, we are currently working on implementing ultrasound, multi-parameter monitors from the best professional manufacturers worldwide.
Medgadget: What are the next steps for the company and the Nomadeec platform in 2018 and the years to come?
Mr. Maurin: Thanks to CES 2018, we have set up exciting and amazing projects with international medical partners from the USA, Latin America, and Europe. Most of them are confidential, but we’ve clearly initiated our international development at the best level of medical innovation because we are the first to demonstrate a proven and secure mobile telemedicine platform using mixed-reality.
Our company has entered a new development cycle and we have initiated fundraising to concrete our growth.
