- Written by: Steven D. Domenikos
Disruptive innovation enables powerful human-computer interactions through the natural sense of touch. The whole of human experience is largely based on us making contact with physical objects in the real world.
In this article we will provide an updated definition of the next generation internet, the Tactile Internet. Numerous innovations in the information and communication technologies have enabled exponential growth in network capacity, leading to the emergence of smartphones and a rich user experience. The Internet of Things (IoT) connects devices, or objects, to increase their efficiency by exploiting the potential of networking. These different embodiments of the Internet will be dwarfed by the emergence of the Tactile Internet, in which novel haptic software platforms, ultra-responsive and ultra-reliable network connectivity will enable it to deliver physical haptic experiences across its users. Its ambition is to enable a democratization of skill, and how it could be delivered globally. It will have a marked impact on business and society, introducing numerous new opportunities for emerging markets and the delivery of essential public services.
We will explore the critical technologies to make the Tactile Internet a reality and will identify the paradigm shift the needs to take place to enable broad-based adoption of haptic technologies that can be convincing enough to make it worthwhile.
Touch is the most powerful of all our natural senses; the first sense to develop in the womb; the last sense to go. Like our other senses, touch comes in gradations. It is governed by an exquisite array of receptors that can distinguish minute variations in the external environment. Fast, slow, or in between? Hard, soft, or something else? Hot, cold, warm? Some receptors react only to caresses. Some send pain signals. Some tell us that we have an itch. Each type activates a different part of the brain, making us feel soothed or hurt, comfortable or distressed, angry or calm. In his recent book “Touch: The Science of Hand, Heart, and Mind,” the Johns Hopkins University neuroscientist David Linden cites “the electric touch of romantic love, the unsettling feeling of being watched, the relief of pain from mindful practice, or the essential touch that newborns need to thrive.” All of these diverse sensations, he writes, “flow from the evolved nature of our skin, nerves, and brain”.
Even so, our human-computer interfaces today mostly rely on our auditory and visual senses with touch having little or no participation. There are three main reasons for this: The haptic interfaces, that comprise from the electromechanical systems that mediate our interactions with a computer environment are not well developed; the richness of that mediation is not convincing enough and limits the resulting experience. Secondly, there are no yet consistent development standards using haptic technologies to develop touch-enabled applications. Lastly, perhaps the most important reason is that we have been so conditioned to rely on our auditory and visual senses for our human-computer interactions, and therefore acquiescing that nothing more is needed.
Our senses allow us to perceive our environment and decide whether to adapt ourselves to that it or modify it. Our perceptual processes limit the speed of our interaction with our environment. We experience interaction with a technical system as intuitive and natural only if the feedback of the system is adapted to our human reaction time. Consequently, the requirements for technical systems enabling real-time interactions depend on the participating human senses.
Figure 2: The paradigm shift
The Theme™ Architecture
We propose an architecture that will address the needs for transmitting and receiving the necessary tactile and kinesthetic information sets so as to render a convincing sensation among the participants. We name this architecture The Haptic Engine for Mobile Environments, Theme™ in short.
Figure 3: The Theme™ architecture
Figure 3 shows the flow of information being exchanged during a typical haptic-enabled session between a human and a machine, a robot in this case. The architecture identifies the data types that must be transported between the two network nodes.
To achieve its task, the human user sends out interaction control commands to the robot to move, apply pressure, close its hands or any other modification of its physical state. The human can also send a video and / or an audio stream if the robot is interacting with a human being, such as in a medical situation.
In return, the robot sends position and force feedback to the human to confirm the instruction was achieved, as well as audio and video feedback for the human to take the next action.
To augment the experience, the robot streams haptic information identified as the haptic track or haptic channel, about the objects it is manipulating, such as force feedback, texture, softness, roughness, smoothness, temperature and possibly other modalities. This data set, which corresponds to force and acceleration vectors is obtained as the robot is interacting with its environments through sensors and actuators. The data set is then encoded to a format that is convenient for transport, transmitted using an appropriate haptic protocol, then decoded, processed and rendered on the human side. For audio, video, and control the appropriate protocols, formats and codex exist. The fundamental haptic protocols are already in place and continue to evolve to accommodate the transport of richer haptic data sets.
The promise for a new dimension of digital human expression and online interaction is taking shape through new advances in haptic technologies, networking, and ICs. This new dimension is the sense of natural touch to augment like never before the visual and auditory experiences of today’s internet.
High fidelity tactile experiences will be first manifested for nearly universal use cases in our smartphones and touch screens through enhanced UI controls such as buttons, keyboards dials, and sliders where they would recreate the tactile feeling of the mechanical equivalents. Then, touch-enriched peer to peer communication will inextricably be part of our daily interaction. Image a gentle tap or a nuanced pulse to accompany a message from a loved one. Following that, we see touch-enabled remote-control applications with haptic feedback to find their way into our daily lives. The idea of touching and feeling what someone else is touching and feeling across the world in real time brings exciting possibilities and sets the stage for yet unimaginable opportunities. Ultimately, touch-enhanced content would be the expected norm. Consider photos, images and movies that will incorporate the tactile track as a way to a fully immersive, almost addictive experience. Can you then imagine an online world without touch?