What are Natural User Interfaces?

A growing list of devices built with natural user interfaces (NUIs) accept input in the form of taps, swipes, and other ways of touching; hand and arm motions; body movement; and increasingly, natural language. Tablets and smartphones are the first in a growing array of devices that allow computers to recognize and interpret natural physical gestures as a means of control. These natural user interfaces allow users to engage in virtual activities with movements similar to what they would use in the real world, manipulating content intuitively. The idea of being able to have a completely natural interaction with your device is not new, but neither has its full potential been realized. What makes natural user interfaces especially interesting this year is the increasing high fidelity of systems that understand gestures, facial expressions, and their nuances, as well as the convergence of gesture-sensing technology with voice recognition. Users interact with their devices in an almost natural fashion, with gesture, expression, and voice communicating their intentions. The next wave of NUIs will likely be electrovibration, while involves the use of an electrostatic force to produce detailed tactile sensations that users can feel. Seen as the next evolution of touchscreen technology, it will allow users to not only provide touch-based input, but also tactile output via a wide variety of textures, topography, and other features as they interact with the screen.

INSTRUCTIONS: Enter your responses to the questions below. This is most easily done by moving your cursor to the end of the last item and pressing RETURN to create a new bullet point. Please include URLs whenever you can (full URLs will automatically be turned into hyperlinks; please type them out rather than using the linking tools in the toolbar).

Please "sign" your contributions by marking with the code of 4 tildes (~) in a row so that we can follow up with you if we need additional information or leads to examples- this produces a signature when the page is updated, like this: - Larry Larry Feb 8, 2012

(1) How might this technology be relevant to the educational sector you know best?

  • Its already happening as we read in the clippings from the previous stage - with users aged 1-101 already swiping their screens, anything less intuitive will simply feel too unattractive to even start with.- shazan shazan Oct 30, 2014
  • I'd like to add to this, because it's really not swiping that we're talking about here.http://www.digitalartsonline.co.uk/news/interactive-design/gesture-based-interfaces-are-out-of-control/ One potential for this technology is that it can be engaging for people who don't own or didn't bring their own smart devices - in fact even if you have your smartphone with you, it's sometimes a delight to do something bigger and more active. - dhegley dhegley Oct 31, 2014
  • The next generation of this is really exciting-- to be able to create more visceral experiences has so much potential in terms of adding layers to visitors' engagement. - jfoley jfoley Nov 5, 2014

(2) What themes are missing from the above description that you think are important?

  • Companies like http://ideum.com/interactive-exhibits/ have been doing this for many years, but now that mobile technologies are so integrated into our daily lives, these kinds of experiences will be replacing traditional PC's and mice. - shazan shazan Oct 30, 2014
  • Not sure about replacing - I think augmenting is more likely. One issue that is a big hurdle right now is that there is no agreed-upon standards for gestures. People know the basics of moving a mouse, clicking, right-clicking, etc. And they know about touch screens - swiping, enlarging, "clicking". But when they encounter a gesture-based interface it's often unique and unlike others. Swiping is usually not too tough (wave your arms across), but how do you select, click, go back, etc.? People often refer to the Tom Cruise scenes in Minority Report https://www.youtube.com/watch?v=7SFeCgoep1c, but there you can see the need for special gloves and some kind of "hold up your arms and look foolish" interaction design. The biggest hurdle for these interfaces will be in finding some way to normalize how they are used. For one example, simple human acts (walking, sitting v. standing, waving, nodding, etc.) could the the basis for tech that reacts accordingly. At least it would be a start. - dhegley dhegley Oct 31, 2014 - jfoley jfoley Nov 5, 2014
  • add your response here

(3) What do you see as the potential impact of this technology on education and interpretation in museums?

  • If you cant swipe an artwork in a gold frame then at least you can swipe the panel located next to it. - shazan shazan Oct 30, 2014
  • No swiping, no touching at all! The potential impact is huge for art museums, where deeper content could be accessed with no need for physical touch. At its simplest, imagine the automatic starting of a video (or even 3D video) in a Period Room upon a visitor's arrival, and the ability to control that experience by simple gestures or larger movement. - dhegley dhegley Oct 31, 2014 - jfoley jfoley Nov 5, 2014

(4) Do you have or know of a project working in this area?

Please share information about related projects in our Horizon Project Sharing Form.