Microsoft patents mood-sensing tech for the HoloLens augmented-reality headset
The patent, granted to Microsoft the day before its Build conference kicks off, covers a "see-through, head-mounted display and sensing devices" that "detect audible and visual behaviors of a subject in a field of view of the device."
That, of course, sounds like the HoloLens, an augmented-reality headset Microsoft has shown reporters behind closed doors, and probably will bring to market, eventually. Virtual-reality devices like the Oculus Rift fit over the user's eyes, enclosing their field of view. But augmented-reality devices like Google Glass and the HoloLens superimpose a head-up display over the real-world objects that a user sees. One of the differences between Glass and the HoloLens is that the latter projects its images in color, and at a higher resolution than Glass.
Why this matters: It appears Microsoft believes the hardware must not only "see" but understand what it's seeing. Because people interact with each other so much, mood interpretation seems like a powerful tool. But will people see the tech as too intrusive and creepy, thus hampering widespread social use My guess is yes.
How it works
The version of the HoloLens that Microsoft has shown also includes a depth camera, so it can "scan" objects in front of the viewer. The same depth camera is part of the Microsoft Kinect, which can interpret data like a user's heart rate and skin temperature by closely observing his or her face. Microsoft's patent suggests scenarios in which the headset itself performs the interpretation, but also works in conjunction with a smartphone or other device.
Microsoft's patent offers a variation that interprets facial gestures (smiles, frowns, narrowed eyes) as indicators of a particular emotional state. Although the real-world examples Microsoft uses to illustrate the patent include a speaker gauging the emotional response of his audience, another stands out as somewhat creepy: "Other social situations in which the technology may be useful include romantic situations involving a one-on-one relationship between individuals."
Creep-factor aside, the technology could help those with Asperger's, autism, or other social disorders by interpreting emotional and social clues. Or, conceivably, the mood sensing could be used during sales calls or negotiations as a tool to see how far a client could be pushed before backing out. Finally, one might imagine surreptitiously scanning opponents at a poker table to figure out whether they're bluffing.
So far, Microsoft has said almost nothing about HoloLens, save for what it disclosed about the technology in January at a special event in Redmond. Journalists will have another opportunity to try out the technology at Build--but whether Microsoft uses the same closely guarded demos, we won't know until the show. Nevertheless, HoloLens looks amazing, and its powers apparently are growing by the day.