Microsoft Research Demos Motion Sensing Keyboard with Gestures

Microsoft Research has demoed a new augmented mechanical keyboard that allows users to use motion controls either on or slightly above the keyboard to interact with their computers. The prototype was curiously made using keycaps from an Apple keyboard.

gesture

A low-resolution matrix of infrared (IR) proximity sensors is interspersed with the keys of a regular mechanical keyboard. This results in coarse but high frame-rate motion data. We extend a machine learning algorithm, traditionally used for static classification only, to robustly support dynamic, temporal gestures.

The gestures that the keyboard can recognize include traditional gestures like "pinch-to-zoom" and swiping to static gestures like holding a single finger above the keyboard to advanced gestures like turning an imaginary steering wheel.


It's unclear whether the augmented keyboard, referred to as the Type-Hover-Swipe, could eventually turn into a real product, but it's another step in Microsoft exploring motion control after its Kinect device. Apple, too, has been exploring motion sensing with its purchase of PrimeSense, the 3D body sensing firm that developed the technology for Kinect. The Type-Hover-Swipe keyboard is also similar to the technology of Leap Motion, who former Apple iAd chief Andy Miller worked for for close to two years.

Popular Stories

Aston Martin CarPlay Ultra Screen

Apple's CarPlay Ultra to Expand to These Vehicle Brands Later This Year

Sunday February 1, 2026 10:08 am PST by
Last year, Apple launched CarPlay Ultra, the long-awaited next-generation version of its CarPlay software system for vehicles. Nearly nine months later, CarPlay Ultra is still limited to Aston Martin's latest luxury vehicles, but that should change fairly soon. In May 2025, Apple said many other vehicle brands planned to offer CarPlay Ultra, including Hyundai, Kia, and Genesis. In his Powe...
Apple Logo Black

Apple's Next Launch is 'Imminent'

Sunday February 1, 2026 12:31 pm PST by
The calendar has turned to February, and a new report indicates that Apple's next product launch is "imminent," in the form of new MacBook Pro models. "All signs point to an imminent launch of next-generation MacBook Pros that retain the current form factor but deliver faster chips," Bloomberg's Mark Gurman said on Sunday. "I'm told the new models — code-named J714 and J716 — are slated...
iOS 26

iOS 26.3 and iOS 26.4 Will Add These New Features to Your iPhone

Tuesday February 3, 2026 7:47 am PST by
We are still waiting for the iOS 26.3 Release Candidate to come out, so the first iOS 26.4 beta is likely still at least a week or two away. Following beta testing, iOS 26.4 will likely be released to the general public in March or April. Below, we have recapped known or rumored iOS 26.3 and iOS 26.4 features so far. iOS 26.3 iPhone to Android Transfer Tool iOS 26.3 makes it easier...
Apple MacBook Pro M4 hero

New MacBook Pros Reportedly Launching Alongside macOS 26.3

Sunday February 1, 2026 5:42 am PST by
Apple is planning to launch new MacBook Pro models with M5 Pro and M5 Max chips alongside macOS 26.3, according to Bloomberg's Mark Gurman. "Apple's faster MacBook Pros are planned for the macOS 26.3 release cycle," wrote Gurman, in his Power On newsletter today. "I'm told the new models — code-named J714 and J716 — are slated for the macOS 26.3 software cycle, which runs from...
iOS 26 Home Feature

Apple Gives Final Warning to Home App Users

Tuesday February 3, 2026 8:55 am PST by
In 2022, Apple introduced a new Apple Home architecture that is "more reliable and efficient," and the deadline to upgrade and avoid issues is fast approaching. In an email this week, Apple gave customers a final reminder to upgrade their Home app by February 10, 2026. Apple says users who do not upgrade may experience issues with accessories and automations, or lose access to their smart...

Top Rated Comments

Gasu E. Avatar
154 months ago
This is how you know Microsoft still doesn't have it. Regardless of whatever this technology is, You don't go out and make a video showing off how innovative you are with some demo technology. You use the thing in an actual product to prove it's real worth.

Microsoft Research does research. Hence the word "research" in its name.
Score: 21 Votes (Like | Disagree)
2IS Avatar
154 months ago
The point a lot of people here are trying to make is that at least with the Apple way, it's a shipped product that you can go out now and purchase with your money.

The Microsoft way just exists in this video. Good luck actually using it any time soon.

Yeah? And when was the last time Apple did something remotely exciting? There's nothing wrong with showing off a concept. If there is a viable application for it and can be successfully marketed and sold, someone will run with it.

Thinner iPhone 6? How exciting is that? I can tell you, when I purchased my iPhone 5, I wasn't thinking "I'll bet the iPhone 6 will be fatter"

Someone's thinking outside the box, and it isn't :apple:

I personally don't have any interest in this keyboard, but as an enthusiast of new technology, I'm glad to see someone isn't afraid to do something new.
Score: 15 Votes (Like | Disagree)
2IS Avatar
154 months ago
This is how you know Microsoft still doesn't have it. Regardless of whatever this technology is, You don't go out and make a video showing off how innovative you are with some demo technology. You use the thing in an actual product to prove it's real worth.

I will guarantee this thing is a PURE gimmick. They're comfortable showing you a video of something they themselves don't know the value of.

Contrary to what you may think, the Apple way isn't the only way, nor is it necessarily the correct way.
Score: 14 Votes (Like | Disagree)
imageWIS Avatar
154 months ago
Interactive porn in 3, 2, 1...
Score: 13 Votes (Like | Disagree)
Gasu E. Avatar
154 months ago
I don't want thing because it's a thing yet! :mad:

Ain't no thing.

----------

agreed, maybe they should put the time and money into a developing a secure OS before they make gimmicky gesture keyboard.

Ah, the traditional MacRumors "false trade-off" fallacy.
Score: 12 Votes (Like | Disagree)
therealtruth Avatar
154 months ago
This is how you know Microsoft still doesn't have it. Regardless of whatever this technology is, You don't go out and make a video showing off how innovative you are with some demo technology. You use the thing in an actual product to prove it's real worth.

I will guarantee this thing is a PURE gimmick. They're comfortable showing you a video of something they themselves don't know the value of.
Score: 10 Votes (Like | Disagree)