As far as 99.9 percent of the world population is concerned, Microsoft is a stodgy, old-guard technology company. Its bottom line is fully leveraged against PC operating systems and business software—hardly the building blocks of a future-thinking portfolio, right?
But scratch that cold, conservative, pedestrian surface, and you’ll find a Microsoft that’s a veritable hotbed of cutting-edge innovation. Indeed, the company doesn’t just loosen its purse strings when it comes to research and development. No, it practically throws money at really big thinkers to build a more wondrous, fantastical future. In 2011 alone, Microsoft’s R&D budget reached a record high of $9.6 billion (yes, with a “B”). That’s a lot of Benjamins, and they’re being spent on some decidedly awesome projects.
Let’s take at some of the more interesting examples.
Blending touch and touchscreens
Carnegie MellonSeveral Microsoft Research projects have revolved into transforming everyday objects into fully interactive computing surfaces. If these initiatives bear fruit, you may one day conduct your morning Facebook check on the back of a cereal box rather than on your phone.
First up is LightSpace, which uses a plethora of cameras and projectors to create interactive displays on everyday objects. The system needs to be calibrated to the room it’s installed in, but once it is, users can interact with projected menus and screens using their hands, or even move a projected display from one object to another. Don’t feel like trying to crowd your team around a projection on a small desk? Drag it over to the wall, instead. You can see a basic version of LightSpace in action in this intriguing demo video.
The OmniTouch project—a joint project between Microsoft Research and the Human-Computer Interaction Institute at Carnegie-Mellon University—mounts a rig containing small pico projector and a Kinect-like 3D scanner on the user’s shoulder. The projector displays graphical images onto virtually any surface, while the 3D scanner’s depth-sensing capabilities transform the projection into an interactive, multi-touch-enabled input—and, thanks to some technical trickery, there’s no special calibration or training required. Check out the video below for a demonstration as well as a more technical explanation.
Meanwhile, the SkinPut project—another CMU and MSR joint venture—also uses a projector to beam interactive displays onto your hand and arm, but its secret sauce is a sensor-packed armband. When you touch the projected image on your skin, the sensors analyze the vibrations in your arm to pinpoint the location of your touch and respond accordingly—hence the name. Yes, SkinPut turns you into both a monitor and a mouse simultaneously. The website shows the technology in action.
Becoming More Kinected
You may know the Kinect as a gaming peripheral for the Xbox 360, but Microsoft hopes that it will become far, far more than that. The company continues to pour effort into researching ways inexpensive cameras can be used to interact with computers.
There are many examples. Kinect Fusion allows for continuous, real-time scanning of an environment in order to create interactive 3D models, and it’s coming to the Kinect for Windows SDK sometime soon. KinectTrack decouples the system’s IR emitter and camera to precisely track a user’s motion in multiple dimensions, mimicking the utility of expensive sensing systems with a $99 console accessory. SuperKid lets children create movies in real-time, complete with an array of interactive and customizable props. It doesn’t sound revolutionary, but check out the awesome video below.
Systems like Kinect are likely to drive new interfaces that rely on movement rather than touch. Users could check email while washing dishes or pause a video from across a room. Kinect started as a gaming peripheral, but it might one day turn the PC into an ever-present device that can be controlled from any room at any time.
And that’s not even touching on the robotics angle, where the Kinect has proven nothing short of revolutionary. The video below shows a robot playing catch, pseudo-juggling with multiple balls (and the help of a human with a second hand), and shaking its head in shame whenever it misses a catch. At the heart of this Disney Research creation? You guessed it—the Kinect.
Holodeck
The Xbox was Microsoft’s ticket into the living room. Now that it’s there, it has big plans—plans that may eventually transform your family space into a something like Star Trek’s famous holodeck.
Microsoft appears to be approaching the idea from at least two different directions. One method attempts to create a “magic wall” by combining a massive display powered by innovative “flat lens” LED technology with motion tracking and touch-input technology. The imagined result is an interface that can display detailed content and respond to a variety of touch and motion gestures.
MicrosoftMicrosoft’s current prototype “window” already supports glasses-free 3D by beaming specific stereoscopic images to each of your eyes, and it’s able to beam different images to different users. Basically, you could be immersed in one scene while your friend standing next to you stares at something else entirely. (And yes, it tracks your head motion.)
Another approach, recently unveiled in a patent application, uses 360-degree projection that could turn your living room into a virtual environment. The television remains the central point of reference and the projector is used fill in peripheral details. Motion-tracking is used to enhance the simulation and keep the projector from sending light towards the user’s eyes. The patent focuses on gaming, but it’s not hard to imagine the same technology be used for virtual tours of distant locations or movies that provides 360 degrees of immersion.
Neither path is likely to become a consumer product soon—Stevie Bathiche, Microsoft’s Director of Research in the Applied Sciences Group, wouldn’t even hazard a guess about the consumer availability of the first example of holodeck technology—but both are promising ideas. Microsoft’s combination of hardware and software expertise gives it a unique combination of knowledge that will be needed if virtual reality is to ever be practical and affordable for the average consumer.
Foveated Rendering
Computers will need a tremendous increase in capability to render graphics on the scale envisioned by Microsoft’s virtual reality research. The increase in resolution alone would increase the graphics power required by an order of magnitude. In other words, even if the other technologies required to create the company’s holodeck were perfected tomorrow, it would still be a dream. Modern home computers simply couldn’t handle it.
Microsoft ResearchMicrosoft’s working on a solution, and it’s based on the infirmities of the flesh. The human eye can only view a limited area in full detail. Our peripheral vision is much less sensitive. A computer with eye-tracking hardware—like, say, the holodeck mentioned above—can take advantage of this by determining where we’re focused and rendering objects in the periphery with less detail, using an antialiasing algorithm to smooth out the lower resolutions found off-center.
Microsoft calls this technique Foveated Rendering and has already conducted successful trials. Users couldn’t tell the difference between the normal image and the one with reduced detail. Yet the less detailed image required up to six times less power to render! “The result looks like a full-resolution image but reduces the number of pixels shaded by a factor of 10-15,” the research team notes.
This technology, if it came to consumers, would have broad implications. Game consoles would appear more realistic without quicker hardware. High-resolution displays would become more practical. And virtual reality would be much, much easier for a PC to handle.
Kinect Glasses and augmented reality
All of this technology may sound fantastical, but Microsoft-driven augmented reality may pop up sooner than you think. Earlier this year, a document containing information about the next Xbox leaked to the press. It was quickly mopped up by Redmond’s legal teams, but the document was available long enough to leak numerous details. Most of it was expected: The next Xbox will be more powerful, will offer a better version of Kinect, and will have even greater focus on digital distribution.
On tidbit came out of nowhere, however—Fortaleza, also referred to as Kinect Glasses. The leak showed artist renderings of people using augmented reality glasses in conjunction with the next Xbox to play games and navigate its operating system. The glasses would even be Wi-Fi and 4G capable, which suggests they might be usable without the game console.
Fortaleza is not Microsoft’s only experimentation with augmented reality and wearable peripherals. The company has demonstrated a wrist-mounted gadget, called Digits, which can translate a user’s hand movements directly into a virtual space. This concept lets users control a PC without direct interaction or a Kinect-like system—which would be useful if you’re on the go. Researchers are also looking at small-scale augmented reality with Kinect-derived technology that lets users manipulate projected objects.
Holodeck and its related virtual research projects are awesome, but augmented reality is a more likely near-term goal. Wearable computing will be hitting store shelves in the next few years—if not from Microsoft, then from Google—so there’s a real need for the company to invest in this future.
Cloud computing’s continued evolution
All of the projects we’ve touched on so far are real, concrete research that has produced real, concrete results in the lab. That kind of research isn’t cheap… which is why it’s surprising to learn that projects like these take up only 10% of Microsoft’s R&D budget. The rest the $9.6 billion dollar tab is being rung up by the cloud computing division.
Microsoft Research/CMUWindows 8 already takes steps toward the cloud. Users can sign in with their Windows Live account and access SkyDrive to store and share files. Office 2013 can use this storage to automatically sync files and settings between multiple PCs. This approach works both online and offline and is integrated in the Office interface.
The leaked Xbox document provided another example of how this research will let Microsoft offer new capabilities. It referred to the use of cloud computing to create a console with becomes more powerful over time. This would be accomplished by off-loading some compute tasks to remote servers. A feature like that, if it worked as advertised, would give the company a huge edge in the ongoing console wars.
It’s easy to imagine syncing a PC’s entire hard drive to the cloud, or apps that can run on multiple versions of Windows through cloud-powered virtualization, or cloud compute “upgrades” that let users temporarily speed up their PC. And if those uses can be imagined, well, it’s hard to say what Microsoft might be working on behind closed doors, especially when you watch the video above, which puts images to the company’s vision of a productive, connected future.
Microsoft: More innovative than you’d think
Some pundits speak about Microsoft as if it’s already dead—a shambling, stuffy company sustained by locked-in legacy software and little else. Microsoft certainly has challenges ahead of it, but as I’ve outlined today, the company does not have its head stuck in the sand.
Redmond spends more on R&D than Google and Apple combined. Think about that the next time someone tells you Microsoft doesn’t have a future.