If you want to understand the potential of 3D Touch, the new of method of tapping and pressing on the screens of the latest iPhones, forget about the marketing lingo. Don't think about Peeks or Pops or Quick Actions. Instead, think about reading—the kind you do with a textbook, highlighting text and scribbling in the margins. The kind of reading that you basically can't do on your phone.
"Compare [reading a book] to reading in the New York Times app. Are you scrolling?" asks Georg Petschnigg, CEO of FiftyThree, the company behind the excellent Paper app. "Are you flipping articles? Are you selecting text? It's actually really painful." We don't think about how dumb these processes are, but when you hear someone describe it out loud, it feels ridiculous. "If you want to select something you have to press, pause," he says. "Selection handles pop up, you have to drag them out, and then you have to wait, then you have to do an action, you have to wait for that stuff to appear. Then you move down."
All those steps will soon turn into this: Press extra-hard on the screen and swipe across the letters. Presto; highlighted. "Instead of being an entrenched action that's really full of friction," Petschnigg says, "it becomes something that's really intuitive."
Apple has already shown a couple of 3D Touch-based improvements here—you can now press extra-hard on your iPhone screen to define a word, for instance. But that, as with all of the tech inside 3D Touch, is just the very beginning. What's lost inside the flash and branding of Apple's new features is that your iPhone now has a pressure-sensitive display—and Apple's providing data about it to developers in real time. On the iPhone screens are incredibly sensitive, and incredibly responsive. ("It's very clean, very linear, very high resolution," Petschnigg says. "That's technical speak for, it's rock solid, it's totally accurate. You probably could build a scale using that stuff.") Developers are still trying to wrap their heads around what all that means, but when they do, it could turn 3D Touch from glorified right-click into really, truly, the biggest interface innovation since multitouch.
The key change, Petschnigg says, is that pressure can help you distinguish between selecting something and doing something to it. Until now, those have been the same—as soon as you tap the screen, the thing under your thumb snaps to your control. But if you separate selection from manipulation, you get much more powerful, much more natural control. Things move more naturally, with weight and inertia. You can move the same things different ways, and different things can happen. "Say, a building block," Petschnigg says. "Kids know that there's a difference between lifting up the silicon block, and pushing it." We lost that nuance with multitouch, and pressure touch can give it back.
It's all very heady and philosophical—Petschnigg apologized a few time during our conversation for having his head so far in the clouds. Developers are still figuring out what this all means. Petschnigg imagines you could use Peek and Pop to look through your notes faster, for one thing. And who knows what else? "We know basic selection, text selection is going to change," he says. "Object selection is going to change. We know on the tools side we gained an entirely new dimension of expressiveness." They're prototyping a lot of new ideas. "Diagram tool!" he proclaims at one point, like he just remembered it. "In our diagram tool, if you want to pick up a shape, duplicate a shape, stamp a shape, these all start to feel totally natural. "
There's one more example he's excited about: window management. As the world moves from mouse and keyboards to touchscreens, even for productive uses, how do we deal with having a dozen apps running at once? Right now, Petschnigg points out, the metaphor fails. "You know, you click on the window, it comes to the front. The same with ordering of shapes on the screen." When you want something else, you Alt-Tab, which no one does, or rely on some hacky workaround. "Now," he says, "you can push things back. You can't push a window back today. Now, all of a sudden, the street that used to be one way is now two way. Things will change."
Again, theoretical. Who knows how all this will shake out? Right now there's really one big upside for FiftyThree: the $49 Pencil stylus lots of people already own just became way better. The Paper app knows the shape of the Pencil's tip, and can read its changing geometry as you move it around; now, thanks to pressure touch, the app also knows how hard you're pressing on the screen. "With our sketch tool," Petschnigg says, "you'll be able to not just vary the width of a stroke, but the opacity, the lightness of it. Now it really feels like you're carving on the screen as if you're carving with a pencil."
Here's an easier example to understand. Magic Piano, Smule's popular ivory-tickling app, uses 3D Touch in the most obvious of ways: to figure out how hard you hit the keys. Smule CEO Jeff Smith says this changes everything. "What's happened with this new technology is we've moved from the harpsichord to the Steinway." Before, the only way to change the tone and feel of a piece of music was to play a note longer—now you can play it louder. Or softer.
Apple's data is rich enough that Magic Piano can measure the force from multiple fingers in real time, so you can pick out a single note in a chord to play a little more strongly (that's called "voicing"). Pressure touch has single-handedly turned the iPhone and iPad into "an instrument that can now be expressive in terms of dynamic—loud, soft, but also articulation," Smith says. "How notes are connected. For the first time, now, you can actually be quite expressive on the iPad and the iPhone, as you might be on a Steinway."
Most people, though, won't know this tech exists. And many more won't know what to do with it. So Magic Piano now comes with a new slider in the app—slide it to the left, and the app does all the crescendo and dynamism for you. But slide to the right, and you're entirely in control of whether "Somewhere Over the Rainbow" sounds stiff, lively, happy, sad, whatever you want. The slider solved a key problem, Smith says: "How do we find the balance of opening this up to people, but over time giving people the tools to literally perfect piano playing?" He suspects that most people who use Magic Piano will start with the slider to the left, but soon start moving it right and taking more control over the sound of their music.
After a week of using the new iPhones, I'm not blown away by what 3D Touch looks like now. Quick Actions are great, Peek and Pop are handy in spots, but none deserves the praise lavished upon them so far. So far. It's been a while since I've talked to developers so excited about the possibilities of a new feature, simultaneously trying to integrate it and wrap their head around how big the possibilities actually are.
3D Touch is going to make using your phone—with your finger, with a stylus, with the tip of your nose—more natural, more obvious. It will let you do things you've never been able to do before, and it'll let you do things in a way that actually makes sense. You'll swipe to move something, press hard to select it. You'll stop pinching—which, if you think about it, is a non-intuitive gesture—and start moving things with a single push. But it's going to take a while.
Think back to the first introduction of the iPhone, in 2007. "To unlock the phone," Jobs said, "I just take my finger and slide it across." The audience gasped. "Want to see it again?"
Petschnigg remembers that moment well. "There was a physical thing on the screen," he says. "You had to select the button and move it over." It was better than the existing ideas, sure, but what was that button? It moved too freely, you didn't know how it would move or how to make it stop. "By iOS 9, Petschnigg says, "you can actually use the entire screen to unlock. And that's right — you don't need to move a button around. The button was a holdover from old times. The entire screen can take the gesture."
His point: That took years to figure out. And it was just a lockscreen! But we're learning, slowly but surely, how the digital world should (and shouldn't) reflect the natural one. And with pressure touch, we have more tools than ever to help us do it. Eventually, they won't just come from Apple and that Huawei phone you can use to weigh an orange. Just as multitouch did, this kind of technology will be everywhere, fast. Everyone will have their own branding just as ridiculous as 3D Touch. But together, they'll reinvent the way we use our phones.
In the meantime, it's going to be a hell of a lot of fun. "We now have an opportunity for people to be even more expressive on these devices than ever before," the piano-app-maker Smith says, "and in fact to begin to perfect technique that's like in the real world." Then he corrects himself, sort of. "I'm not saying this is as good as a piano. If you really want a piano, go buy one. But this is pretty good. And it's getting way better!"
