Ned Notes

Beyond Microsoft BOOK: Chapter 5

BEYOND MICROSOFT: the next generation of software innovation is a prescient preview of emerging forces that are changing the entire revenue model for the global software industry. Written by a software insider who has worked as a leading business forecaster inside the industry for over a decade, Beyond Microsoft provides insight to today’s rapidly changing technology ecosystem and the cutthroat software industry.

Read the prologue and three complete chapters for free online.... and buy the book, forthcoming in 2010!

Chapter 5: The Invisible Interface

“[I]increasingly rapid technological changes are likely to significantly change user interfaces. We are at the dawn of an era where user interfaces are about to break out of the ‘desktop’ box…” – “Past, Present and Future of User Interface Software Tools” September 15, 1999 Brad Myers, Scott E. Hudson, and Randy Pausch / Carnegie Mellon University

The iPod Secret

Five years ago, I walked into a small Silicon Valley room to meet a tiny interface company called “Pixo.” I was there with a team of Adobe senior scientists to present our vision of the future. We hoped that Pixo might be able to help us achieve that dream.

Yet as our Adobe engineers described the coming wave of mobile devices and multimedia players, there were sideways glances from the Pixo people. Every now and then they’d gulp back something on the tips of their tongues. We were on the right track, but they were hiding something big. Finally, someone blurted: “Have you talked to Steve Jobs yet?”

We hadn’t yet talked to Steve Jobs or to any of his people at Apple. His office had said that he was too busy to meet with us. But the forecast we wanted to share was that a mobile entertainment device with an extremely simple interface would rapidly accelerate the mobile device market – and that very soon, some company would ship precisely this kind of market-shaking device.

A few weeks after our meeting, Apple shipped it. The device is now called the “iPod.” And the people at Pixo were responsible for the interface software.

What’s most interesting about this story is that you have probably never heard of Pixo. They’re an invisible company outside Silicon Valley. Their partner on the first version of the iPod interface was PortalPlayer, an equally invisible player in the iPod’s multi-billion dollar success.

In fact, almost all the companies that my Adobe team met with in 2001 and 2002 are now getting to that projected future of always-on Internet access, “smart-phone” systems and voice-activated applications. Yet most of them – even the incredibly successful ones – remain invisible. Their software products are almost invisible as well.

The world is rapidly accelerating past the need for behemoth software companies as “invisible interfaces” become pervasive. In this chapter, I’ll explain further why companies like Adobe and Microsoft are dead, and how “invisibility” is the future of software.

A Complex System, a Simple Result

The two most important parts of any software system are the user, and the function. If you are an accountant, you’re a user. If you want a spreadsheet that automatically calculates profit and loss, that’s a function. A writer is a user too. Writing words on a page is a function.

You’ll note that I left out of this equation any mention of the software layer – the interface itself. That’s because human beings typically operate in a way that assumes we aren’t using anything except our own organic bodies to do things. We like to act that way – it feels natural.

To make the point even more clearly, let’s look at Stephen King. He’s one of the world’s best-selling novelists. Over the years he has owned many, many computers. Reputedly, he knows Microsoft Word inside and out. Yet King recently said that a blank pad of paper and a fountain pen are the “best word processors ever made.”

I laughed when I first read this. Then I thought further, and I sobered up quick.

Stephen King’s statement is seriously funny because it has two different layers of meaning: 1) a fountain pen has nothing at all to do with the software-facilitated “word processing” as we understand it in the 21st century, yet it also reminds us that 2) the act of “processing” or creating words does not actually require software in any form.

Of course, publishers and distributors of books need huge enterprise-class computers to deliver the printed words that you are reading across the country to or whatever bricks-and-mortar bookstore you use to buy books. But human beings can create words, edit words, and understand words perfectly well without any intervening layer of computing software or hardware in between.

The problem is that in the last decade, all the complexity of a publisher’s enterprise-class databases and software layers has been pushed onto the end user – people like you and me. Today, in order to create simple words on a page, we often need to understand which buttons to push, which programs to launch, which protocols to use, which disk to save things to, and which network port will support our printer. Just to write a page of words, we often need to be functional Information Technology experts! For example, let’s look at the steps necessary to create the page of words you’re reading:

    1) I plugged in my computer and printer, pushed a button and waited for
    2) The Windows or Mac operating system (OS) to launch
    3) Then the OS instantiated various application programming interfaces, which allowed me to
    4) Click on another button, and launch a word-processing program.
    5) Type on a connected keyboard until all the words I wanted were on the screen.
    6) Push an icon (or keyboard shortcut) which told a printer to put ink on paper
    7) Deliver printed page to publisher, who then collated pages into this book.

You use a similarly complex system every time you compose a memo at work or a letter to your child’s teacher at home. Today, we realize that all the pieces have to be compatible, all the buttons have to be “clicked” in the right order, and all the software has to push signals to the hardware in the right sequence, or it won’t happen. If everything works right, you get a beautiful end product: black ink on a white page.

The trick is to make this beautiful end product easier to get. We don’t want to make the complex system go away -- but if we could make it invisible to people like you and me, the world would be a better place. No doubt, you’ve installed a printer or figured out how to use a new mouse on your computer when it didn’t “recognize the device.” You’ve also probably had your share of operating system snafus that you’ve had to de-bug yourself (or you’ve called in a real expert to do the dirty work), and you’ve probably had to re-install a piece of everyday software. I won’t even mention the time you downloaded that thing that made pop-up ads appear all over the screen and corrupted your e-mail program for six months. That one doesn’t need to be repeated.

A Simple Process, a Simple Result

Today the complexity of our tools gets in the way. In fact, the few pieces of software that don’t get in the way have used this “feature” as a selling point! “Papyrus” is a relatively new desktop publishing and word processing suite that competes with Microsoft Office. The marketing tagline for this new software is simply that it “Allows you to work on your documents ... without having to worry about your software.” Amazingly enough, not worrying about whether the software will get in the way of a user who wishes to perform a function is now unusual!

What if we got past the worrying? What if we didn’t even know – or care – that the software was there? What would a world look like in which complex computing operations were largely invisible?

In the back of Stephen King’s best-selling 2006 novel Cell, there is a page of words that constitutes the beginning of his next novel, Lisey’s Story. The words there are a precise reproduction of his original manuscript. The page even states that it is “in the author’s own hand,” and that hand reference is quite literal, for the words were created with his fountain pen: it’s handwritten. The fact that a published novel includes an entire handwritten section reminds one that King’s task to create that page was much simpler than our computer driven process. Here’s his list:

    1) Pick up pen, place tip on page
    2) Write words, moving paper along with his fingers, flipping pages as needed
    3) Deliver the final pages to the publisher, who then copies and collates into a book

Very simple. Three steps, and only the final one requires anyone to know anything about how a computer interface works. For King himself, there’s no knowledge or expertise (or downloaded software patches) required at all. In fact, there are hardly any moving parts – all those complex things that happened at the publisher to make his words appear in a book were “invisible” to Stephen King.

But that’s only the beginning. What if we made Stephen King’s process even simpler?

What if the words were delivered to the publisher as soon as they were written down? Even better would be to have someone speak, and words would simply appear on a piece of paper. Or what if we could just see Stephen King then speaking – in video, on a thin piece of paper. No computer screen or video camera visible. No printer needed. No special software to install. Just say it, and there are the words or the video of the speaker.

Even more impressive would be telepathy, where the writer would simply have to think about my ideas and pages and pages of written documentation would immediately appear. Think of a word and it appears in print. Now with all of these ideas – instant written words, instant video, and telepathy – we seem to be moving into the realm of science-fiction, or as Arthur C. Clarke tells us “any sufficiently advanced science looks like magic to those who are seeing it for the first time.” Magic.

What’s interesting about my proposals in the above paragraph is that I bet you immediately said to yourself – “Yeah, that would be pretty cool.” And I’ll bet further that the next time you sit down at your computer to write an e-mail or a memo, you will think of how nice it would be if you could simply think or speak that memo, and have a printed or video copy of your work appear, as if by magic.

Now I’ll tell you the reason why you immediately think that technology capable of telepathy or automatic printing would be cool.

Romance and Computers

When the interface is invisible, then we fall in love with our machines. The closer an interface gets us to an actual function we wish to perform – the more the interface becomes invisible to our use of it – the more natural it feels, and the more we find whatever technology allowed that experience to happen to be irresistible. We love it!

This romance is precisely what happened with the iPod. Put ear-buds to your ears, touch a wheel to spin or play your music. Out come the tunes. No drivers needed, and you never know who Pixo is, or how ParallelPlayer recognizes each of your music tracks and places them in the right play list. No need for that. Touch a button – hear the tunes.

Romance is also precisely what happened with Google. No need for complex search fields or Boolean restrictions on your search. Nothing will get in the way of the answers you’re seeking (the function of a search engine) is right there on Google’s opening page: white space, a company name, a search field. Nothing else. No advertising, no “toggle this,” or “push this button to pay for extra-special results” or even “search these categories.” Just enter a word, a phrase, a sentence, and up pop results that are pretty darn close to what you were looking for. User gets results. Fast.

Google understood the power of invisibility. That’s why we still love Google.

Since the beginning of the company, Google’s corporate motto has been “Don’t be Evil” – which basically means ‘don’t do things that generate revenue, but don’t serve your users.’ As we’ve read in previous chapters, Google founders Larry Page and Sergey Brin wanted to distinguish themselves from predatory software companies like Microsoft and Oracle. For Google, this mantra has resulted in products like the simple front-page of their web site, and easy-to-use search results. For years, Google’s motto could well have been “Don’t be too Visible” or “Don’t Get in the Way.”

It is important to note that Google’s success is the result of a mind-boggling amount of very complex mathematical algorithims. Today’s Google keeps functioning and growing because of the machinations of a team of thousands of highly skilled computer scientists. Yet all the complexity of the mathematical algorithms that make systems like Google work are hidden from consumers like you and me.

The reason that Google and the iPod are so loved by millions is that we don’t see any of complexity. We go to the iPod for music. We go to Google to search. Nothing gets in our way of experiencing these two “functions.” In both instances, the entire “interface” disappears in the face of functions that matter. The moral to the story is that good software can hardly be seen or acknowledged: in the coming world, all software will function on an almost subliminal level.

The romance of a good piece of software is its essential invisibility while you’re just accomplishing your goal. In fact, the most successful software products have always succeeded with this strategy.

Love Stories: from Palm to the Wii

In the 1990s, people fell in love with small pocket-sized devices. Computers were appearing on every desktop, and the calendaring and e-mail information on them was becoming increasingly crucial to business users. People needed computing power to do their jobs – they needed their schedules, their e-mail and their notes. Unfortunately, the gargantuan PC towers and oversized laptops of the era didn’t travel easily. Enter Palm and Blackberry.

In terms of computer software and hardware, Palm devices didn’t have much of an interface: gray text on a tiny greenish screen. Even by 2002, Blackberry had even fewer bells and whistles. But what both Palm and Blackberry did was to bring some simple functions closer to the users. Palm ruled the mobile device market in the 1990s because they allowed users to easily get calendar and address book information, anytime, anywhere. Blackberry did the same thing with e-mail. Users got functions. Although there had been plenty of mobile devices before Palm and Blackberry, this time the interface didn’t get in the way of doing something. Even though most mobile phones today are more functional than these early devices, many people still have an obsessional love affair with their “Crackberries.”

In 2007, Sony is hoping to kindle the same sort of romance with a new reading device that precisely mimics the natural reading experience. Older “e-book reading” devices inserted a layer of obtrusive software into the reading experience – anyone remember the “Microsoft Reader for Windows 2000, Special edition with downloadable patches for new high-featured secure e-books”? Doesn’t that mouthful of words sound like the opposite of a simple reading experience? Fortunately, the new Sony Reader isn’t about patches or Windows or special secure features.

It’s just about reading.

Sony has created the first of a new generation of reading devices that use “e-ink” – the Reader’s text looks identical to text in a printed book. Instead of making a book look like a computer, they’ve made a computer act precisely like a book – no interface required. The technology precisely mimics paper by turning pixel-sized balls of “ink” into new letter shapes at the flick of a page – and by so doing, the technology layer itself becomes “invisible.”

The reading is natural. Users get their desired function – and more, with 1000 easily readable books in one pocket, instead of just a single paperback. It’s natural to fall in love with this kind of transparent technology.

Making fun into a natural function is also working for Nintendo in 2007. The romance with Nintendo’s new gaming system, the Wii, has already surmounted all expectations. Most game consoles make you mash Button A, and Button B, and then push down Control C and hope the tennis ball on the screen reacts the right way.

With the Wii, you just swing your wrist like you were holding a tennis racket. No need for Buttons, Controls or mashing them in sequence. The complex accelerometers and infrared software that make the Wii function are invisible. Hold the Wii Remote like a racket, swing your arm, and bang goes the ball across the court.

Now keep in mind that the Wii doesn’t include Windows – or any highly profitable bells and whistles operating system – but very few people have gotten hung up on the Wii’s largely invisible interface. It just feels natural.

Already, Nintendo has been ringing up the love to the tune of $1.5 billion in Wii sales in the first quarter after the console’s release. Nintendo swung hard, and hit the ball!

A Next Generation Preview

A few pages back, we were talking about science-fiction. What if Stephen King could write and the words from his pen would automatically be published? What if he could talk and words or video would simply appear on a page? Magic. Fantasies. But a funny thing happened on the way to fantasyland: the fairy tales turned real.

Decades ago, software engineers began to think about voice-interaction. Over the last ten years, their work has moved from clunky “trainable” verbal processing units with Arnold Schwarzenegger-like accents to beautifully articulate (and sexy) natural language software. Today, any clunkiness has disappeared from voice-operated computers, and no one has to “train” a computer system to understand them anymore. It’s all automatic. Next generation voice technology has taken a quantum leap even since 2004, and today people hardly realize how many computer-driven voices they’re already hearing.

We’ll learn more about the “natural language voice interface” after we preview a few other “invisible” interfaces that will create a whole new magic in the years to come.

The Sony Reader was just the first step towards computer reading that looks and feels absolutely natural. A dynamic new company has already created an e-ink “screen” that looks like paper and can be flexed like a pliable magazine page. Talk about an invisible interface.

The next step could be if the interface is all around you. What if you work in a computer-generated 3D environment, so that every action is computer assisted?

The sound of derisive laughter can be heard as we get to this idea. This is because immersive 3D environments are the great wet-dream of every computer scientist. Who doesn’t love The Matrix and the Star Trek holo-deck? Science-fiction. Magic. Yet the laughter might be checked if we begin to remember that immersive 3D environments are already a multi-billion dollar industry: any neighborhood gamer could tell you about WarCraft, EverQuest or other Massively Multiplayer Online Games.

Games like these include an interface that obeys our laws of physics (unlike your Microsoft Windows or Mac desktop). WarCraft is fun because it feels natural. Eventually, we may transition work itself into 3D environments: our office could be located in an immersive 3D environment called WorkCraft or WriteCraft or ArtCraft. What’s interesting is that several companies have already built working prototypes for a functional 3D “WorkCraft” type space. Several large high-tech companies have workers already experimenting with working full-time in a 3D office. It’s only a matter of time.

But what if you don’t want to go “into” the office, even if it’s a virtual office? What if you could take it with you? What about a normal-looking pen that sent all your scribbles to your computer, as editable text? What about a jacket that pulses to tell you about upcoming appointments or cell phone calls, refrigerators that tell you when to buy beer, a checkbook that can tell you how much is left in your account, office desks that can tell you where you left that file you need, and cars that give you directions?

Oh – I should tell you that last one became old hat in 1999, with the advent of OnStar and other in-car navigation systems. In the 1980s, when roadwise voice-driven navigation computers were first proposed, people laughed. It’ll never happen, they said.

Surprisingly, naysayers continue to argue that computer interaction will never really be popular outside the office environment. Such skepticism is unwarranted: all of the scenarios I’ve described are in products that are already shipping: the car was simply the first object in which the engineers put a chip. A pen, a desk, a refrigerator, a checkbook and a jacket are not that much more difficult in which to put embedded computer technology.

Today, complex circuitry and chipsets have become so miniaturized and flexible they can fit inside almost any article of clothing or tiny piece of hardware. In Asia, there are numerous proof points; in the U.S. Apple’s “iPod Shuffle” music player is the most pervasive example – the Shuffle is about an inch big, and it just clips to your clothing. With new forms of magnetized “flash memory” that connect with these tiny devices, even huge amounts of digital memory are now instantly available to us: in 2007, Toshiba announced it was shipping a one terabyte “personal hard drive” about the size of your old-school 3.5 inch “floppy” disk. Only a few years ago, a one gigabyte hard drive was considered a huge accomplishment (a gigabyte is 1/1,000 the size of a terabyte). We’re soon going to have powerful computers in everything from pens to watches to tiny (yet immensely powerful) terabyte filing systems that look like paperclips and paperweights.

The trick is that without a screen or a keyboard – how do you tell these embedded micro-computers what music to play, what cell phone calls to accept, which beer to buy, and what file you’re looking for under the mess on your desk? If the iPod Shuffle gets any smaller, it won’t have room for that handy scroll wheel and button presses. If video-cell-phones shrink to the size of a pinkie finger, only the fingers of 1st graders will be able to punch the buttons and make calls. Human fingers and eyes aren’t going to miniaturize any time soon, no matter how cool the device. So how to interact with tiny computers without plugging in a clunky keyboard or hooking up a big menu-driven screen?

The answer can be found in the “interface” and “interaction” tools we already have built into our bodies and our social communication.

Vocal Gymnastics

In 2005, a character actor named James Montgomery Doohan died in Redmond Washington. A few decades before he died, I’d met Doohan at his house. He wore peculiar futuristic sideburns because he was about to do a publicity appearance for the movie “The Voyage Home: Star Trek IV.” After all, Doohan was famous as “Engineer Scotty” in the inimitable “Star Trek” TV and film series. Every time I think of Scotty, I remember a hilarious scene of person-to-computer miscommunication in Star Trek IV.

In the scene, Scotty and his Star Trek crew have been transported from the future back to 1986. Scotty sits down at a 1980s computer so that he can re-create a 23rd century compound, and save the world. Scotty says: “Computer.” There is no response.

“Computer?” he says, picking up the mouse and trying to talk into it like a microphone. “Hello, computer?” A 1980s scientist tells him to use the keyboard. Scotty looks down at the desk, and says, with some disdain: “A keyboard. How quaint.”

The keyboard is indeed rapidly becoming quaint. In fact, most of the hardware that we currently use to do our work on computers is really made from a series of “hacks” – jerry-rigged devices to do artificially what we can’t yet do naturally. Things like drop-down menus, digital ‘desktop’ operating systems and registry files are all a vestige of someone’s hack. Today, most computers are not yet equipped to respond automatically to our voice commands: we still use quaint things like keyboards as input devices. Yet perfectly natural interfaces – like voice and gesture – is coming closer every day.

After all, few people even think about a good voice-activated system as a computer “interface.” That’s because the interface menu – the levels that you can choose between – are not imposed on the user as a rigid set of options.

Instead your interactions seem to happen naturally, just as in a conversation with a friend: “Are you looking to see a movie tonight? Or should we order a pizza?”

“Let’s get a pizza.”

“Okay – look, the last three times you got one, it was Canadian Bacon. Do you want the same? Or something different?”

“Nah, go for it – same as last time. Supersize the root beer. Thanks.”

“You’re welcome,” says the sultry voice. And twenty minutes later, there’s your pizza and Supersized root beer at the door.

Now Donna the voice-system won’t be sharing that pizza with you. But without even thinking about it, we are increasingly slipping into these kind of naturalistic conversations with computers.

It bears mentioning that a voice-driven computer system even improves on a real pizza gal – after all, you can interrupt them in a way that would be rude in a human conversation. In fact, one of my colleagues used voice systems so much that he began to interrupt his secretary in the same blunt manner, until she caught him one day, and told him that she wouldn’t be ordering any more pizzas for him until he treated her like a person, instead of a quick-n-dirty ordering service.

When was the last time you mistook your secretary for a Windows PC?

In 2006, voice recognition vendor Nuance Communications announced that they were releasing new software that would allow anyone – on almost any computer device, from a PDA to a cell phone to a laptop – to interact with their system using just their everyday voice, with 99% accuracy. That’s better than a lot of conversations I have with my secretary.

Nuance’s work has been tested in the field – today, most customer service call centers use a great deal of voice recognition and response software, and many customers don’t even know it. According to a recent study, 85% of customers found the voice software “satisfactory” or “excellent” – 65% of these same customers preferred speech interaction with a computer to touchtone responses.

Just like Engineer Scotty, customers want to use voice – not a quaint keyboard.

What’s really interesting is that when customer service call centers start using today’s cutting-edge speech software, 62% of them see an increase in customer satisfaction. It seems that customers even prefer the voice software to a real pizza gal.

Invisible Interactions

A modern voice interface feels so natural that it doesn’t seem to exist at all. Yet voice interaction is merely one instance of the coming era of the invisible interface.

For thousands of years, human beings have enjoyed communicating information through writing and reading dark ink on white parchment. We’ve evolved as readers and writers, and the closer that technology brings us to our innate aptitudes, the more invisibile it will become.

What Sony introduced with their “e-ink” Reader in 2006 was just the beginning of a whole new wave of naturalistic “invisible interface” computers that mimic the way we read, think, and write. In early 2007, a small pen came out that instantly and quietly recorded every pen stroke you made – and then instantly downloaded it to your desktop computer as editable text. Your brilliant late-night scribblings on a bar napkin can now be instantaneously e-mailed around the world.

Yes, it may be invisible technology, but I didn’t say it was idiot-proof.

Later in 2007, two companies – Polymer Vision and Plastic Logic – will introduce computer-based reading devices with beautiful white pages that simply roll up like a papyrus scroll, or fold in your hand like a magazine. Furthermore, this flexible interface will soon be able to display multimedia, and will ship on newer cell phones. When in use, the display will open out from the phone to a useful size – like a book opening. When not in use, the screen rolls up, simply disappearing back into the casing, allowing you to slip a full-functioning computer and screen in your pocket. Now you see it, now you don’t.

In point of fact, today we often don’t “see” a lot of the information that is available to us. Quick – where is the McGregor file? And where exactly did you park your car at the mall? Which of your kids borrowed your cell phone? Is your UPS package sitting at your doorstep, or is it still en route from Chicago? All of this information is easily accessible via a new series of cheap and readily available Radio Frequency Identification (RFID) tagging chips that are already being embedded into every item from books to CDs to packages to cell phones to toys.

In 2007 and 2007, these chips will beam out the basic RFID information like name, date, contents, even ownership data – plus it will now beam out a Global Positioning System (GPS) location. In the next few years, almost everything down to individual file folders and children’s toys will contain an invisible little RFID/GPS chip. At that point, any item you own (with a tiny embedded RFID locator) will be able to be found with pin-point accuracy anywhere in the world, and accurately identified separately from other similar items. With the right Google Earth search, you could see the precise location of every “Tickle Me Elmo” doll in the world, or every copy of this book.

Majority Movement

The world gets really interesting when people start embedding themselves – or even each individual arm and leg – with these nearly invisible RFID chips. Remember the Nintendo Wii gaming system and its naturalistic tennis games? In a world where your arm has a tiny locator just under the skin, it won’t even be necessary anymore to hold a remote or a “game-controller.” Your own body is the game controller – with “gesture technology” you become the interface to the system. An example of this kind of technology at work can be seen in Stephen Spielberg’s film Minority Report. In a near-future world of 2054, Tom Cruise dons special gloves to move information around on a massive computer screen simply by sweeping his hands back and forth.

The possibility is a lot closer than 2054, and it won’t require any gloves. RFID embedding is already being done in different places around the world. As this accelerates, gestures and voice are going to be our primary “interface” with computers by 2014. With the advent of RFID/GPS and micro-computer embedding, there will soon be the equivalent of hundreds of computers all around us in almost every object. And the majority of us will be controlling our computers with movement.

The financial and technical implications of being able to locate important items – like your customer – and move other important items – like your products – accurately and instantly, with just a sweep of the hand and perhaps a few voice instructions (“Make that package urgent – and have Larry Page sign for it!”), will be found in the next decade, and are covered at some depth in the third section of this book.

The near term implications of “invisible interfaces” are explained further in Chapter 6: “Software as Utility,” which explains how to add value (and gain that slim margin of profit) in a world where computer functions have become a commodity as readily available as water or air.

Creative Commons License
This work by Ned Hayes is licensed under a Creative Commons
Attribution-Noncommercial-No Derivative Works 3.0 United States License


Reaching Me: Ned Hayes · Seattle WA · 206.321.7981 · ned AT