• processing speed, storage, and bandwidth, resulting in almost all modern systems constituting sophisticated general-purpose networked workstations;
• the incorporation of richer media and more robust user interface experiences overall;
• the emergence of the suite of standards we know of the internet and all the tools for development for applications and content that have emerged from that;
• the convergence of other media (television, music); and,
• the ability to store and manipulate all this media on localized systems.

While all these improvements have converged to result in the computing environment we know and use today, the basic modes and methods through which people interact with all this new capacity has changed very little. We still have the WIMP (windows, icons, mouse, pointers) interface that came into existence in the late 1970’s and, it could be argued, that there really has not been a substantial breakthrough in the interface notion since. We are still saddled with the notion of sitting a user at a fixed system, fixing their gaze on a flat, two-dimensional screen, with some kind of user interface metaphor operating in the foreground and in the background, and then using a text input and pointer device to move and manipulate them. The interface places the user’s hands on a keyboard in front of them; along with the fixed forward gaze, it creates a very narrow physical and sensory path. Adding mobility does not change this model; in fact netbooks and other “smart” portable systems are even greater abusers of the human sensory system than are desktop systems.

While overall system capability has improved over time, with increased processing power leading to increasing inclusion of multimedia in the interface, there has not been an expansion in the fundamental capacity of these systems to enable higher levels of user interaction for addressing complex interactive informational demands. If you look at the current computing environment, it has reached its limit in terms of not being able to support a dynamic, agent-based processing environment. The current interface paradigm offers little in terms of progress.

Starting in the 1990’s, we were working on developing new models of computing that would support teleworkers. Specifically, we worked to develop systems that optimized the productivity of individual workers who were part of physically dispersed workgroups, using the communications and processing resources at hand to be functional.  We recognized that there were design points for serving people operating in these environments that were not being addressed; the assumptions of physical co-location and hierarchical command-and-control structures were still very much the norm. The assumption of scarce resources (processing, bandwidth, etc.) drove system design in directions that inhibited the development of systems that placed user needs and interaction requirements in the foreground. The focus was on optimizing access to the scarce computing resources at the cost of the presumably ample resource of user time and attention.

The active interface notion evolved from working to develop an effective platform for developing agents that would enable users to develop and use software code and processes for automating and streamlining routine events and interactions. Agents allow individuals to use the information workspace – all the possible communications and processing resources they found at hand to automate procedures and processes that would be running either in the foreground or the background. These agent procedures would go out and connect to reach other and bring back information in context.

Agents were a clear developmental focus at that point; supporting teleworkers required focusing on the notion that much of the support infrastructure around an individual worker could be modeled into software-based agents that would automate a lot of the auxiliary tasking and procedures that a knowledge worker deals with. HP New Wave was one of the first agent-oriented environments that provided tools for knowledge workers that were widely distributed, if not necessarily implemented. The concept was sound, but the primitiveness of the Windows 3.X user interface imposed far many too constraints; only capable programmers could effectively model and code agents in this environment.  It became clear that a new approach to user interface was necessary to drive this important notion.

When we look at this notion of developing agents, we can look back to examples such as HP’s NewWave that arose in the late 80’s/early 90’s and see the potential of an agent processing architecture that sets people up to operate using automated means, accumulating large numbers of streams in a global processing environment – this was before the web and before the capability of technology to support reach media affordably.  All these notions were hovering freely as unfulfilled requirements in the broad information environment. Prevailing interface notions that we had were plainly inadequate for enabling a full-featured agent-processing environment.

Through the 1990’s, we worked on developing the necessary structures and frameworks for enabling a global processing infrastructure. We developed fundamental notion during this time –poly serving poly servers – this design concept was developed to flesh out what we referred to as a peer-global infrastructure that fit emerging distributed collaborative work patterns. This notion fits hand-in-glove with the Polyopticon™ – it is the front-end device in this architecture –the forward facing, user navigation unit.

Current computing systems are still constrained to viewing users in this way: there is an individual conducting serial tasks, with multiple tasks in the environment existing as foreground, background, or deep background tasks. There is no interface mode or medium that we have encountered that enables users to hold the current state of all of his or her interactions and processing requirements in one place, and to develop interface notions that incorporate the temporality of the accomplishments of these tasks.  There certainly isn’t a device that allows users to shift views and contexts, play out different scenarios, and operate in a much freer, considerably less constrained mode than is imposed by the current computing paradigm.  The current interface model is still defined by an individual focusing down a narrow sensory and cognitive pathway. There has not been, until recently, the possibility of working in an expansive mode, taking advantage of the exquisite physical and sensory capabilities that humans are born with.

Bear in mind that humans are fundamentally hunter-gatherers. Our physical and sensory systems are optimized for finding and gathering resources and tending to them. Our physical orientation – an upright bi-ped – and the coordination of our visual and auditory systems, the allocation of fine and gross muscle skills – even our somatosensory system – all these are a single integrated unit optimized for adaptation and self-awareness in complex, shifting, uncertain environments. The human sensory system is oriented toward finding success by seizing upon suddenly apparent opportunities through a combination of strategic and tactical approaches, and doing so in a coordinated fashion. The challenge is to optimize those capacities – to recognize they exist and develop systems that let us take advantage of them, rather to constrain and narrow them, as the current paradigm does.

Fundamentally this device optimizes the hunter-gatherer capacity of human beings. It frees us from the shackles of the 2-D interfaces that legacy computing systems have imposed on us. It enables us to function as active sensing and coordinating agents in an information-dense environment, constructing modes and methods for functioning that are appropriate for adaptation to the environment.

In the system architecture we are illustrating, the user is an actor or agent coordinator operating inside a global processing sphere. This device provides the resources to enable the user to carry out actions and activities and use their mental processing capabilities – to use their sensory system – in ways that expand and improve upon the existing capabilities using the WIMP interface.

So it really is time to look outside the box, so to speak – to look outside legacy interface notions and to think about other approaches than can be taken. What dimensions are missing? If we look at it this way, it is clear that from a dimensional perspective, the current model fixes the user’s attention and their entire thought process into a rectangular screen – working all of this through in their own mind space, funneling perception down a very narrow experiential and sensory pathway.  It forces them to create a synthetic, internal 3-D world—perhaps a 2.5-D world—and places the entire cognitive overhead on the user to maintain the logic and relationships within this imaginary processing world. There is a limit to how much a person can process in such an imaginary world, yet none of the tools we have developed for processing information provide any real solution to this increased mental burden.

Most of the design patterns that define current computing systems have traditionally started by forcing the human sensory system – one of the most sublime inventions in the universe –down a very narrow tunnel, making sacrifices and compromises at every turn. One of the key elements at play in this design is the paradigm shift of moving from scare to abundant processing resource. The Polyopticon™ starts with an assumption of abundant resources – certainly abundant CPU processing, display processing, memory, bandwidth, and storage. This device is an attempt to leverage the emergence of abundant computing resources and to build a device for channeling deep cognitive patterns into a richer, more coherent whole.

• Orient multiple information windows in context
• Manipulate information in a 4-dimensional (3 physical dimensions plus time) format
• Build relationships among screens / applications / devices represented on the display

The patent-pending temporal-haptic interface device enables new modes of operation in command-and-control situations by modeling and recording physical cues and interactions with large data sets and virtual application environments on the back end.

Central to the Polyopticon’s™ functionality is the 4-D perception device. By implementing an internal gyro-positioning / accelerometer unit, and tying the output of this element to the device’s CPU, the Polyopticon™ tracks the user’s physical navigational movements in time and space. The device allows users to manipulate data in an individual window or panel, and “drag” or carry it to another.

The 4-D perception device allows for the creation of a new and novel set of interface metaphors based on selection, movement, storage, and playback of sessions with the device. In short, the device enables new kinds of interactions with dense, complex information sets served using virtual resources.

The Polyopticon™ surface is selectively touch-sensitive, enabling virtual input from a a “soft” keyboard – areas representing keys projected “virtually” on to the device’s surface. In the same way, a mouse or other 2-D tracking device can be implemented. The Polyopticon™ reduces both physical and mental stress by providing degrees of visual and kinetic freedom that traditional 2-dimensional interfaces cannot match.