Introduction:
The scenario I am about to propose is not about a specific device or application within an Interactive Workspace. Rather, the scenario describes a pattern of interaction with the applications and devices within a room. I'm interested in enabling a user's freedom to use the interactive room as he/she sees fit--as opposed to how an application or device designer believed the room should be used.
I certainly don't mean that application and device designers are trying to maliciously restrict users. But these designers are only human. There will be (there are even now!) a multitude of workspace configurations. Some rooms will have many large displays, some may have none. Some will have 4.1 channel THX-certified sound systems, and others will not. There will be Phantom devices, 3d displays, augmented physicons, small devices, big devices, temperature sensors, motion detectors, video cameras and still cameras. And to complicate the IW landscape even more, different users will compose and use these devices in different ways.
Application and device designers can not be expected to project how users will compose these devices together into a single Interactive Workspace. They should not have to individually design their systems to be prepared for tomorrow's new technologies. But we still want their systems to work as if they had.
The Several Sections:
1. The Physical Space and Activities Within It
As described above, the physical space I'm describing has potentially any configuration. It might be inside or outside, a personal room, or a corporate conference room. It might be a specific-purpose workspace, designed for holographers, or it might be a general-purpose workspace. But regardless of the activity for which the space was designed, the space should allow as many sorts of activities within it as possible. If it is physically possible to do some activity (even if it's not optimal), neither the application nor the workspace should limit the user. The only limitation should be the physical capabilities of the devices in the space.
For the remainder of this page, I'm going to concentrate on one specific activity: a stock trading application (that seems popular these days). The ideal room for this application would have many displays, each showing some ticker data, or graphs of trading volume, price, etc. A couple displays would show news headlines and CNN. Within the room would be one or more day traders, each person would execute private trades on his or her own PDA or laptop. If someone was alone in the room, or didn't care about privacy (e.g., a brokerage firm where the traders are "on the same side"), the traders could show summaries of portfolios on screen, hear reminders over the speakers, etc. This is the ideal room for stock trading.
What's special about this? The application of stock trading isn't very interesting (unless your the person whose money is on the line, then you might be interested). A room with lots of displays doesn't sound terribly interesting. But what if we decouple the application from the displays by putting a big layer of abstraction in the middle, so that the application no longer knows what it's displaying to? The application becomes a source of data, while the routing and formatting of the data for a display is determined through user preferences and application hints.
2. The Components
Once we've decoupled the application from the devices in the room, the user is free to add and change devices, reroute data to different displays, and the room's infrastructure (the abstraction layer we added) handles reformatting data: transforming a table of numbers into a graph for display on the wall; turning a VRML 3D model into OpenGL commands for the Mural; or synthesizing speech from text. What's in the infrastructure to facilitate these transformations?
The components that are needed are:
Much of this software is already available from various sources, and should be able to be stitched together to develop the sort of Interactive Workspace where the user is in control of how the workspace is utilized. Jini provides device discovery services, as does the Service Discovery Service (ninja.cs.berkeley.edu). Most data transformers one would want are already available as standalone programs. I myself wrote a service to compose data transformers together last year as part of another project. Combining these existing services together into one architecture still requires work, but most of the problems to creating at least a minimally working system are already available.
What's missing is a good way to specify user preferences. How can a person in an Interactive Workspace tell the infrastructure to route all full-motion video to the big display while watching a film, otherwise remder stock ticker data as a 3D graph and show that? And of course, the user must be able to change these routings dynamically. What sort of interface is presented to the user by the room to effect this sort of control? Perhaps this control panel can also be routed to different devices by the user? ("Show a control on everyone's palm pilot")
It is worth noting that applications in this Interactive Workspace must externalize their data in a convenient and useful format in order to enable flexible routing and usage of this data. One simple example is outputting a 3D model of the data in some platform neutral format (VRML?) as opposed to OpenGL commands (OpenGL might work well for a visual display, but what happens when you want to output the 3D model to a Phantom device? --Disclaimer: I do not know whether the Phantom device accepts OpenGL... if you know, please tell me).
Also, note that most of what is described above is not specific to application output but can also apply to user input (mice, pointers, keyboards, joysticks, speech interfaces, etc.) to applications. However, there are complications when user input must be coupled to an application's output (e.g., user input through a GUI). The question of how you might render an arbitrary user interface for display on an arbitrary device is also quite difficult to say the least (e.g., what sort of user interface description would render well to both a large screen display, a small screen PDA, and to an aural/speech interface?).
3. A Short Scenario
Unbeknownst to his advisor, PhD student Bob is shirking from his academic duties to day trade stocks. He and a number of other students are using the Civil Engineering department's Interactive Workspace to track their own portfolios and display vast quantities of market data in real-time.
The Stock Trading application is a source of data: the current prices of stocks, trading volume, etc. The application is tied in to Bob's ETrade account so that he can execute trades. In addition, Bob runs a separate television application tuned to CNN, and a news clipper that retrieves news from the clarinet.finance newsgroups.
Bob has set his user preferences to route video to any one of the wall screens. He says he wants his portfolio summary to be routed to his Palm Pilot. He wants to be able to trade stocks on the table display. His preferences route Clarinet Finance news through a filter, and then display them only if the article mentions a stock he owns.
Notice that Bob is composing two separate applications in the last step. He's transforming the Clarinet Finance news and his stock list together to produce a display of news relating to his stocks. Since both of these applications make no assumptions about how their data is used, it's simple to use a 3rd-party transformer (the filter) to compose applications in 'novel' ways. If both the Stock and News applications had insisted on sending data directly to a screen display, this composition would have been much more difficult to accomplish.
Now, let's say a new device is added to the Interactive Workspace. Bob decides he wants a calmer environment while he's day trading, with fewer displays flashing data at him. He buys a small waterfall fountain to place in the room. The pump in the waterfall has a variable-speed, and can send water trickling or gushing through the fountain. Bob networks his waterfall, and changes his preferences to route the level of activity to it. He routes the trading volume and prices of the stocks in portfolio, and the number of news articles relating to his stocks, weights them each accordingly, and sums these numbers together. This sum is then routed to control the speed of the waterfall pump.
Bob has added a completely new device to the Interactive Workspace, and connected it to existing applications by writing two data transformers. One transformer to weight each data stream according, and another transformer to sum all these weights together. He's quickly extended the workspace in a way the original designer most likely did not anticipate. But he didn't have to change anything in the existing system to do it. Because the applications externalize their data in a useful format, and the infrastructure makes it easy to route and manipulate this data, the process of integrating new devices into the workspace has become much simpler.
Summary:
An Interactive Workspace should be controlled dynamically by the user. And as much as physically possible, this control should not be limited a priori by the designers of applications or devices within the workspace. By designing an infrastructure to support arbitrary routing and transformation of the data output from applications, we can decouple the application from the devices in the room. By doing this, and by encouraging application designers to externalize application data in a useful form, an Interactive Workspace will become more customizable and more extensible by the user.