Appendix B
Rendered Information
Sensors and Human Input
The perceptualization environment is designed to serve the intensive information needs of communications/information specialists in any domain open to the benefit of such a technology. To give a context to what we are proposing for sensors and human input we could use an example from a refugee camp scenario which includes medical complications. We have prototyped a system which allows us to
a. collect data from diverse sensors on the ground
· physiological: heart, blood pressure, blood glucose, etc.
· chemical: air quality, water quality
· whether: temperature, wind direction
b. collect data from human input devices
· PDA’s: Palm Pilots equipped with software that allows a user to walk through a refugee camp and enter data into a form covering elements like number of water containers, pounds of soy protein, lamp fuel, pints of blood, bandages, and so on.
The images below are screen shots of the system we are developing for the sensor and human input into the perceptualization environment.
In this case, the representation is a topographic map of Hawaii with some emerging geometries. These shapes may represent features on the ground as indicated in the categories to the left.
As more information comes in through sensors and human input via the PDA into the form fields of those doing assessment on the ground, the interface’s representation is added to and perceptually enriched in real time. Here we can see place names and geometries emerging on the heels of the data streams.
This image shows more fully the evolution of the representation in real time as information continues to come into the system. The images seen here are not complicated. However, the innovation we bring to this element of the interface is in the abstraction of some information as geometries. These geometry data are then fused with the topographic map to represent many parameters from ground data localized by means of GPS coordinates. The latter are provided as part of our ground based sensor/PDA technology. The next and very important piece of this perceptualization picture is in the representation of information queried from electronic databases.
Text into Geometries—Bibliomics
For a number of years Mindtel has envisioned a technology for accessing and representing data contained within electronic data storage systems in new ways. The majority of information an intelligence[1] user would need to get at is stored in databases. These databases are often located in geographically diverse locations. Moreover, data is often stored in different formats with different securities. The goal for the perceptualization environment is to have data from these many and diverse sources available and accessible to particular users in particular contexts of need.
Bibliomics is a representational technology Mindtel conceived to handle the vast and growing sources of data needed by a whole host of users from those working the areas of intelligence, medicine, disaster relief, stock trading, and many others. The idea of Bibliomics is to take textual/numerical data and begin to find ways of abstracting critical elements of it for presentation to a user who either has scarce time for its raw form, or else who needs to ‘see’ features of that data normally invisible when it exists as text or numbers. The latter covers that information in data we do not even know could be there. New representations will often disclose what was not know to exist before it could be represented.
This means that textual
representation will be substantially transformed because human visual
information processing is hindered as text can be a non-optimal representation
when confronting the factors of content volume and time limits (e.g.,
intelligence emergencies). Therefore,
signifiers other than text ought to be explored and experimented with. Mindtel is doing this work by looking to
visual forms of representation which are open to multi-modal features for
conveying meaning (e.g., geometric shapes, colors, movements, shape changes,
etc.). Below are some examples of how
we might transform textual data into new kinds of signifiers for rapid and
meaning-rich visualizations. These are
only arbitrary forms which if useful would then be assigned multi-modal significance. For example what if visual representation
for Nasdaq output could be condensed and more meaningfully abstracted in some
way like this? 
Or what if movements of terrorist operatives and their diverse resources could be condensed from multiple data stores and more quickly and meaningfully visualized?
For the sake of example, think in the following manner about this image. The small flat image on the left is a geographical topography akin to the one of Hawaii above. Whereas the geographical images representing Hawaii ground information were more basic, the image on the right side above is a much richer geometric complexification of the diverse intelligence data initiated on the left. In the above example the following elements could be represented: the intelligence user has input a query requesting everything from the geographic location of suspected or known terrorist personnel clusters, to amount of money and locations of their bank accounts, to their spending patterns of the past 2 years including what/where they have purchased and in what amounts. Furthermore, queries could look for the location of crop dusters both owned and for sale, or else one could query certain chemicals, where they are stored, who is buying them, in what amounts. The queried information could incorporate a large number of variables. The image on the right would be a typical Bibliomic data fusion and dynamic geometrification of the many database streams collected in the search.
Or what if a disaster relief team was concerned about water supplies in a region?
This set of images could be an abstract and dynamic visualization of the needed data.
These are merely examples of the kinds of thinking we are proposing for how to address the problem of altering the representations of information. The essential point is to open up new ways of thinking and talking about the perceptualization and expression of the vast and diverse data which we and our sponsors will desire from the perceptualization environment.