I was saddened today by the news of Jim Gray. I heard about it from my colleague who pointed me to the efforts of Michael Arrington at TechCrunch and Werner Vogels at Amazon. I feel somewhat connected to the effort because of the hours spent on Michael’s site, and our development of a new internet business using Amazon’s S3/EC2 systems. Mostly I feel connected because finding things in the ocean using imagery is what we do.

My first thought was we can help, particularly after I saw that the NASA ER2 flew with a hyperspectral imager. This is what we do. We recently demonstrated (see here as well) the capability to NOAA NESDIS to collect and process nearly 4000 square kilometers of coastal ocean hyperspectral (5 m resolution, 256 channels in the visible and near infrared) and multispectral (0.8 m resolution, 3 channels) data in less than 18 hours. Our flight imagery is ~1 TB in raw form, and up to 5 TB processed, and we are some of the best people I know at the imagery and processing game. I figured that since we have an EC2/S3 account for WeoGeo, so we could upload some of our image processing software and get in there and help.

It was then that my colleague had to rein me in. Jim Gray had been missing since last Sunday, and the ER2 data was very limited. The oceanographer in me took a deep breath, and after reviewing more about the availability of the imagery, I realized there was probably very little that we could do to help. The ocean is a big place, and while the amount of imagery was large, the ocean was a lot larger.

In addition, the visible imagery was limited to just a few bands. Just a few bands means that there are limited degrees of freedom to use automated feature extraction techniques (that is a techie term that just means to use the computer to sift through the imagery to yield the information for which you are searching). The fewer the bands, the more that sensor, illumination, and environmental noise dominate the imagery, the less likely you will be able to find the object of your search.

Werner Vogels sought to use one of the best tools he had available, the Mechanical Turk. It was one of the quickest methods to put eyeballs on the imagery. By using S3, they had the means to store and distribute large volumes of imagery. Unfortunately, people’s eyes are just not that sensitive to noisy, low spectral information. It is very hard to “see” something in ocean imagery. Particularly if it has been compressed in some part of the processing, which frequently removes all the targets you are interested in finding. That’s why we use high resolution spectral and spatial data and develop the processing algorithms to have the computer render these volumes of data into the maps that tell us something important. In military parlance, it is call actionable geospatial intelligence. In this case, it is about saving lives.

Spectral imaging is not the only means to find things on the water. There are other systems that can be used for ship tracking. Microsoft’s Vexcel has the capabilities to use SAR data for this purpose, and I am sure they will put these to use. It is a credit to Werner and the community that the have been able to respond as rapidly as they have. However, I am still feeling a sense of failure. Our community (scientific, engineering, imaging, GIS, etc.) knows how to accomplish these types of mapping goals to save lives and property. The problem is that there has not been enough demand in the results to justify the expenditures at the current price of the systems and products.

The systems that we fly are $1 million+. The processing costs are $10,000s (sometimes up to $100,000s) per day of operation. The issue is one of scalability and demand pull. For an integrated Search And Rescue (SAR) system to have provided help to Jim Gray, it would have needed to be a fraction of those costs, rapidly deployed on manned and unmanned vehicles flying at high altitudes (including space), delivering actionable maps within hours (if not minutes) of landing or downlink. Such technology is obtainable, but the capital investment is large.

We are trying as hard as we can, to the best of our abilities, to change the mapping game by creating and sharing knowledge, not just pictures. This will take time.

My heart and prayers go out to the friends and family of Jim Gray. I just wish we could help today.

Update: 1730 EST, February 5, 2007
I spoke with a contact at NASA JPL. It appears that the NASA ER2 flew without the hyperspectral sensor, but with another imaging package. WPB