The panchromatic limitations of WorldView-1, recently launched by Digital Globe, have brought a few posts (e.g. free geography tools and the confused life) on the fusion of high spatial resolution panchromatic imagery (PAN) with lower spatial resolution multispectral imagery (MSI). I thought I would briefly comment on image fusion because over the years it has become easier to accomplish, but the results or limitation of the fused product may be difficult to understand.
There are many ways to accomplish pan-sharpening including band substitution, color space transformation and substitution, and Principle Component Substitution (Jensen,2005). As mentioned on the confused life, temporal decorrelations introduce artifacts into a fused, or PAN-sharpened image. However there are other artifacts that can be equally important if one is trying to create a quantitative product for classification mapping or target detection.
The inherent difficulty with all of the PAN sharpening methods is that they are fundamentally based on the technical and environmental conditions under which the PAN imagery was collected. Since it is difficult, if not impossible, to accurately correct for illumination and atmospheric conditions in PAN sharpened imagery (subject for a much longer post), the PAN-sharpened images may be limited to classification and detection within a scene. Inter-scene comparisons (i.e. change detection between scenes or cross scene classifications) using spectral properties require the aforementioned corrections. In addition, when the instantaneous field of view (IFOV) of the PAN and/or MSI sensors are too large, spectral and illumination changes will be present at the edges of the image, making even within scene classifications difficult. Because of these issues, PAN-sharpened multispectral images are frequently used to identify features based on relative color differences within an image, rather than target identification or environmental characterization based on a spectral signature itself.
Figure 1. The fusion of high spatial resolution MSI (left figure) with lower spatial resolution HSI (middle figure) into a high spatial resolution, high spectral resolution image (right image). The bottom row of images represents the spectral plots at the pixel located at the center of the red cross hairs in the images directly above them.
We have done some work in this area, mainly focused on sharpening hyperspectral imagery (HSI) with multispectral imagery (MSI). Figure 1 shows the results of some of our efforts. The left image is a high resolution MSI from an Applanix DSS. Underneath it is the digital value of the RGB channel of the image. The middle image is the lower spatial resolution HSI; and underneath it is the full spectrum resolution of the HSI vector (~3 nm resolution). By fusing these two images together (right image), we were able to create a high spatial resolution sharpened HSI image whose spectral vector matched reasonably well with the spectral vector from the original HSI image. The use of atmospheric- and illumination-corrected HSI imagery means that we could make classification comparisons or target detections using these spectra much more robustly across scenes in time and space.
When making fused, or derivative, mapping products the value of the map is critically determined by the base mapping material and the skill of the map producer. Understanding the limitations of the base mapping material as well as the fusion techniques themselves is a critical determinate in the value of a derivative mapping product.
References
Jensen, John J., Introductory Digital Image Processing: A Remote Sensing Perspective. Prentice-Hall, Englewood Cliffs, NJ, 2005, 526 pp.
Tags: FERI, Hyperspectral, mapping, Remote Sensing
Interesting post.
1. WorldView-1 data products will be no use to Google Earth etc. for now and in the future, UNLESS Google wants to include some up-to-date b/w images for some special events. I guess the majority of us – the general public – just ogle beautiful color pictures.
2. In theory, there is a methodology called “pan-sharpening” to fuse high-res b/w imagery with low-res colour imagery for making high-res color composite. But with this, it is crucial to have both inputs captured almost at the same time. Recall that our earth surface and its objects are dynamic and changing always.
3. What is pan-sharpening? An introduction is here
http://www.geosage.com/highview/imagefusion.html
http://www.google.com/search?tab=nw&hl=en&ned=us&q=pan-sharpening
I second Bullsworld comment – very interesting.
How do you handle abrupt transitions from one kind of surface to another, where they’re cleanly resolved in the MSI imagery, but not in the HSI?
@Leszek – Good question. Much depends on the sharpening technique used. Consider the simple case of a PAN sharpened HSI image in your question. If using a rotation/substitution approach, where most of the variance was contained in the first eigenvector of the rotated HSI image, then even a PAN sharpening of the HSI should do fairly well in resolving the spectral signal between features that are cleanly resolved in the high spatial resolution PAN image. There are some caveats to this for really dark targets in a bright scene, but in general this should be true.
The difficulty occurs when the rotation displays a lot of variance across a lot of eigenvectors. This is where the MSI provides some additional degrees of freedom with which to play your sharpening games. In the case shown here, a well understood, well calibrated set of instruments can deliver a lot of hidden information. We have a long history in hyperspectral imaging, including designing, building, calibrating and deploying these sensors. The images used in the post were from a coupled sensor package that we built and flew ourselves.
As part of the calibration suite we perform before deployments, we measure the instruments’ Spectral Response function (SRF) to the same radiometric standards. These calibrated sensors provide us with high quality images MSI who’s known SRF and radiometric characteristics provide us with more degrees of freedom than a PAN image with which to play sharpening games with HSI imagery.