Downloading spatial datasets these days is really no different than it was 10 years ago.  Rather than taking advantage of new technologies, these data sharing sites are really just HTML wrappers to FTP sites.  You navigate them like you navigate the folder structure on any computer.  This makes downloading datasets very difficult because you may have to drill down to get one GIS file and then jump back up and drill down again.

A couple years ago I needed to download some DEMs in California from their GIS data website.  These DEM’s were organized, as expected, like the DRGs.  So they have that wonderful folder structure that none of us can remember.

Since my area was small, I just wanted to grab the 7.5 minute quad map DEMs for my area.  So navigating to the 7.5_minute_series_albers_nad83_trimmed folder you get this:

Years ago I had access to a wonderful printed index to all the USGS Quad Maps in the USA.  Of course I no longer have this so trying to remember what Quad map number I need is almost impossible.  Now there are websites that will tell you this and you can download an index, but it is a huge pain in the rear and time consuming.  You can start seeing the problem though.  If all my DEMs I need are under one folder, I can just grab them all.  But the world never works that way and I had to grab DEMs from 4 folders for my area of interest.

So as I described above, I started to drill down into each folder and download the data I needed.  But because this was all HTML, I had to right click on every file and download it.  The story of course ended with my just writing a Perl script to scrape all the DEMs for California into my local directory and off I went.

The problem of course with this is that I grabbed way more than I would ever need.  Because the process was so painful, I just assumed to download everything locally and if I ever needed a new area, I had it available.  California (thanks guys) paid for bandwidth that I used up with my Perl script that just downloaded everything in its path.  The inefficiency of this process is totally clear.  I just wanted to get DEMs in my little area, but because the organization of this data was so archaic, I had to result to brutal measures to get what I needed.

Now what would have been better?  Well first off, I would have loved to just crop out the dataset I needed and then get only that as a download.  When you think about it, it is really just a simple GIS clip function.  We do it all the time, but why don’t we allow our users on the web to do it themselves?

Lets take this example in Florida in my WeoGeo Library:

If we look at the right panel we can see that the dataset has a huge size, almost 900 MB.

If I’m only interested in a little section of this data, why would I either pay for the whole thing or deal with a huge ENVI file when I really only want a small jpeg?  So as you can see below, by zooming into the small section I want and changing some setting to match what I want my output to be, I can reduce the file size from 835 MB to 8 K.  This means that rather than downloading a huge file and causing a lot of bandwidth costs, I was able to quickly get the file that I needed, for the area I needed in the format I needed.

Also this means that I as an organization wouldn’t have to have multiple versions of the same dataset on my web server.  I can keep the data in GeoTIFF and let people transform them into other formats.  Because we leverage Safe FME on the backend, we can transform to and from hundreds of data formats.  If you have users that need that in Swedish KF85, WeoGeo can enable that even if you don’t have any software on your own that can read or write it KF85, just upload any supported vector format and WeoGeo supports it.

Discovering data is only part of the story on the GeoWeb.  Getting this data in formats you can consume and use is really where the value happens to sharing data.  People are using data in so many different ways that trying to figure out how to support all these needs is becoming very difficult.  Because WeoGeo gives you the ability to allow your users to extract only the data they wish from your geodatabases and into one of the hundreds of formats that WeoGeo supports, you’ll have much happier customers, save money on your IT infrastructure and know that no matter what happens down the road you’ll be able to address those needs quickly and without worry.