Archive for the ‘WeoGeo’ Category

Spatial Is Special – There’s a (M)App for That

Thursday, November 12th, 2009

Part 2 – A New Hope

There have been some interesting conversations across the blogosphere on Google’s entrance into the basemap and navigation business. From an interested spectator point-of-view, I am a bit of a fan of the sheer audacity of Google’s mapping effort; creating a US basemap from scratch, building a navigation application, and rolling out a real-time sensor network (with 76 million devices) disguised as a mobile phone, all in the space of a couple of years. Well, that’s just crazy.  However, as much as I am a fan, this event is just one more example of the impacts of rapidly changing technology in our field (Part 1), making previous niches obsolete and forcing individuals (James Fee) and companies (e.g. Cloudmade) to adjust business models to just survive.

sadfa

Geo-powered apps are changing the game for developers

There were a couple of themes from these conversations that I found interesting. The first was represented by Peter Batty (and links therein), which could be broadly interpreted as “Content Matters”. That is, the cost (or lack there of) of the navigation application was less than important than the base content within it. The second theme was summarized by a podcast at Directions Magazine, which posited that the data didn’t matter. Adena and Joe’s point of view was that the application was going to overshadow the importance of the content. They felt that the application cost, functionality, and ease of use would generate more relative value than the content. I personally believe that both these points are two sides of the same business coin, and to be successful in this space you will have to have both great content, and a great marketing and delivery application.

Let’s start with the Batty et al.’s assertion that “Content is King”. In a frictionless marketplace of ideas and data, where the process of disintermediation has reached its peak, the lines of competition will be drawn around product price, branding (think Crest versus Colgate) and quality (think GM versus Lexus). This suggests that our market will continue to separate between the aggregation of (free) commodity data (10 meter USGS Digital Elevation Maps) and premium data (nationwide parcel data). I have been a proponent of premium content for a couple of years and I see it as a bright spot for geospatial professionals because good content tends to be localized providing niche geo-opportunities locally, across the globe.

An example of “premium-content-is-winning” may be found in the newspaper industry

Wall Street Journals Premium Content Drives Growth

Wall Street Journal's premium content drives growth

The Wall Street Journal has maintained its subscription-based revenue model all through the “content-should-be-free” era of the internet. It is now the only major newspaper in the US with a growing circulation, and is the nation’s largest newspaper by circulation. They demonstrate that the quality of content matters, and will continue to matter moving forward. Looking more closely at the difference in content provided by the #1 Wall Street Journal and the #2 USA Today, the Wall Street Journal might be considered a “premium” niche player, compared to the generic “consumer” news offer by the USA Today. Generic news may be found in a lot of places (for free). In the spatial industry, we might compare this to the “free” generic DEMS offered by the USGS and the “premium” high-resolution LiDAR surveys by companies such as Merrick. Clearly, quality content has value and people are willing to pay for it.

But Adena and Joe have a point. We consume digital content differently than other commodities such as toothpaste. Even if today’s premium navigation data becomes a commodity by virtue of the “less-the-free” data offered by Google, the data must still be delivered via a service and application that people wish to use. The consumption of content, and the value-added services built on top of the content, will depend directly on the quality of the application that delivers that content. In this case, the application will matter as much as the content. If either fails to deliver, the whole effort will fail.

While we may currently consider ourselves either data providers or application developers/integrators, I think our industry will move towards a Data-as-a-Service (DaaS) industry, where the data and applications are inextricably tied. I believe this will be the case for all platforms, including web, mobile, and desktop. And in this new DaaS era, the geospatial integrators may have an edge because they are used to thinking about data and applications simultaneously.

In addition, the economic process that is currently squeezing the revenues of integrators and GIS professionals may be turned to work in their favor, as the many sources of free, low cost, or open source software becomes their application supply chain. In the past, these integrators thought about their customer’s data and its integration within a proprietary mapping software stack. In the future, they have an opportunity to become vendors of DaaS, either for their customers, or for themselves. The combination of accessible data (low-cost or free) with low-cost applications and hosting will enable these professional to create premium DaaS niches, recouping the profit margins lost to the commodization of their IT skill set.

This is one path towards a new, hopeful future where the geospatial professional controls the fate of their destiny, rather than trying to survive on the leavings of giants.


A new hope for Geo-Jedi’s

Productivity and Cloning in the Spatial Data Industry

Wednesday, October 21st, 2009

Last week there was a brief twitter thread in the geo-community about cloning Adam Estrada. It started innocuously, but as such things often do amongst our chatter class of geospatial professionals, it degenerated rapidly. One could only imagine what the world would be like with multiple copies of a rabid Georgia Bulldog fan.

The technology for cloning GeoNerds has been developed.

This thread was followed by some great follow-up discussions from last week’s post on the new parcels in Google Maps (and see a better article here). It seems that our way of life in the professional mapping industry is going to change as a result of new data and technology that will be available at increasingly lower costs. That means that the players in the industry will have to change or die. Or put a little less morbidly, we will need to become increasingly more productive in order to stay ahead of the “free stuff” curve.

Yet, how do we become more productive with our time, e.g. make more money with less effort? Productivity varies across industries, and sectors within industries. Our industry can be basically broken down into three groups – software vendors, data vendors, and professional service vendors. In Adam’s case, he is in the professional service sector, working for Zekiah Technologies, and is in a similar position to many in the geospatial industry wrestling with the problem of being more productive.

The productivity problem with the professional service sector is that it is a consulting business. Revenues are generated as a function of billable hours, e.g. how many hours it will it take you to install an ESRI ArcGIS Server or build a map of Maryland. Therefore, to make more revenues, one has to generate more billable hours. Yet, the billable hours per person has an upper limit, say 40 hours per week (unless of course you are a lawyer, in which case it is greater than the 168 hours available to us mere mortals). And therein lies the conundrum, the professional services industry does not scale beyond the bodies you can hire. This means in the good times your upside is limited to the amount of dollars per hour you can charge times your total workable hours. And in the bad times, you have the fixed overhead of salary support for someone who isn’t fully billed out. Large downside risk, limited upside potential. Not the best of business combinations.

Given this combination, it is easy to see why Adam might wish to clone himself. If he could scale the number of hours he could work to the available billable hours he could sell, and not incur the additional overhead and risk, Adam would be less stressed, and would make more money. Of course he may have to share his Bulldog tickets with his clones, but there are disadvantages to bulldog cloning that I’ll leave for your imagination.

The geo-data industry sector is different. A lot of effort goes into developing the data product, but once developed this product may be sold multiple times. This make-once, sell-many product model is a great example of a scalable business. Like the software business, the cost of development is incurred upfront, after which the only marginal expenses are sales and marketing. In essence, instead of cloning yourself to garner more revenues, you clone your product (which is a bit easier than cloning yourself with today’s technology).

The problem with the data business model is the upfront costs, as well as the sales and marketing expenses once the data product has been produced. These costs can be significant and there is always the risk of, “if you build it, they won’t come.” So while it scales well, the risks are high. Given this risk and expense, many geo-professionals choose to continue along the path of lower risk, but lower upside, consulting business; and like Adam they dream of cloning themselves while working late in the evenings, instead of watching their favorite college football team.

Is there another path? I would like to think so, but it requires a blending of the data and professional services business model. In this new path, people like Adam, who create content (or advanced data transformers and geospatial analysis tools) on a daily basis for others, would retain an explicit stake in their days work at creating valuable content. Much like songwriters, who retain residuals on the songs that they write, GIS professionals who create products for others could retain royalties and derivative rights on their products. These royalties and rights would allow them to resell their works in a manner that would generate new revenues, which would scale far better than the original hours required to produce the original work.

There would need to be some trade-offs for the original purchasers, e.g perhaps they get a lower price for the job if Adam was able to retain the derivative rights to his works. In this case, Adam could bet a little of his current revenue in the hopes of generating a better scalable revenue stream. This would be a win-win for both parties, as the original purchasers would be getting a product a lower cost (and thereby increasing their overall productivity); and Adam would be able to generate more revenue with less effort, which could possibly create a new business model for greater success in our industry.

The Internet Is Not the Computer – Yet

Tuesday, October 6th, 2009

Has an IT professional in your organization told you that Software-as-a-Service (SaaS) just does not work for your enterprise?  I am going to help their argument; then I’ll tear it down.

The biggest problem for a professional engineering organization with SaaS is latency.  Also called disk-access-time , data transfer rate, bit rate.  Rapid communication between processing units and data storage pools is critical for today’s professional computing efforts.  If you don’t believe me, try opening (and editing) a large GIS or CAD file via a VPN from a network storage device operating in a satellite location from across the country.

For example – Our realized transfer rates from our facility in Portland to Amazon Web Services (AWS) in Virginia is 3.5 megabits per second (mb/s).  We have a 100 megabit per second dedicated service pipe to our facility.  This latency is not Amazon’s fault, but rather it is a function of the speed of light and the number of Internet “hops” or network transfer points between WeoGeo and AWS.  From Florida, we average about 7 mb/s to the same facility.  This issue is one of the fundamental reasons AWS released their Import/Export feature to S3.

To put this into perspective, in a typical disk drive on a desktop computer (circa 2008) operating at 7200 rpm operates at a disk access transfer speed of 560 – 2400 mb/s (70 – 300 megabytes per second).  That is 2 – 3 orders of magnitude faster than accessing the same file via the Internet.  The last time desktop computer users “suffered” thru 5 mb/s disk access transfer speeds was in 1987 when IBM released the PS/2 with a 5 MegaByte ST-506 Seagate Technologies hard drive.

When will latency be reduced to the point where we might consider “the internet is the computer”?  That is kind of hard to say.  Even with ubiquitous broadband access at >100 mb/s to all business, we will stuff suffer the “speed of light” problems – photons can only go so fast.  In addition, every time you have to go through an Internet junction or telecommunication switch (i.e. hops) you will increase the packet transfer times.  A report from the National Broadband Coalition (also covered here) suggests the pipe speeds to small and medium business will not approach those of hard disk drives until sometime between 2015 and 2020 (see table below).

This suggests that the internet-as-the-computer to replace your desktop is still some time away, maybe as long as 3 – 5 years.  This latency argument is what many in corporate IT departments would use to strike against SaaS, PaaS, or IaaS services within your organizations.  (I am purposely ignoring security, but will address this issue in another post).  I would counter that those arguments are very similar to the ones once used against the IBM PS/2 as a corporate workhorse back-in-the-day; and ultimately I believe they are rooted more in bureaucratic inertia than true cost/benefit analysis to the enterprise.

Internet-based services have a place in today’s enterprise environment.  Depending on the use case, they can be more efficient in managing your limited IT dollars.  These services can also provide greater, timelier, software support, which together with the lower costs, increases the bottom-line productivity of your organization.  More importantly, these managed, off-site, services will have a greater place in your organization tomorrow.  Data transfer speeds will increase to a point that the latency issues will be negligible for the services you require.  As a decision maker in your organization – are you planning for greater productivity and enhanced profits tomorrow, or adding to the “mainframe” infrastructure of today?

Why the Platform Is Important

Thursday, September 17th, 2009

Sometimes we can miss how large marketplaces can be, and their importance to development new ecosystems and technologies. The iPhone app market gives a glimpse as to why a SaaS-based application platform is so important for future geospatial business growth.

While the iPhone article describes a consumer based platform (mobile phones) for applications, the concept for a geospatial application platform is similar. By providing a platform on which developers can “write their own visions” of products, more products are developed, tested, purchased, and discarded at a much higher frequency than previous enterprise development cycles. This increase in the velocity of the product development cycle will provide opportunities in the future beyond those previously seen in the traditional software business. It will also increase the risk of marginalization of current software packages.

Our mission has been to develop a means to bring platform technologies to the geospatial industry. We focused on content first, because most of the current value of geo-knowledge is “stored” in the content produced under professional services and consulting contracts. The geospatial content are also larger and has proprietary file issues that are not necessarily seen in the consumer web space, which add to the difficulties in providing a true application platform to the Spatial Data Infrastructure (SDI).

However, it became clear to us about 2 years ago that cracking the content index, search, customization, and delivery process required developing an application platform along with the content management platform. So we began the process of creating an application platform that is accessible through APIs, which we then used to build our content management platform. Furthermore,we sought a partnership with a vendor who could supply additional transforms and ETL functions (Safe Software), which we could expose via our SaaS offerings, to allow others to build their own applications. We have thus created a geospatial application platform that will allow others to rapidly create their own small, but very targeted applications. By providing this application platform at the same time as providing the content management and financial services, all on a scalable SaaS-based architecture, we are poised to create an ecosystem around developer-driven applications and content.

This application/content (de-)evolution from the traditional software vending models is happening all around us. It will come to the SDI, probably a little later than other consumer driven niches, but it is coming. We strive to be that platform that others use to make money, and in that process help change our world.

Businesses Bucking Cloud Computing

Wednesday, September 9th, 2009

I just returned from holidays with my family and am spending hours catching up on my blog reading. The advantages to reading so much, so quickly, is that articles separated by time and distance take on new meaning when compared together. This happened when I read about the Gmail failure and this article on a survey of Small and Medium Business (SMB) use of cloud-based services.

The survey begs the question, “Why are SMBs not rushing to use cloud-based services?”   This got me to thinking about our own experience at WeoGeo with cloud-based content management services, as well as our discussions with hundreds of SMB customers. I think I can break this down into 3 key points.

  1. Compartmentalization – The top cloud-services used in the survey are conceptually simple to understand and compartmentalize from an IT and business perspective. These web-hosting (#1 in the survey), email (#2), and data archival (#3) services can be broken apart from the daily operations of any business unit and provided back to that business unit as a service, either internally by an in-house IT operation, or externally from a service provider, i.e. Google Mail. The workflow within a business unit is not impacted by the internal or external delivery of these services.
  2. Uptime and bandwidth – These services require a high degree of availability, whose functionality can be felt by every single person in the organization. Ask an IT manager what makes their cell phone ring more – the downtime of a processing computer, or the downtime of the email server. In addition, the internal IT solution downtime can be measured and compared easily against the performance metrics of external “cloud” vendors. Prior to the GMail failure of 100 minutes this past week, Google was apparently averaging 10-15 minutes per month. That is better than most in-house operations.
  3. Cost can be measured and is favorable for the use case – When we switched from our own Microsoft Exchange Mail Server to Google Mail Premium, the cost savings was enormous. We estimated our all-in costs on the Exchange Server at ~$30,000 per year for 15 employees, including IT staff, outside contractors, data center charges, storage and archival expenses, and spam filters. $2,000 per employee per year. Google Mail Premium is $50 per email box per year. The math is pretty simple.

What about the other Infrastructure-, Platform-, and Software-as-a-Service products? My own thoughts are that outside of the tech industry, these other services do not appear to be as easily compartmentalized within an organization. Most business applications seem to run on workstations or internal services quite fine, and the downtime of those processing units just don’t seem to resonant with the IT and business leaders of an organization. In addition, the “simple math” of extreme costs savings (a la Google Mail) of moving more complicated operations to cloud services has yet to be definitely proven.

There are other concerns like the security and integrity of production or sensitive data that also have slowed the uptake of these services within the SMB community. However, I think that providing compelling solutions to business problems that can be (1) compartmentalized and (2) provide a high degree of availability at (3) a dramatically lower total cost of ownership will help allay those concerns. Make this case, and the greater adoption of cloud services by SMBs will be assured.

WebMapSocial Meetup in Mountain View, CA

Monday, July 20th, 2009

I will be speaking at the WebMapSocial Meetup tomorrow, hosted by Google at their Mountain View facility. I will be following presentations by NASA WorldWind and BrightKite.

If you are in the Bay area and want some free dinner with other geo-types, come by for some fun.

The WebMapSocial Meetup is organized by Catherine Burton of Endpoint Environmental LLC.

WebMapSocial

Scaling FME Engines on WeoGeo

Friday, June 19th, 2009

I presented the movie below as part of a presentation at the Safe Software FME User Conference. We had a great time and the Safe crew put on a marvelous show.

The movie shows WeoGeo scaling up to 64 Safe Software distributed FME Engines in the production of tile caches from a world-wide elevation database. The FME Workspace script was created by Dmitri Bagh, and processed on WeoGeo’s FME Constellation built on Amazon Web Services.

The scaling occurred automatically, spinning up FME Engine AMIs, and then shutting them down when the job queue was completed. This is one of our first examples of bringing scalable processing to difficult geospatial tasks.

Examples of the tiles created by Dmitri’s script for Virtual Earth (Bing Maps for Enterprise) and Google Earth can be found here.

Panel 1 (upper left hand corner) refers to the total number of engines in the constellation processing job.

Panel 2 (upper right hand corner) refers to the total constellation utilization percentage. The constellation is polled and when the utilization exceeds the pre-set threshold (50% in this example), it increases (doubles here) the number of engines until it reaches the pre-set maximum number of engines (64 here). The downward spikes occur when each new set of engines are added.

Panel 3 (lower left hand corner) is the average job processing time. There is an increase in velocity when the number of engines exceeds 16, which may be a function of increased overhead costs on the FME Core or bandwidth to the database.

Panel 4 (lower right hand corner) is the total number of jobs completed. 2000 jobs were submitted for this test. The job completion rate accelerates until the maximum number of engines are brought on-line.

Mapping Down Under

Friday, February 20th, 2009
Moreton Bay Research Station Computer Room trying to unlock the mysteries of physics-based inversion algorithms

Moreton Bay Research Station Computer Room trying to unlock the mysteries of physics-based inversion algorithms

I am still involved with various spectral mapping efforts from aircraft and space. I spent the past week at the Moreton Bay Research Station on North Stradbroke Island in Queensland, Australia with a bunch of people trying to determine sea grass distributions from imagery. Most of the time we were locked into this room –

But I was able to get out one day to look at the environment we were trying to map, with guys who actually know the environment and can tell one form of sea grass from another.

One of the sea grass species we were trying to identify from aircraft imagery.  (Hey Stuart and Chris, help me out here, what is the species name?)

One of the sea grass species (Cymodocea serrulata) we were trying to identify from aircraft imagery.

Below we were engaged in putting the “map on the pixel” instead of the traditional other way around (I am the geek in the hat).

I rarely write about such endeavors, mostly because of time, but a thought crystallized in my mind during this trip that is relevant for what we are trying to do here. The mapping industry has created some expensive tools (consider that the LandSat follow-on is likely to be over $700 million). Yet the “last mile” of knowledge is always created by someone in the field, applying their skills and knowledge to the data coming from the survey instruments.

Our mission is to make the human part of this process more efficient and more rewarding. When we succeed in this mission, we will be creating a healthier place to live, with better funding for the experts to pursue their studies and acquisition of knowledge.

map3

On the left, Chris Roelfsema is showing where we are on the map. On the right, we are "putting the map on the pixel".

My thanks to the down under crew of Stuart Phinn, Arnold Dekker, Chris Roelfsema, Mitchell Lyons, and the Moreton Bay Research Station for an excellent workshop and their wonderful hospitality.

A TerraColor Example

Friday, February 6th, 2009

We have a client (TerraColor) that was looking for better ways to market their content on the WeoGeo Market. So we created an iFrame and KML link with their full content listing to have some different ways to find and purchase Terracolor content on the Market.

He took the iFrame and embedded it on his site to directly brand and market the maps to his customers. He now has a fully functioning hosting and eCommerce site for a fraction of the cost and time of doing it himself.

With the release of our APIs, the Market experience can be further customized to a unique customer desired solution, generating many different ways to store, host, search, and deliver geo-enabled content. This is just one example of how we are trying to move towards platform-level services for our customers.

earthstar_geographics_sml

It’s About Changing the Rules

Wednesday, May 30th, 2007

We launched WeoGeo yesterday at the O’Reilly’s Where 2.0 Conference in San Jose – a milestone I’m obviously very happy to have reached, as we have been working on WeoGeo for nearly the last two years. During this development period I have been constantly asked, “What is WeoGeo?” I am finally glad to be able to tell our story.

It’s about breaking the content cycle.

The map exchange business currently requires large investments in equipment, software, and personnel to facilitate the sale of most data maps, yet our revenue opportunities tend to be limited to negotiated contracts, where each map product is delivered to a single buyer. Why? Because our ~$4 billion market is one of stove-piped verticals, where mappings skills do not translate well across the horizontal breath of mapping needs. This market is growing, yet fragmented; ripe for the necessary restructuring that will enable significant growth.

It’s about shifting the exchange paradigm.

This paradigm in the mapping business limits content creation, value-added application development, and, ultimately, profits in our industry. We seek to create a new paradigm in the mapping world where content and skills are as important as contacts and capitalization. And so to accomplish this, we are launching a “NeoMapping Market,” where buyers and sellers of mapping content can communicate via a transaction-based exchange. A marketplace where the fees are kept to a minimum to support enhanced availability of supply and increased demand of products. A digital marketplace where a product need only be made once yet can be sold many times.

It’s about reinventing the license and royalty model.

Ultimately, the highest valued product on this planet is intellectual property (IP). The potential value of geographic IP itself makes it one of the greatest untapped resources of our era. So how do we enable the creation of a viable geospatial marketplace for the exchange of this data? To create a true digital IP marketplace, we have to change the Digital Rights Management (DRM) model. Mapping products are ultimately about the skill and tools used to create the content, yet the vast majority of our supply and demandmost useful mapping products go largely unused because of onerous licensing terms. The current licensing model adopted by our industry creates a supply (S0) and demand (D0) curve similar to this graph. Yet if we want to move from a $4 billion market to a $40 billion market, we must transition to the curves much more like S1 and D1, where the total size of the market is greater, with greater availability and accessibility to products. To accomplish this, we must break the current constraints of the mapping DRM model, and develop new ones where content creators (sellers) are paid for their efforts regardless of who uses or buys the content while buyers are allowed to develop value-added derivatives without limitations.

It’s about changing the buying process.

A marketplace for the highest value geographic IP will not be successful without a consistent and easy search mechanism. We have sought to create such a place, where sellers have the capability to list their high-resolution products on a global level, using a simple, yet powerful, interface. An online market allowing sellers to offer portions of their data at similarly fractioned prices, thus enabling buyers to select only what they require and spend only what they need. For buyers, such a marketplace offers the ability to quickly sort all mapping products in an area, including derivative products from popular maps. It also lets buyers track any lineage of derivative works to assess the quality and popularity of each derivative product.

It’s about re-envisioning Web 2.0 technologies.

Our mapping products are global in nature and (greater than) petabytes in size. An easy interface alone is insufficient for a world-wide geo IP market; it must be backed by the necessary technology to move such mountains. Specifically, WeoGeo is built on Amazon Web Services (AWS). As a global leader in ecommerce, Amazon has created their own a paradigm shift in the cluster computing market with AWS. From commodity computing cycles to scalable bandwidth and storage capacity, AWS’ services provide WeoGeo with the necessary platform to make a NeoMapping Market a global reality.

Join us in making it happen. Let us change the way we value our skills, ideas, and mapping content. And in the process, let us change the way we value the world.