Archive for the ‘WeoCEO’ Category

Scaling FME Engines on WeoGeo

Friday, June 19th, 2009

I presented the movie below as part of a presentation at the Safe Software FME User Conference. We had a great time and the Safe crew put on a marvelous show.

The movie shows WeoGeo scaling up to 64 Safe Software distributed FME Engines in the production of tile caches from a world-wide elevation database. The FME Workspace script was created by Dmitri Bagh, and processed on WeoGeo’s FME Constellation built on Amazon Web Services.

The scaling occurred automatically, spinning up FME Engine AMIs, and then shutting them down when the job queue was completed. This is one of our first examples of bringing scalable processing to difficult geospatial tasks.

Examples of the tiles created by Dmitri’s script for Virtual Earth (Bing Maps for Enterprise) and Google Earth can be found here.

Panel 1 (upper left hand corner) refers to the total number of engines in the constellation processing job.

Panel 2 (upper right hand corner) refers to the total constellation utilization percentage. The constellation is polled and when the utilization exceeds the pre-set threshold (50% in this example), it increases (doubles here) the number of engines until it reaches the pre-set maximum number of engines (64 here). The downward spikes occur when each new set of engines are added.

Panel 3 (lower left hand corner) is the average job processing time. There is an increase in velocity when the number of engines exceeds 16, which may be a function of increased overhead costs on the FME Core or bandwidth to the database.

Panel 4 (lower right hand corner) is the total number of jobs completed. 2000 jobs were submitted for this test. The job completion rate accelerates until the maximum number of engines are brought on-line.

WeoCEO Emerging From Private Beta

Thursday, October 18th, 2007

WeoGeo has created a scalable, fault-tolerant infrastructure to manage its use of Amazon Web Services Elastic Compute Cloud (EC2) operations. I’ve written about it a couple of times (see this link for a listing of the Amazon tagged blogs). The latest version of WeoCEO (Version 0.1.0) is ready for release and with it we are moving from private to public Beta.

This version includes the Assistant to back up WeoCEO (see this feature described in this Amazon Web Services StartUp Event Slide Show). WeoCEO Version 0.1.0 also provides enhancements to the stable IP addressing, failure detection, and automatic scaling and load balancing. These enhancements include automatic emailing to your site administrator during trouble events and detailed logging capabilities.

WeoCEO Version 0.1.0 (including the load-balancing and auto-scaling capabilities) will be free of charge at least until December 1, 2007. It will continue to be free if you only use the stable IP addressing and auto-recovery features for a single client instance.

There will be a charge for the load-balancing and auto-scaling features of WeoCEO, which support running multiple EC2 instances and optimizing your network. The charge for these features will be $0.05 per managed client instance per hour. The charge will be on the average usage over an hour, calculated at <15 minute intervals.

You can obtain a WeoCEO ISO with the setup and installation instructions, by visiting http://www.WeoCEO.com and clicking the “Signup” button, or by clicking the Signup button below. We are still in beta, so constructive comments on any of the components that make up this service will be met with exuberance and free goodies.

WeoCEO – How to Use the True Power of Amazon Web Services

Friday, March 9th, 2007

As mentioned earlier, Amazon Web Services (AWS) is offering an innovative solution that effectively provides a flexible outsourced data center for web services. The Elastic Computing Cloud (EC2) offers access to scalable computing power, with a “pay as you go” approach that allows users to increase and decrease their usage without penalty. This new concept eliminates upfront costs and offers incredible flexibility, but also has some limitations.

These limitations are significant. The most critical issues for EC2 are dynamic IP addressing coupled with a lack of a 24/7 service level agreement. Amazon’s current service agreement does not promise 24/7 operation of an AMI, so when problems bring down your web site’s AMI, you’ll also lose your web site due to the loss of a valid IP address for the “A” domain name record in your DNS service. Not only will you have to restart your AMI and all web services, but you must also repopulate the global DNS tables with the new “A” record.

WeoCEO is a proprietary application originally developed by WeoGeo to manage the use of EC2 in serving WeoGeo clients. This solution has already brought affordable scalability to WeoGeo and its own Web 2.0 applications, and is now being offered as a private beta product for developers of AWS applications, in order to enable others to tap into the true power of AWS.

The WeoCEO application, working within the EC2 environment, eliminates the above mentioned limitations, and maximizes the power of EC2 by providing automatic and instantaneous scaling, load balancing, and fail-safe supports, including a stable IP environment. These critical functions optimize usage and provide true 24/7 operations to make EC2 a powerful, intelligent solution for businesses of any size.

The EC2 model allows scalable capacity to accommodate anticipated changes in traffic levels and growth, but management of this is labor intensive and requires personnel to oversee traffic levels. WeoCEO is an intelligent manager program that fully automates those tasks, and provides efficient usage and appropriate capabilities to handle growth, cyclical needs, and sudden spikes in demand. A sudden influx of traffic generated by something like a TechCrunch or Digg article can cause catastrophic failures at the exact moment a new site is trying to capture users. With WeoCEO, the increased demand is automatically addressed with increased capacity to handle the load without delay. And when demand decreases, WeoCEO eliminates the excess capabilities, to eliminate excess costs.

WeoCEO also provides the critical fail-safe support in case of failure that will ensure true 24/7 operational capability. When problems cause a temporary loss of your website, WeoCEO’s automated system will retrieve a duplicate image, and have your site available again within moments. With redundant systems that automatically regenerate and provide a stable IP address environment, your site’s functionality is maximized.