Monitis: Cloud Monitoring Blog

Chief information honchos at companies are hot for cloud computing and virtualization in 2010, says a new report by Gartner.

In the study, Gartner asked CIOs to list their top technology priorities for 2010, and respondents ranked virtualization as their top area of focus. Next came cloud computing and web 2.0. That makes sense because Gartner researchers predicted that firms will invest more in hosted solutions this year and less in hardware – as budgets continue to shrink and be constrained.

And what are CIOs most concerned about? At the top of the list: business process improvement. The next biggest concern is reducing IT costs, according to the study.

I’m very happy to see that CIOs, the ones who hold the purse strings, are not just paying lip service to the cloud and virtualization. In fact, in a story I read elsewhere, Gartner has boldly predicted that one-fifth of all businesses will own absolutely no IT assets come 2012 – due to migration to cloud computing.

CIOs are getting more serious about incorporating cloud tools into their cost-saving strategies. Seems everyone wants a piece of the pie, as cloud computing and virtualization solutions continue to prove big money savers for companies, and more companies invest in the technologies.

As we start the New Year, I want to take a quick look back to review some of the incredible milestones we’ve witnessed and orchestrated here at Monitis. We’ve watched the cloud industry expand and grow – with new cloud providers launching their platforms and a variety of new service providers bringing their brands and tools to the cloud.

Last year was an incredibly successful year for Monitis, as more and more companies making the move to the cloud relied upon our monitoring services (such as external monitoring, back-end monitoring, web traffic monitoring, transaction monitoring and EC2/S3 cloud monitoring.

We grew our customer base by 400% and our revenue by 500% last year. As you might guess, I’m ecstatic about those numbers, not just because it means we’re doing better, but because it represents growth for the whole industry, too. And I think it proves growing recognition by companies that accessing services, information and apps via the cloud – but safely – is the future of IT.

One of our biggest thrills this past year happened in November, at The 451 Group’s 4th Annual Client Conference in Boston. We competed with a group of other technology companies in the event’s “Innovators Showcase,” and we gave a presentation called “Monitoring in the Cloud: Monitor Anything from Anywhere.” We won! and we were named “Most Innovative Start-Up of 2009.”

2009 was also a year of continuous enhancements and innovations to our services. I’ll list them below, and as you read through these developments and click on the links to learn more detail, you’ll be able to chart just how far we’ve come in delivering state-of-the-art, cloud-based network and systems monitoring .

I’m very proud, and I’m grateful beyond words to you, our customers. I sincerely hope you’ll help us usher in a new year of enhancements and services that, I am certain, will help you grow and expand your own businesses. We’ve got some exciting things planned!

Our 2009 Milestones:

- Launch of ‘Top 10′ service – providing network and systems engineers with a holistic view of processes and applications that are consuming the most resources, enabling them to quickly diagnose or prevent problems and properly match IT infrastructure capacity to business needs.

- Launch of asset management as a service, which automatically creates inventory of software, logs usage patterns and proactively suggests optimization to help companies reduce IT cost.

- Announcement of remote monitoring of system events.

- Launch of performance testing as a service, enabling companies to instantly run site performance tests, as well as keep historical records and manage performance scripts.

- Launch of public reporting and widgets, features that enabled our customers to make their websites’ uptime statistics publicly available – for the benefit of their own customers.

- Launch of our cloud monitoring service

- The addition of a monitoring location in China – expanding our global coverage

- The addition of a new external monitoring location within Amazon EC2 cloud network, allowing customers of the cloud provider to check their users’ web experience locally.

- Tweet alerts for network failures on Twitter

- Management Information Base (MIB) browsing for MonitorSNMP. A MIB is a type of database used to manage the devices in a communications network and comprises a collection of objects in a virtual database used to manage a network’s routers and switches.

- Introduction of remote monitoring to work on the Sun’s Solaris platform

- A free links checker service to detect broken and dead-end website links and alert users

- An interface for command-line tools to help make IT folks more productive and efficient

- A web ecosystem visualization service called WebMap, which helps IT engineers and managers understand their networks better in terms of status, health and manageability.

- Monitis S3 Monitoring, an on-demand cloud storage and usage service, enabling customers to independently monitor Amazon S3, notifying them when they reach prescribed thresholds.

- Database performance management and load testing from the cloud.

- User-friendly enhancements, such as the ability to manage all external monitors from a single, central location, enabling customers to edit network settings, monitor timeouts, change monitoring locations, schedule maintenance, and schedule and set up notification rules for all monitors from a single place.

- A new scheduling feature for our on-demand load testing

- Special holiday pricing for e-commerce monitoring – to help prevent site downtime

- MonitorSNMP, an enterprise-grade SNMP network monitoring service available via the cloud

- A time-saving universal cloud monitoring framework that enables external and internal monitoring from all cloud-hosting providers including Rackspace Cloud, Amazon Web Services and GoGrid.

- The ability for customers to create custom, end-to-end locations for server monitoring locations

- A free service to instantly check website response times from different locations

- Enterprise-class options for system failure and performance outage notification management

One of the ‘hype’ messages about the Cloud is that it is green – meaning environmentally friendlier than companies running vast resource-wasting machines that need constant energy for maintenance. Well, is this merely hype, or is there any truth to the claim.

Cnet news raises the question, too, and quotes a great blog post on the subject that brings up some realistic points about the Cloud’s level of greenness. First, the problem is that is difficult to measure cloud greenness with the information that’s out there because none of the major cloud companies are providing utilization data.

I especially like this quote: “We now have the ability to run our applications on thousands of servers, but previously this wasn’t even possible. To say it another way, we can potentially use several years worth of energy in literary a few hours, where previously this wasn’t even an option.”

So, wait. Doesn’t that make Cloud computing more of an energy gobbler than companies’ current IT methods? The author says: “…hypothetically we’re using more resources, not less.”

But conversely, if companies possessed those thousand servers and ran them (underutilized), the power usage per utilized server would be significantly higher. The author adds: “But then again, buying those servers would have been out reach for most, so it’s not a fair comparison. There we are–back, at where we started. You may use 80 percent less energy per unit, but have 1,000 percent more capacity, which at the end of the day means you’re using more, not less energy.”

You might look at this issue with the U.S. Interstate Highway system in mind. Eisenhower kicked off the project in the 50s, and the development of super-highways cutting across underdeveloped lands opened those places to development – and more consumption of resources.

Another cloud blogger quoted in the cnet piece added: “…Cloud computing is all about providing standard components as services (it’s pure volume operations). The problem of course is that we will end up consuming more of these standard components because it’s so easy to do so (i.e. in old speak, there is less yak shaving) and it becomes easier to build new and more exciting services on these (standing on the shoulders of giants). ”

Certainly, there are plenty of stories out there about how governments and companies are saving and being more efficient by running apps and data centers on the cloud. But that’s not the same as green computing, is it?

But I hold out my judgment on the greenness of cloud computing until we as an industry come up with an efficient, standardized way of measuring it.

Virtual desktops – the combination of virtual access, application virtualization and virtual machine software – aren’t new, but why aren’t they in more widespread use? a recent article that I read asks.

After all, desktop virtualization offers IT folks help managing desktop systems – from dealing with both operating system and application updates to unexpected failures to security for mobile systems.

On this blog, I even wrote about the birth of a new cloud-based “D” or desktop by Sun Microsystems.

But blockages to adoption by enterprises may include the following:

- Psychological: Don’t mess with it if it “ain’t” broke, and leave well enough alone.

- In today’s world, running apps and OSs directly on physical, internal servers works – although it certainly isn’t efficient, and it’s costly. So, even though it’s not the perfect solution, lots of IT managers are afraid to change it to something they feel adds risk. Besides, most desktop virtualization approaches require changes to operation of desktop systems, applications and the like. It’s extra – and many might say unnecessary – work!

- Service issues: Even though the performance issues of virtual desktops are decreasing as time goes by, support issues are a reality – although increasingly minor on a person-to-person level. But large organizations can count up support instances and consider service a larger factor.

- Aversion to being “lassoed and dragged.” I love that quote, and bravo to the author of the piece for his creative way of describing how users of desktops issued by their company are averse to the move to virtualization because they’ve customized their systems with Twitter and the like, and they don’t want to lose that. Unfortunately, rather than deal with a whole lot of one-off conversions, IT often lassoes and drags desktop users in a standardize fashion. “People don’t like to be lassoed and dragged,” he says.

What I really connected with in this article is the last point: that the development of the cloud and the trend toward “cloud-based personal productivity applications” such as email and document and calendar management might change minds about moving to virtual environments.

I’d like to add the economic factor here, too. Yes, it’s tough to change human nature, but at some point, many companies, large and small, will be convinced of the savings and efficiencies of going virtual – in part due to the cloud – and that will add fuel to the fire of change. And once companies begin to feel comfortable with the reliability of virtualization and the cloud and cloud services (such as transaction monitoring), there will be more migration.

With IT – just like other corporate departments – under the gun from senior management to centralize and reap economies of scale, distributed systems seem the dream answer to cost-cutting directives from above. To be sum it all up, distributed systems offer an attractive solution – centralized control along with dispersed physical assets and overhead.

In a recent article I read, the author put it just right: “The relentlessly increasing cost and complexity of maintaining IT departments and infrastructure makes on-tap computing power very attractive to modern enterprises.”

But beyond this simple cost-efficiency based explanation of the benefits of cloud computing for businesses, here are five reasons to make the switch:

- Scalability: PaaS is the end goal of businesses moving to the cloud. It’s the idea that a platform – hosting SaaS apps, can itself be run as a service. Consumers already use the cloud in a SaaS environment. SaaS applications running on a PaaS are easy to deploy, just the opposite of the complex and difficult environment at most companies today – where companies exhaust resources and manpower deploying, maintaining, and upgrading software systems “using skills way outside their businesses’ core function of interest simply because the resulting application is indispensable,” says the story. How true! Over a grid, the power of existing systems can be scaled exponentially, maintained seamlessly, upgraded transparently, and redeployed at will.

- Speed: Grid-powered PaaS and their apps can help increase productivity and a business’s competitiveness with fast deployment of new systems or the speedy ramping up existing ones. In addition, users can tap SaaS apps the run on grid-based PaaSs from anywhere globally…fast and without any special training.

- Deployment: Developers can pare weeks off integration and configuration time with hosting environments for their cloud platforms. On-demand network computing power and storage capability means that companies don’t have to invest the bank and everything under the mattress in static hardware.

- Cost: As I said above, this is one of the biggest advantages for companies making the move to the cloud. Why? “Grids pull across projects to provide more predictability at less cost with fewer software licenses, ” says the story. On top of that, PaaS, combined with SaaS, allows businesses to pay only for the resources they actually use. And don’t’ forget about the savings from not investing in expensive hardware!

PaaS, distributed computing infrastructure and virtualization are working together to create powerful and fast networks. It’s the imaginative and powerful apps, managed services, such as network, server and transaction monitoring, and end-user apps that will complete the picture to bring about a true cloud revolution.

There’s a great, although somewhat scientifically-centric story on the future of cloud computing that’s worth reading.

Essentially, the story asks: “Is cloud computing a useful computational platform for the computational problems you and your institution need to address?”  In other words, why is computing on the cloud better than maintaining internal servers, hardware and all the resources needed to support them?

Right off the bat, the story points out a great reason: “Through the use of pre-configured virtual machines, scientists avoid any of the technical issues associated with getting an application to run — they just use it.”

But a good point to consider is that, since cloud computing is on the Web, the first and most obvious question to ask is whether the Internet connection used by your cloud computing service provides enough bandwidth to handle your app demands. And does it add too much latency? This author says that the cloud was designed as a “latency-tolerant network,” and may not be suitable for certain users – such as scientific and HPC computing. Interestingly, he points out that “many HPC algorithms and applications… bottleneck and die” even after just a few microseconds.

But for many other types of users, enterprises from large corporations to small and mid-sized web-based businesses, cloud providers offer plenty of bandwidth – even considering the presence of virtual machines. VMs can add to latency issues because of the time-sharing nature of their underlying hardware. This is so even though users may never have a clue of a VM failure running in a cloud environment. Some cloud computing providers take snapshots of the VMS as they run jobs, and if the hardware fails for one VM, then all can be restarted from that snapshot.

But while the author of this article may have mixed feelings about the cloud for the scientific community, he underscores, however, that “one strong appeal of cloud computing” is the simplicity of contracting with an Internet provider to access a lot of free-floating computational resources. It’s an “astounding bang for the buck” platform that needs very little capital expenditure, he notes.

Further he gives a boost to the Amazon Elastic Compute Cloud (EC2) as one example of a cloud computing service that is available to the scientific community, as users can specify VMS that can run on unused hardware resources within their datacenters as those resources become available. This way, scientists only pay for those resources they actually use.

That’s a key benefit of the cloud for all users – not just scientists, I might add.

In the end, after an analysis of the cloud for scientific computing, he adds: “The future looks bright, although there are some big clouds out there! Happy “cloud” computing.”

I, as the owner of a cloud-computing service provider who takes great care to provide proper bandwidth, security, cost-savings and other benefits to my customers, couldn’t agree more.

It helps to have some guidance at hand when you are in the market for cloud computing services.

That’s why I’m recommending that you check out a recent article on “Five Competitive Differentiators” for cloud services. The piece covers SaaS, PaaS and IaaS. Actually, the article is written from the vendor point of view, noting: “If you are selling these services, consider this an outline for your next requirements document.”

But I think it will help businesses know what is unique to each cloud computing service and therefore help them make the right buying decision.

First mentioned is ease of operations, and this touches directly on the types of services Monitis offers. The story asks, “For instance, how does a company with hundreds of applications in the cloud strewn across a dozen or more vendors monitor and manage those applications to manageable service levels?”

User interfaces are bound to set some providers apart from their competitors, but it’s the work that goes on behind the scenes, such as APIs, publish and subscribe event streams, transparency and auditability systems, etc., that “will make the most significant differences between providers,” the story says. Even when the industry adopts more standardized processes, providers will still differentiate through extensions, quality of features and custom interfaces.

Next up: configurability. The story says that “…ultimately the complexity of the problems we wish to solve with information technology will dictate the amount of configurability we require from our infrastructure systems–even if they are delivered as a service by a third party.”

IaaS vendors can distinguish themselves with network architecture and data and server storage options, as well as services to enhance infrastructure – like security systems, message queuing, and storage tiering.

Performance, or speed, is another big differentiator. Cloud providers can fine tune processing speed, memory speed, storage access, read and write speeds, latency and bandwidth. And they can do this either through technology acquisition or via plain, old-fashioned superior engineering and operations expertise. So ask your vendor their speed-per-dollar rate!

Improved reliability and security is sorely needed in today’s cloud infrastructure – in light of all the very public outages and down incidences on consumer cloud applications. The author of the story calls it “risk mitigation,” and providers can demonstrate differentiation in both of these categories. For example, features such as redundant distributed data stores, early warning DDoS detection events, auditability API, transaction and EC2 monitoring can increase the transparency of both operations and security. Businesses need to trust that providers have made the cloud safe and dependable.

Lastly, cloud computing providers can stand out in the area of customer service. The author of the story says: “In a tongue-in-cheek post from early 2008, I noted that system administrators should “get good at waiting on hold for customer service representatives.” In reality, there is truth to that, but the providers have a lot of room to craft that experience.”

In my opinion, it’s worth it to ask your provider how you can:

-Collect data when a problem occurs

-Report a problem with a click

-Create a “self-service” case with fields to track the progress you’re making on issue resolution, which can be “mined” by the vendor’s support organization to discover trending bugs, etc.

-Easily search providers’ documentation for work-arounds or solutions

-Get help from a chat session

-Buy premium support for a single incident

###

There’s an interesting article you should check out on the lofty barrier to entry for cloud service providers. Yes, the cloud promises to reduce costs and increase efficiencies, but there’s often huge issues around investments in infrastructure, redundancy, staffing and market development for small and mid-sized companies.

As a result, new partnership opportunities in delivering and selling the cloud are arising. And according to one study, there are a variety of different cloud computing partnership models that enable solution providers to claim a stake in the cloud with varying degrees of investment. Competition in at least one area that I’m well familiar with – cloud-based transaction monitoring – continues to heat up.

Partnering provides a path to adoption and a place in the cloud, but not all partnerships and vendors are equal. A new white paper, “Finding Your Silver Lining in Cloud Computing,” offers potential cloud services partners insight and best practices into how to check each other out before combining talents.

But I also think that it’s worthwhile for customers of the cloud, including small businesses who want to access apps and store data, to understand the nature of their provider’s professional relationships and how services are delivered, among other things.

According to the white paper, investigation is worthwhile across eight different areas. These categories are:

- market position and value,

- vendor financial health,

- required investments and potential ROI,

- adoption requirements (organizational changes),

- vendor performance track record,

- customer relationships (protecting your customers),

- market awareness and marketing support,

- legal liabilities and regulatory requirements.

I’m sure there are a lot more things to consider and question for both cloud services providers who want to partner, as well as for actual customers. But these areas seem like good places to start for anybody who wants to go into a cloud service relationship – or, on the other hand, access those services – and with eyes and ears wide open.

I think it’s very valuable (and sometimes enjoyable) to read what top technology executives think about the future of the cloud. It not only gives you an idea of where their companies are going, in terms of their own development, but you sometimes get a good overview – and you pick up some surprises – of where the whole industry is headed.

That’s why I so enjoyed reading a recent article that interviewed Microsoft Architect David Chou. He believes that Infrastructure-as-a-service (IaaS) is the area of cloud computing that will make the most noticeable impact in 2010 – especially for startups and small to medium-sized businesses.

In the story, Chou says product planners, marketing, business executives and other non-IT people in businesses are the biggest category of people who are currently not using the cloud – but who should be. He said: “IT people tend to look at the cloud as an infrastructure option, but we think the cloud is an enabling technology for new business offerings and capabilities.”

Asked how important SOA was to the cloud – and vice-versa – Chou said: “Absolutely important!” Chou stressed that SOA was even more crucial on the cloud than traditional SOA patterns of enterprise-wide “transformational efforts” (big SOA). He thinks that, for the cloud, it’s more important to integrate with private clouds, other cloud services and applications, breaching individual cloud vendor boundaries and enterprise firewalls. And so SOA is essential in spanning all types of distributed services.

Also, interesting to note: In the piece, one of the top five cloud computing companies Chou names is Microsoft. Gee, what a surprise! The others he mentions, by the way, are: Amazon, Google, Salesforce and VMWare. He thinks the landscape will change quickly, though, as IBM, Oracle, and SAP “solidify/clarify” their plans.

Chou points to the East (China), too, and mentions some possible rising stars: companies like Baidu and Tencent/QQ.

Nice to get somewhat of a big picture view, and also very cool to read about Microsoft’s view of the cloud as an IaaS tool. And, if the non-IT audience that Chou mentions migrates more to the cloud, I’m predicting that their companies will step up monitoring efforts to ensure things run smoothly.

At a recent Gartner symposium in Orlando, FL, one of its analysts said that the cloud has topped the company’s list of the 10 top tech that IT people need to plan for.

While acknowledging that cloud computing is complicated and presents security challenges for some, Garnter analyst Dave Cearley advised companies to figure out:

- what cloud services might give them value,

- how to write applications that run on cloud services,

- whether they should build their own private clouds that use Internet-style networking technology within a company’s firewall.

The advice on cloud computing came as part of a talk on top 2010 trends that companies should incorporate into their strategic planning, if not their own computers.

Other trends on the list:

2. advanced analytics

3. client computing

4. IT for green

5. reshaping the data center

6. social computing

7. security–activity monitoring

8. flash memory

9. virtualization for availability

10. mobile applications.

Another analyst highlighted virtualization as a top trend, not just broadly, in the sense that virtualization lets a single computer run multiple operating systems simultaneously, but also as a means to keep computing services up and running despite computer failures.

At the symposium, Carl Claunch said that virtual machines can be moved from one physical machine to another today. But later, by keeping two machines synchronized, a failure in a first machine can be eased over rapidly by moving the active service to the backup.

“We should start seeing this roll out in the next year or two from vendors,” he said.

Read more of the story on cloud computing and virtualization.