Monitis: Cloud Monitoring Blog

Next week, from November 3rd to the 4th , I’ll be in Boston for The 451 Group’s 4th Annual Client Conference at the Boston Marriott Longwharf. Monitis will be part of a special group of companies in an “Innovator’s Showcase.”

We’ll have our own kiosk, and, if you’re attending, it would be a great opportunity to say ‘hello!’ I’d love it if you could stop by and introduce yourselves, and perhaps I can tell you a bit more about the cloud-based monitoring services we provide.

Plus, if you stop by, I’d like to hear about your IT challenges and how you’re managing change in today’s volatile landscape.

If you’re a client of The 451 Group, check out their Conference agenda and register! I look forward to seeing you there.

Learn more at our recent press release: Leading Analyst Rates Monitis as a Cloud Company to Watch

According to the Azure Journal, Microsoft on October 15th released the SQL Azure October CTP (Commercial Technology Preview) 2. This CTP represents the complete feature set that will be available in the SQL Azure at PDC.

This CTP has been deployed to one of Microsoft’s go-live production clusters, one that is larger and more powerful than the machine cluster that has been supporting the August CTP. But it is a completely separate machine cluster serviced by a dedicated developer portal.

Microsoft said that accounts for all existing users of its August CTP have been automatically provisioned for access to its new October CTP and environment.

This is one of a slew of Microsoft Azure product announcements and enhancements in recent months.

Earlier in October, the company said that, at launch time, Windows Azure will feature an improved logging system and give users greater flexibility over what information is logged and how it is collected. Its first CTP provided a logging API, enabling developers to write custom messages to an append-only log.

The API was built on top of the event tracing capabilities of Windows. But Microsoft said its new logging system will enable IT managers to collect other kinds of data, too. One example: performance counters. Users can automatically push their logs to Windows Azure storage at specified intervals and in an easy-to-query format.  In addition, users can reconfigure logging on an ad-hoc basis, meaning they won’t have to figure out up-front exactly what data they’ll need to debug problems. That can be decided as needed.

And back in September, Microsoft released a preview of the Windows Azure Service Management API to help IT managers, well, manage their deployments, hosted services and storage accounts. The API preview is REST-based, which means that users can code against in their toolset of choice to manage their services.

Curious? You can find the documentation (along with the rest of the Windows Azure documentation) here.

This Azure API uses X509 client certificates for authentication. Users can upload any valid X509 certificate in .cer format to the Windows Azure developer portal and then use it as a client certificate when making API requests.

This preview API currently supports:

Deployments – Viewing, creating, deleting, swapping, modifying configuration settings, changing instance counts, and updating the deployment.

Listing and viewing properties for hosted services, storage accounts and affinity groups.

Microsoft also made available a small tool, called csmanage.exe, to help IT managers interact with this API and manage deployments.

The Delaware Department of Technology and Information (DTI) has deployed the industry’s only virtualized application delivery solution. The agency needed to rejuvenate its aging application infrastructure, and it wanted to scale application delivery statewide with security and performance. The move involved virtualizing the agency’s old load balancers – not replacing them.

On a Cisco blog about the virtual computing development, Scott Huffman of Delaware’s DTI, explained:  

“We were able to take several of the older devices, collapse them down into one ACE [application control engine] module and virtualize it, and we quickly found that we no longer needed separate hardware. We were able to make virtual instances and were even able to take some of the devices from our previous load-balancing solution and virtualize them in ACE. That was definitely a winner for us.”

 ”Given the state of our current economy, anywhere I’m able to find savings in the IT budget puts our group in a good light. And the fact that we were able to package several of the older devices down into one ACE module was huge.”

Congrats to DTI. I think it’s great that they were able to use both existing and virtual load balancers to save money.

But I think it’s even wiser that they invested in a virtualized app delivery solution instead of replacing old load balancers with new, more powerful ones. They saved money (Isn’t t hat what everybody wants these days?), and they instituted high performance, scalable and secure app delivery.

Regardless of which solution you may choose for your organization, it’s imperative that you keep your sites, networks, servers, apps and databases accessible and running efficiently and safe. That’s why it’s always a good idea to invest a bit more in monitoring systems that keep track of performance 24/7.

I wouldn’t wish rain on anybody – especially not from the cloud, or the promise of cloud computing.

But I wanted to share five hurdles in getting apps to run on clouds, from a recent article I read. Again, I know it’s a bit negative, but I think it’s good to be aware of these issues as the hype about cloud computing gets louder.

1. No one cloud platform is like the other; expect conflicts. Each cloud is different; specific migration, support, cost and capacity issues vary from vendor to vendor. Legacy apps have a lot of integration issues with your other systems. Usually, the integration has to be fast, and there’s a lot of direct database calls from one application to another. That may not work well when one app is outside the perimeter. Beware.

2. Security: the locks on clouds aren’t as fully tested as those on legacy apps. And many companies need to know who is using the apps or accessing the data and whether they’re authorized or not. Also, a multi-tenant cloud is not an option for firms with security or compliance concerns – because the monitoring and control tools have not been addressed. Perhaps a private cloud is the answer for those firms, but they still don’t offer a detailed audit trail that firms rely on to meet financial or privacy regulations.

3. Licensing and interoperability concerns. Neither major software vendors nor cloud providers are making migration of legacy apps easy. They’re built with databases, communications or data-translation modules, etc. So that means vendors like Oracle, Siebel, SAP and others would have to change their licensing to support cloud computing’s model, for instance, paying only for capacity that you use. Legacy apps typically also don’t support the latest technology – except in the user interfaces that aren’t part of their cores, and that’s exactly the technology on which cloud platforms are built.

4. Do you know what’s behind your legacy apps? Chances are, no. You may know your company’s line
of business apps, but not necessarily the endlessly-customized codes, interfaces and forms. Many times, you’ll need specially designed development, analysis and conversion tools to extract data, business logic and security information from legacy apps so they’ll run on the cloud.

5. Migration is mostly manual; few tools will help. Even at their best, the above tools only convert a portion of apps and data, and that leaves the end-user or service provider to deal with the rest. Solutions are in the works, though.

Here’s the link to the complete article on obstacles to converting apps to the cloud.

Are you an IT manager at a small business? Chances are you’re wearing many hats, one of them being IT. If that’s the case, then cloud computing may be right for you. It’s not just for huge enterprises, you know.

That’s the assessment of a recent article taking a look at the state of cloud computing in India. Even though the story focuses on that country, I think the points are pretty valid for small businesses everywhere.

Cloud computing offers flexibility and cost effectiveness that makes it perfect for small businesses. The reasons: Small businesses have a smaller employee base and lesser resource requirements than big companies. But their big challenge is capital investment – raising the money needed to expand infrastructure. “Cloud-based solutions such as SaaS and Infrastructure-as-a-service (IaaS), which cater to specific requirements, fit the bill for adoption by such organizations,” says the article.

Both small and large companies, too, can depend on cloud computing services for application development projects.

Ironically, it might even be easier for smaller companies – rather than large enterprises – to jump hurdles to cloud computing, according to the story. Why? Big companies already have a large amount of legacy infrastructure, so, until cloud computing is commoditized, “they can’t shift from an all-on-premise infrastructure to an all-on-cloud infrastructure.” So to speak, they’re dipping their toes in and using the cloud for certain aims, for example, data storage.

Also, for smaller firms, there’s the attractiveness of cloud-based apps. The story pointed out that, traditional outsourcing providers are now offering cloud environments where SaaS providers can come onboard and provide services. This is not only a good fit from the perspective of the cloud provider, who has the reach and managed services capability and can use the complementary services from SaaS providers, but it also means cheaper computing for small businesses.

These are interesting angles on how cloud computing can help small businesses, and I’m happy to share the article.

Should your private banking information or secret government files on where the Al Qaeda is hanging out in a cave in Afghanistan be stored on cloud data centers?

While a recent CIO.com survey found 46 percent of IT decision makers are investigating cloud storage services, there are lots of folks out there who don’t think certain types of information have any business there. A recent story on DaaS that I read in a mainstream (non-techie) newspaper quotes Charles King, an analyst with Pund-IT, saying that it’s not right for every type of data. Certain kinds of data will probably never make it to the cloud, the article quotes King as believing, including financial information and sensitive government documents.

Why?

Because privacy regulations and some very well-known data center security breaches make IT managers “squeamish,” says the story. I have certainly seen this attitude or sentiment among some of our clients, and I can understand why. It’s hard to pin down the nature of the cloud – where you aren’t sure where your data is stored. It’s a tough sell sometimes because companies like to know where their data is, and, more importantly, that it is encrypted during every single transmission.

It’s like you and I wanting to know where our teenagers are every night at 11 p.m. I can’t see them in their beds, so how do I know they’re okay.

So will our industry ever come up with a satisfactory way to address the doubts about security from CIO’s and IT managers? Take one area of security: There’s a long way to go before there is a single sign-on capability for all Web storage, although standards such as OAuth (to enable secure data access) are helping. In the story, a CIO at a company using the cloud for employees who need data backup for laptops, says cloud vendors will eventually figure out more secure ways of storage.

It’s as sure as when the IT industry “figured out network storage arrays, shared mainframe access and resolved other once-unattainable complexities.”

Read more about cloud storage issues.

OK, if you’re not convinced of cloud computing’s financial benefits yet, here’s a bit of information that should help convert you. In a recent article on the growing government cloud, I saw that the U.S. General Services Administration (GSA) has achieved significant cost savings by moving USA.gov to cloud infrastructure and services provider Terremark Worldwide.

Just how much? Remember now, this is money you and I, taxpayers, that is, shell out. It doesn’t come from some private corporation’s operating revenue. In the story, the GSA revealed that the annual cost of maintaining USA.gov in the cloud is $800,000. While that may seem like a lot of money, it pales in comparison to the $2.5 million GSA used to spend before the transition. The earlier figure included hardware costs, which the cloud vendor now covers.

Said Martha Dorris, deputy associate administrator for the Office of Citizen Services, which maintains USA.gov, in the story: “We were in a situation where our infrastructure required a lot of staff resources to maintain it every day. [Moving to the cloud] freed up resources, which we have used to upgrade our content management system to give us the ability to incorporate new functionalities and features on USA.gov to make the citizen’s experience the best it can be.”

Elasticity of infrastructure is crucial because USA.gov gets about 342,000 visits on any given day, and traffic fluctuates wildly depending on external events. So, recognizing that it’s important to plan for infrastructure overload and avoid downtime, Dorris added that the GSA recently bought additional space in the cloud to create a mirror image of the site in case the original goes down.

It’s easier, too, to upgrade USA.gov now that it’s operating in a cloud. It generally takes a day – versus up to six months the old way.

Still, despite the advantages of cloud computing, it apparently took the GSA IT staff some convincing to make the switch. “”It was a big change for the technical staff that worked in this environment,” Dorris noted in the article. “We took a lot of time and effort to make people feel comfortable.”

USA.gov was able to transition to the cloud in only 10 days, with one additional weekend of testing.

In this recession-plagued economy, when taxes are rising to cover the cost of more services and greater debt, it makes more sense than ever for the government to make smart changes – in order to be more efficient and save money. Glad to hear I know I’m saving some money, at least to keep USA.gov going.

See also How Taxpayers Are Saving via Cloud Computing

Versus

Have any of you IT folks out there ever wondered how Amazon Elastic Block Storage (EBS), a new type of storage designed specifically for EC2 instances, measures up to Amazon S3 on the elastic cloud? Here are some key differences in cloud performance that I ran across in a recent article:

- For a 20 Gigabyte volume, Amazon estimates an annual failure range for EBS volumes from 1-in-200 to 1-in-1000, increasing as the volume rises. So, that means, you should keep a current snapshot on S3, or back-up the contents elsewhere and get to it so that you can restore quickly.

- EBS accounts can have a maximum of 20 volumes unless Amazon requests a higher limit. The maximum size of a volume is 1 TB, and the storage on a volume is limited to the provisioned size. It cannot be changed. EBS volumes can only be accessed from an EC2 instance in the same availability zone. Meanwhile, snapshots on S3 can be accessed from any availability zone. 

- While S3 is subject to “eventual consistency” (delays in writes appearing in the system), EBS has no delays. Also EBS can only be accessed by one machine at a time, while S3 snapshots can be shared.   

- Where performance is concerned, S3 has the higher latency and also has higher variation in latency. In addition, S3 write latency can also be more frequent than read latency. EBS, on the other hand, has lower latency with less variation. It also has writeback caching for very low-write latency.

- Throughput: S3 has maximum throughput (single threaded ) of approximately 20 MB/s (or 25 MB/s  for multithreaded). This is on a small scale. Throughput rises to 50 MB/s on the large and extra-large instances. In comparison, EBS has a maximum throughput limited only by the network, This is approximately 25 MB/s on a small instance and 50 MB/s on large instances, and 100 MB/s on extra-large instances. Yet, as both S3 and EBS are shared resources, they are subject to slowdown under heavy traffic.

- For file listing, S3 is slow and search is by prefix only, whereas EBS has fast directory listing and searching.

See also Monitis S3 Monitoring Debuts

Seems we hear a lot about companies making halting steps to cloud computing by moving certain services off internal servers and onto the cloud, for example, email. So far much of the focus has been on saving money.

But increasingly, there’s evidence that companies are using their business (dollars and) sense to make money on the cloud. A San Francisco Chronicle story I read spotlighted how one company, TransUnion, a credit reporting company, which traditionally built custom software for banks, retailers and other companies to check the credit histories of consumers, is now offering revenue-generating applications to its customers as a cloud service.

The Old versus the New

Here’s how it used to work for TransUnion.

The company would build software for customers, but because development is people-intensive and can be slow, it typically cut into the profits that TransUnion could make on each project. It used to take weeks or months for the company to build a new service. On top of that, sometimes the firm had the headache of dealing with a backlog of requests – stretching sometimes for 90 days or more.

Would one of your customers wait around for a product or service for three months?

Now, TransUnion has begun offering its customers access to its data through APIs. Customers can now build their own applications and access consumer credit data via TransUnion’s private cloud. When a customer requests service, TransUnion can provide access to the appropriate APIs within a week. Big improvement over 90 days!

And as far as cost goes, TransUnion offers the same services and data as it always has, but opening APIs to customers costs less than developing custom software.

The company has not neglected security issues, either. It uses an access management appliance to control which customers can access which APIs. That is based on profiles that TransUnion and the customer collaborate on together.

I suggest you check out the full article on how companies are making money on the cloud. It’s a great read!

The Wall Street Journal today (October 12^th) featured an article, or rather, more of a questionnaire, that provided some basic knowledge about cloud computing – and done in a very fun way. I thought it was significant because such a mainstream business publication was giving subjects such as cloud computing and virtualization so much attention. Clearly, the subject of “the cloud” has moved well beyond the realm of discussions at the water cooler in the IT department.

The Journal said that “cloud computing is attracting a lot of interest from chief information officers and high-tech entrepreneurs. They like the promise of lower cost, less complexity and reduced power consumption.” With a series of 10 questions, the story also tested readers on their general knowledge of the cloud.

Here are a few of the Journal’s Qs&As on cloud computing:

- What exactly is cloud computing? Is it “lightweight software that takes up little space on a hard drive” or “computing resources that can be accessed on demand, like electricity from a utility.” Answer: the latter.

- Which of these techniques is vital for creating cloud computing centers? (Virtualization, Cannibalization, Transubstantiation, or Insubordination?) Answer: Virtualization.

- An internal cloud is: An overhanging threat or a cloud that sits behind a corporate firewall? The first is the right answer if you’re in a lousy mood. But the second choice is the right answer.

- What August event was widely seen as an example of the risky nature of cloud computing? Was it the spread of the Conficker virus, or was it Gmail outage for more than an hour? Answer: the August 31 outage of Google’s Gmail.

I highly recommend reading the WSJ’s coverage of cloud computing and the paper’s very entertaining Q&A!