Beware of Apples vs. Oranges in Enterprise-Software Market Research

A blog post about a quarterly enterprise software market tracker called the Software Insider Index popped up in my inbox this weekend.  I don't know what the numbers are indexing because the entire tracker is not provided. (The blog post is a promo for purchasing the index, which is good marketing of course.) And the source, Ray Wang, is a respected enterprise-software product analyst whose posts I regularly follow because they provide anecdotal backup to our statistical demand-side research about enterprise software users.

However the Software Insider Index post reminds me to remind IT investors again, as I said in this recent post about the Software 100, that it's important to compare apples and oranges in these listings and indices and any other supply-side software-market research device purporting to size or forecast software market growth. IT investors need to realize that if the Software Insider Index is supposed to illustrate software market growth (and I don't know if that is its intention; very few enterprise software suppliers are followed by the analyst according to his post -- 27, he says), the implication of double-digit growth based on the numbers provided in the blog post overstate what is really happening in the market. 

The problem is that the research does not appear to adjust for same-store sales. I did not check every supplier mentioned but -- for example -- the growth-rate numbers shown for JDA (JDAS), SAP, and Oracle (ORCL) appear not to take into account the previous year's comparable enterprise software revenues of i2, Sybase and Sun. There's nothing wrong with that under generally accepted accounting principles but it completely screws up any attempt to size the market and estimate historic quarterly growth rates unless the underlying research also has a -$300 million entry for Sun and -$100 million entries for the other two acquired companies.

And if you can't size the market and understand past growth parameters, you can't forecast it.

There are some other obvious apples vs. oranges problem in the article but again that could be because I am not looking at the entire underlying report. Here are some of the problems I see just based on the publicly available information:

• Odd categorizations.  How is the company CA Technologies (CA), formerly known as CA after it was formerly known as Computer Associates and CAI (and maybe by a few other names that can't be printed uttered by angry investors), all of a sudden an "on premise apps vendor?" I haven't looked at them in a while but their website still says they are in the IT Management sector. So why wasn't BMC included?  Maybe because its license sales were pretty good in its most recent quarter and that arithmetic would ruin the meme about so-called software as a service (SaaS) vs. on-premise vendors?
• Mixing revenue streams. The Oracle (ORCL) number shown under "on premise apps" has some estimation of the Oracle apps group's share of professional services built into it but doesn't seem to include Oracle's $100-million-plus (quarterly) OnDemand revenue. Nor does that revenue show up in the table of so-called SaaS vendors. And why isn't most of Oracle's revenue showing up in the Middleware vendors table? (And does the underlying index capture the Oracle software revenue buried in the new Oracle Hardware Systems category?)  
• Speaking of so-called SaaS vendors. Apparently the bar to getting into this category is pretty low because the analysis accepts all the guys like NetSuite (N), RightNow (RNOW), and Taleo (TLEO) that failed to make it in the real enterprise-software market about 10 years ago -- and therefore slowly morphed to SaaS (as I described here in August 2009) -- but it also includes 20-year-old companies such as Ultimate (ULTI) that just abruptly called its license and maintenance revenue SaaS in 2009 or Ariba (ARBA) that apparently didn't even stop using a license revenue bucket in its accounting but just stopped reporting one.

So are enterprise software suppliers doing better than the disaster that was 2009?  Of course. But if you really want to understand what is happening in the market (and do not want to purchase syndicated research), a better source for likely 2010 market growth is the data coming out of Gartner, predicting growth under 5% for 2010. And if you need company by company outlooks, Gartner and IDC and similar quant firms don't give their research away for free of course but if you are a subscriber they can also discuss individual company chances. 

They will also tell you that a quarter is too short a window and obtuse categorizations are not the way the world works.

-- Dennis Byron

(No financial interest in companies mentioned)

Comments

[Rwang0]

Hi Dennis,

Always very astute observations on your end. I thought I'd provide some clarity here on intent, methodology, and data.

INTENT
Yes, as you've stated, "Software Insider Index is supposed to illustrate software market growth"

The total index has both private and public company numbers. Keep in mind the quarterly tractor is only tracking quarterly YoY deltas. It's there as one data point of many to track market performance.

When we do the yearly index, we do have the estimates of about 80 public and private enterprise software companies. We tend to track companies above 100M in that index. Of course, there's a mix of specialty players, vertical players, mega vendors, etc. There is a market sizing model that drives the trends and predictions. In addition, we're not selling that piece of research either so, this is more informational and directional in intent and augmented by the deal flow we see.

METHODOLOGY
We have a consistent methodology for the past 3 years of taking and breaking out license, professional services, maintenance, hardware, and other revenue from not only the publicly traded financial statements but also privately available data and estimates.

We don't account for currency flux on a quarterly basis but factor this on a yearly basis for the index.

DATA
On data, you state,"..the implication of double-digit growth based on the numbers provided in the blog post overstate what is really happening in the market. "

Actually our private numbers also coincide w/ the forecast on double digit growth for publicly traded and privately held SaaS and middleware vendors. As a quarterly tracking summary, this is not revealed here in the index as privately held companies do not have audited GAAP's we can look at and we are making adjustments.

You state on Oracle, "The Oracle (ORCL) number shown under "on premise apps" has some estimation of the Oracle apps group's share of professional services built into it but doesn't seem to include Oracle's $100-million-plus (quarterly) OnDemand revenue. Nor does that revenue show up in the table of so-called SaaS vendors."

The Oracle number is a pure on-premises apps number. There's no mingling of middleware or OnDemand. Pro Services and Maintenance numbers are estimated from our primary research. The on-demand number is not included here in the on-premises section. You raise a good point, we should pull out the OnDemand number into the SaaS section which is really a mix of subscription revenue providers. More on that below...

You state, "Apparently the bar to getting into this category is pretty low because the analysis accepts all the guys like NetSuite (N), RightNow (RNOW), and Taleo (TLEO) that failed to make it in the real enterprise-software market about 10 years ago -- and therefore slowly morphed to SaaS (as I described here in August 2009) -- but it also includes 20-year-old companies such as Ultimate (ULTI) that just abruptly called its license and maintenance revenue SaaS in 2009 or Ariba (ARBA) that apparently didn't even stop using a license revenue bucket in its accounting but just stopped reporting one."

The SaaS numbers reflect both single tenant and multi-tenant on demand offerings. I'm using the "saas" term loosely here.

Thanks for the opportunity to comment. Always appreciate the good peer review!

- R "Ray" Wang
Software Insider's Point of View