Tim Bass wrote an excellent post the other day in which he questioned the categorization process around “Streaming Databases” and “Complex Event Processing” products.
Tim’s post inspired me to write something that I have been meaning to write for a very long time. This has to do with how some CEP products have won "Fluff" awards over the past few years.
In this (http://www.coral8.com/news/pr/145.html) press release, Coral8 announced that they have won the 2007 Jolt Award. According to the press release, “The Coral8 Engine was selected from among 26 product nominations in the Database Engines and Data Tools category by a renowned team of judges that select Jolt award winners based on how nominees have made it faster and easier to create data-centric applications.”
In this (http://www.aleri.com/news/press-releases/details/?pressID=32) press release, Aleri trots out the fact that they won the 2008 Jolt Award, which I assume tells people that Aleri is now better than the 2007 Jolt winner.
“Being recognized by Jolt’s renowned panel of judges consisting of industry insiders, columnist, and technology leaders demonstrates the impact that our complex event processing software has had on the industry,” said Don DeLoach, CEO of Aleri.
Both Aleri and Coral8 were graded by a “renowned panel of judges”. I guess that Terry and Don both drank the Jolt Kool-Aid (sic) that day :-)
Another award that was just won by Coral8 was from Windows in Financial Services magazine, which is a free magazine given out to people in the financial community. This magazine has absolutely no technical content, and reads like one large advertisement for Microsoft.
“Coral8 was a clear choice for the 2008 WFS Innovator Award because of their demonstrated leadership in providing solutions that deliver results,” said Windows in Financial Services publisher Joe Viviani. “We evaluated companies of all sizes and the companies selected delivered ground-breaking applications and platforms that better serve the securities industry.”
Who did the judging for this award? What were the criteria? It was interesting to see that Coral8 was the only CEP vendor that was showing off their wares at the Microsoft booth at SIFMA. Did WFS magazine happen to consider Apama? How about Streambase? (Oops … sorry WFS readers ... Streambase is written in Java)
Just who are these judges? According to Don, they are columnists and technology leaders. Wow! With respect to my journalistic colleagues like Greg McSweeny, when was the last time a columnist cracked open Coral8, wrote some CCL, and forced market data feeds into the pipe. Have any of these “technology leaders” taken Aleri and built and end-to-end application with it? Most of the columnists I know have trouble understanding the things that they write about.
What were the criteria that these renowned judges used to benchmark the systems? They should let us know, because this might mean that we don’t need the Stac Council anymore. The CEP working group of the Stac Council spends a few hours a few trying to work out comprehensive and objective testing specs for CEP products. The Stac Council is comprised of the chief architects of most of the CEP vendors, plus several customers from Capital Markets firms.
I can imagine the reaction of people like Jon/Jack/Scott from Aleri, Mark/Bob from Coral8, and Thomas from Esper when they read about these Jolt awards. These people have worked hard to provide compelling architectures, and awards like these Jolt Awards tend to marginalize their work.
I am not picking on Coral8 and Aleri here. I love both companies, and both have great products, great technical staffs, and CEO’s who are passionate about their products. What I find disturbing is the fact that they are giving credence to awards that, to me, seem like pure fluff. I talk to my colleagues at our competitors about their evaluation processes, and believe me when I tell you that the Jolt or WFS awards have zero impact on our evaluation process. These awards also marginalize the hard work of Stac to provide real benchmarks that companies can use when evaluating these products.
Awards are fine when the judging process is transparent to all. Until then, CEP vendors should not insult their customers by crowing about fluff. It works to your detriment rather than to your advantage.
©2008 Marc Adler - All Rights Reserved.
All opinions here are personal, and have no relation to my employer.
Subscribe to:
Post Comments (Atom)
1 comment:
Hi Marc,
Think that you have made a good general point about awards and the way in which companies may trumpet about what they have won.
IMHO, this often applies in a more immature market - think of all the awards dished out to dot-com companies.
So where a product vendor has a serious offering, they shouldn't make a fuss about such 'fluffy' awards as that doesn't really mean much. Potential customers want to know about real-world deployments and success stories, not some vague calculation of what might be the best in a very tightly-defined market (and the definition of that market may not be the same as the view from potential customers).
Post a Comment