It's Official: XQuery 1.0 Now a Standard

I was pleased to see in my Inbox RSS Reader this morning news that XQuery has been made a standard by the W3C.  Here's the press release.

When we started Ipedo back in mid-2000, I remember all the vague nods when I would ask people if they knew what XQuery was.  Now, as you can see from the release, there are over 40 products that support XQuery. 

I would like to take a moment to thank all of the contributors to the spec.  As you can read, they have taken over 1000 comments and created over 14,000 (!) use cases, in a really short period of time.  And they have taken flak for it, which I debunked here, since XQuery became a standard in about half the time it took SQL (okay, so I was off by a month...).

For us at Ipedo, not only is this a vindication of our initial vision, but a tremendous opportunity for us and our customers.   The volume of XML has never been greater - from contract documents (ACORD, MISMO, FpML) to RSS feeds, it grows every day.  And with Ipedo being the only vendor with combined SQL/XQuery federation, it means Ipedo can federate over all that XML data in Oracle, DB2 and SQL Server.  Not only have we been keeping our engine up to spec over the years, even being one of the first vendors to run conformance tests, but our customers get the benefit of years of real world production use and tuning in the field.

So today, I answer this question for the final time: "Is XQuery a standard yet?"

A resounding "Yes."

McKnight and Ferguson on EII

Came across a nice post covering EII on B-Eye today.  William McKnight and Mike Ferguson, who both know their way around BI and integration, posted their first in a series on EII.  They start right off with a pretty good general description of the motivation:

"EII is gaining traction for enabling data integration without the need for the physical instantiation of the integration. In other words, EII adds integrated reporting capabilities while minimizing impact on existing systems."

From there they go into a common use cases for EII.  Here's a link to the entry.  I think they pretty much nailed it, with perhaps the addition of SOA making it a complete set.  I look forward to reading the rest of their series and getting a bit more in depth.

Toward Simple Data Services

For those of you who might have missed it, we put out a new release of our flagship Ipedo XIP in December.  The theme was using EII in an SOA to create data services.

What we hear a lot when we're out talking to customers and prospects is that getting the data right in an SOA is hard, and that what's needed are tools to make it easier.   The last thing organizations want to do is write a lot of custom Java code just to get their SOA up and running.  Kind of defeats the purpose.

So what we've done is add a bunch of features to make it really simple and efficient to consume and publish Web Services.  You can see the full release for details, but here are the highlights:

• One-Click Publishing – The enhanced Web Services wizard allows simple one-click publishing of new Web Services, complete with security profiles, performance management, and automated WSDL generation.
  
• Dual Caching Option – Allows a Web Services View to be cached natively as XML, or in relational form, or both. Native caching improves performance and allows versioning of XML results. Using the relational option allows automatic decomposition of XML into relational form for use in other applications.
  
• Web Services Tables Metadata – Metadata enhancements for Ipedo's Web Services Tables display complete lineage and dependency information for inbound Web Services. Ipedo's Web Services Tables allow users to easily map XML-based Web Services into relational models.
  
• Join Hints for Web Services – Ipedo's query optimizer can now accept hints, including join strategy and join order, that minimize the number of Web Services calls needed to fulfill a query request, and thereby improve overall performance.
  
• XQuery for SOA – Ipedo's XQuery engine has been tuned to provide greater insight and better performance for SOA with a range of advanced functions, including XML payload introspection, fast transformation, content validation, and service metadata introspection. In addition, XQuery can be used to trigger subsequent 'cascading queries' based on the payload content of a Web Service.

I'd also like to put in a plug for our long XQuery heritage.  I think people are finally starting to realize how hard it is to deal with all the data in SOA payloads, and how good a tool XQuery is for the job.

Linthicum on the Importance of Data

Came across a nice podcast by David Linthicum today.  As you know from reading this blog, we're big believers in the importance of data in SOA - what we call data services.  In fact, we often wonder how people get their SOA projects off the ground without giving this serious consideration.

In his podcast, David talks about this very point.  Nice to hear an SOA expert using the terms 'data services' and 'virtual database.'  Take a listen.

You might also be interested in a new entry from our technical folks called The One-click Web Service.  To me it illustrates how powerful an EII server can be in an SOA project: all the benefits of SOA, simplicity of deployment, and high-performance data access.

SOA, Virtual Machines and Virtual Databases

I was reading the latest post to Joe McKendrick's Service-Oriented Architecture this morning, 'IBM exec: no SOA-virtualization convergence, yet.'  It triggered something we've been kicking around here for some time - the idea of the virtual database.

Now, the IBM exec quoted in the blog was making the point that SOA and server virtualization - a la VMware - are not literally converging.  Well, sure, but is that really the point?  JavaScript and XML aren't literally converging, but together as AJAX they are powering a new breed of interactive Web Applications. 

The point really is in the motivation, the trend.  Joe's lead in sums it up pretty well:

"What's the common ground between SOA and virtualization? Both SOA and virtualization deal with abstraction of resources from across the network, and offer technology neutral interfaces removed from the original system."  [His emphasis]

Exactly.  And that's where we expect to see more and more attention being paid to database virtualization, aka EII.  Should anyone care that credit history is in one database, and contact info is in another, and recent purchases are in yet a third?  No.  All they want is the complete customer profile.  And if IT decides it wants to move from Oracle to MySQL, move the credit history table from SQL Server to Sybase, or run DB2 on VMware, who cares?  The nice thing about data virtualization is that it makes the data widely available to users and frees up IT to make optimal use of hardware, network bandwidth, and personnel.

With database virtualization, you can still send queries from an application or reporting tool, but the EII server is responsible for examining and routing all the queries - even caching the result locally if it expects 100 other users to want the same data (e.g. yesterday's sales).

Just as nice, if users decide they want the data as a Web Service, all the EII server admin has to do it publish the virtual view - customer profile, yesterday's sales - as a Web Service.  It takes about five minutes.

Todd Biske, whom Joe quotes in the entry, goes on to talk about how virtualization can be an effective tool for IT staff dealing with the complexity of SOA:

"If we don’t give the operational staff the tools they need to efficiently manage those resources, we’ll be in an even bigger mess."

We couldn't agree more, especially when it comes to data.  And no better tool exists for managing the complexity of databases in an SOA world than the database virtualization capabilities of EII. 

Calculating the ROI of EII: Application Maintenance (Example 5/5)

Because the EII layer can easily handle additions or changes to underlying data sources, organizations can focus on maintaining the EII layer and not the maintenance of every application and its connection. If a data source changes, the administrator only needs to update the mapping in the EII layer; all reports remain unchanged as they use the schema defined in the EII layer.

To figure out if this can save you money, think about this question: How much does it cost to maintain point-to-point connections among all your applications and data sources?

                                                                                         

	Example	Formula
Conventional   Approach
Cost to   develop a customized connector between two applications ($)	$10,000	A
Number of   connections required per year	25	B
Application   maintenance cost using conventional approach ($)	$250,000	A * B
Cost to change   a report because a data source changes ($)	$1,000	C
Number of   changes to reports required per year	200	D
Application   maintenance cost using conventional approach ($)	$200,000	C * D
EII   Approach
Cost to   develop a customized connector between two applications ($)	$0	E
Cost to change   a report because a data source changes ($)	$0	F
Cost to change   a data source mapping in the EII layer ($)	$100	G
Number of   changes to reports required per year	200	D
Application   maintenance cost using EII approach ($)	$20,000	G * D
Potential annual   cost savings from EII ($)	$430,000	(A * B) + (C *   D) - (G * D)

In this example, a company that eliminates customized programming for connecting applications and changing reports will save $430,000 by using EII.

• What value-add projects could your IT team complete if they are not changing report formats?
• How much more motivated would your developers be if they spent less time on software maintenance?

That concludes our five-part analysis.  Hopefully you've found some areas where there might be cost savings.  If you did and would like to take things further, here are some suggestions:

1. If you want to use your own numbers in the above examples, you can request the companion Excel spreadsheet from Ipedo.
2. If your business situation requires a customized financial model, one of our analysts can provide assistance with the process, or even build an interactive spreadsheet based on information that you provide.
3.  Once you complete your financial models and convince your team that EII looks good on paper, a logical next step is to develop a proof-of-concept application that demonstrates cost savings and/or productivity improvement in a controlled environment. Ipedo can help with this as part of the sales process.

Calculating the ROI of EII: Data Consolidation (Example 4/5)

Managing data costs a lot of money.  Just ask any CIO.  In any given company, there is a lot of money spent (and wasted) moving and copying data.  Enter EII.

Using EII to create a virtual, centralized database that cuts across disparate systems can reduce the cost of data maintenance, and eliminate the need for additional investment to store replicated data. Even with a conservative estimate of $50K per data mart ($25K hardware purchase and software licensing, $10K development cost, $5 software maintenance license, $10K personnel to manage the data mart), this gets expensive quickly.

The ROI-unlocking question: How much of your IT budget goes towards moving data around the enterprise and storing copies for different applications to use?

                             

Top-Down   Approach	Example	Formula
Annual IT budget   ($)	$10,000,000	A
% of resources   spent on data migration and replication (%)	25%	B
% improvement   with EII (%) (estimate)	10%	C
Potential   annual cost savings from EII ($)	$250,000	(A * B) * C

In this example, a company that spends 25% of its $10 million IT budget on data migration and data replication would save $250,000 (2.5% of overall IT budget) by using EII to complement its other data integration initiatives.

 

                       

Bottom-Up   Approach	Example	Formula
Cost to deploy   single new data mart ($)	$50,000	D
Number of data   marts deployed per year	10	E
Potential   annual cost savings from EII ($)	$500,000	D * E

In this example, a company that can eliminate 10 data marts per year by using EII will save $500,000.

And with all the focus on CDI and MDM these days, these upside benefits are sure to resonate:

• What is the value of having a “single version of the truth” based in the original data sources, rather than copies of data that become obsolete once the source changes
• How much time do you spend reconciling different versions of data residing in different locations?
• What politics are involved in getting permission to replicate data in your local data mart?
• How could you better deploy your IT resources if they did not have to manage a data mart?

That's it for today.  Next time, our final installment.

EII Haiku

In high technology, we often do things that have never been tried before.  I'm fairly certain this is one of them. 

Since I was recently accused of being too serious with my ROI posts, but not wanting to give way to the senseless prattling of many blogs, I offer you EII Haiku.  I like to think of them as lighthearted but deep.  Enjoy.

Queries federated
Distant rows and columns meet
My warehouse unscathed

Web Services, SOAP
Loosely coupled, lightly known
Join them, EII

Data is scattered
Metadata, SQL meet
All becomes as one

P.S. - If you would like to share your own, please feel free to do so in the Comments section.

Calculating the ROI of EII: Developer Productivity (Example 3/5)

With this entry, we'll move from the business side of the house to the development side of the house.  Yes, EII has something for everyone.

Recent IDC research reports that the average enterprise spends 40% of its IT budget on data integration (ETL, EII, EAI, etc.). Custom programming to connect disparate data sources is expensive to develop and maintain, especially in an environment of continual change. The unified abstraction layer that EII provides allows developers to write to one common interface to access multiple data sources, reducing the total development effort for new applications. The ability to write queries and not code allows departmental control of data integration, which saves the time and effort of going through a corporate IT group. These queries can be reused among different groups, further saving money.

So, What percentage of your IT budget do you spend on data integration tasks?   There are two ways to look at it.

                             

Top-Down Approach	Example	Formula
Annual IT budget ($)	$10,000,000	A
% spent on integration (%)   (ETL, EAI, EII, etc.)	40%	B
% improvement with EII (%)   (estimate)	10%	C
Potential annual cost savings   from EII ($)	$400,000	(A * B) * C

In this example, a company with a $10 million IT budget would save $400,000 annually (4% of overall IT budget) by including EII in their overall data integration strategy.

 

                                                           

Bottom-Up Approach	Example	Formula
Number of developers on   integration projects	20	A
Annual cost (including   benefits) of developers ($)	$120,000	B
% improvement with EII (%)   (estimate)	10%	C
Potential annual cost savings   from EII ($)	$240,000	(A * B) * C
Person-weeks for integration   projects over a year	1,000	D
Annual cost (including   benefits) of developers ($)	$120,000	E
% improvement with EII (%)   (estimate)	10%	F
Potential annual cost savings   from EII ($)	$240,000	(D * (E / 50)) * F

In these examples, a 20-person team would save $240,000 annually using EII as a part of their data integration projects.

And, as usual, some additional upside potential:

• What creative revenue generating or cost reducing functions will your team develop in the time they save by not rewriting software to accommodate changes in infrastructure or data format?
• How much more productive will your developers be if they focus on writing interesting applications rather than on more basic data integration tasks?

All for now.

Calculating the ROI of EII: Analyst Utilization (Example 2/5)

Here's one of my favorites, inspired by a conversation we had with a prospect back in 2005.  It's a great product management moment when the benefits of your product dovetail so nicely with a problem a customer has.

To wit, a vice president at a commercial bank that did not use EII told us that the analysts there spend about 60% of their time on data collection and only 40% on analysis. Moving from a 40%-60% split to a 90%-10% split means more than doubling (125%) individual analyst productivity. Thus, one analyst can do the work of slightly more than two. EII allows analysts to simplify the data collection process and spend more time producing useful, actionable information.

So, here's the question to unlock your potential ROI: How much time do your analysts spend searching for data?

                                   

	Example	Formula
Percentage of   time analysts spend on value-added activities pre-EII (%)	40%	A
Percentage of   time analysts spend on value-added activities post-EII (%)	90%	B
Number of   analysts on your team	10	C
Annual cost   (including benefits) of analyst ($)	$100,000	D
Potential annual   cost savings from EII ($)	$555,556	((B - A) / B)   * C * D

In this example, a team of 10 analysts does about 20 person-days of analysis per week pre-EII and 45 person-days per week post-EII, for a 125% improvement. If the amount of work was fixed at 20 person-days per week, a team of 5 analysts (actually 4.44) could do the job that previously required 10.

Additional Upside

Still need more ammo?  Think about quantifying these:

· What new discoveries will your analysts make when they have more time to focus on analysis?

· How many errors will be eliminated by reducing manual handling of data and what will the cost savings be of eliminating these errors?

· By spending more time using data rather than looking for it, how much more valuable will the analysts become to the organization?

· What is the effect on job satisfaction if people are doing what they enjoy, rather than just collecting data?

That's it for this installment.  Three more to go.  Stay tuned.  If you can't wait, check out the complete ROI guide, which can be found in Ipedo's EII Zone.