XQuery for Information Integration

XQuery is a language for querying XML documents. That simple statement obscures some of the most exciting possibilities for XQuery. The XQuery query language is defined in terms of an abstract data model, and while that data model can be realized from XML documents, it can just as easily be realized from relational data sources, CSV files, EDI documents, or any other semi/structured documents. That opens up the possibility for an implementation to support querying any of these different kinds of data using XQuery.

So why is that important? Consider the parallels with EII. Information integration of necessity deals with a variety of data sources. Data models vary across information sources like relational database, web services, spread sheets and like. EII engines have the responsibility of making these disparate data sources look like one. That means, building a virtual data model that represents all these physical data sources, and allowing users to formulate queries over this virtual data model. The use case fits XQuery nicely since the virtual data model can be the XQuery data model, and instances of this data model can be constructed from almost any data source you are likely to encounter in your enterprise.

XQuery does have some disadvantages. It is a new language, in the final stages of becoming a W3C recommendation, and as such is not familiar to many developers and database administrators. Also, it is a bit more complex than SQL, what with its type system being aligned with the dreaded XML Schema spec. But in truth, even SQL can get quite complicated, except that the portions of the specification that a developer needs to be familiar with, to be able to do most productive work, is small. The same can be said for XQuery as well. This should become clear to most developers as XQuery books start inundating your favorite local bookstore.

Some EII implementations have chose to go with SQL as their query language. And given how often relational data sources are part of the mix of data sources being integrated, that makes sense. This is nothing new. In fact, IBM's Data Joiner had done this more than a decade ago. While I think any EII product worth its salt should support SQL as one of its query interfaces, I also believe that supporting non relational data sources should also be an important objective of EII platforms. Using SQL to query non-relational data sources is awkward and non-performant. Often, it is no longer distributed querying, as those engines suck in all that non-relational data and transform it into a bunch of tables. The ideal EII engine would support both SQL and XQuery and allow you to bridge the data model gap where required by allowing the use of SQL/XML constructs like XMLTABLE or something similar. That would meet your requirements if you are only integrating relational data, but also provide a performant solution when you want to add non-relational sources like web services to the mix.