The EII SOA Convergence

The convergence of EII and SOA is an interesting topic that has garnered some attention in recent times with SOA platform vendors promoting the idea of a Data Services layer as part of a service oriented architecture. For the uninitiated, EII stands for Enterprise Information Integration, a new class of Data Integration solutions that complements ETL. EII is rooted in Virtual Database technology and its accompanying disciplines like Distributed Query Processing. In a nutshell, the main purpose of an EII platform is to make a bunch of data sources look like a single database.

There are interesting parallels between SOA and EII. SOA broadly describes an architecture where loosely coupled services collaborate to implement a solution. SOA virtualizes the process space, shielding a application from details like where the services reside, how the processes are implemented, what platforms they reside etc. Technology components like UDDI for discovery, BPEL for process definition, SOAP for payload, internet standards like HTTP for transport provide a standards based foundation for SOA.

EII virtualizes the data space, shielding applications from details like the location of the data source, the data model and query mechanisms supported by the data source, protocols used to access them etc. EII platform vendors provide tools for metadata discovery and modeling the virtual database. EII’s standards foundation is made up of standard query languages like SQL and XQuery, which are used to access the virtual database through standard interfaces like ODBC, JDBC and XQJ.

So there is some conceptual harmony at an architectural level, but is there more? For some EII vendors, integration into an SOA means their product can consume web services and/or it can publish data via web services. For EII products that present a relational model for their virtual database, this means some mapping capability from XML to relational (tabular) structures and vice versa.

In the context of a Data Services Layer for an SOA, you want the EII platform to provide an abstraction (and integration) of all the data sources of interest to your services. You want your services to see a cloud of data, which can be queried using some standard language, standard interface, over standard transports. For XML Web Services, the natural data model for that data cloud will be an XML data model and the natural query language for that cloud will be XQuery.

In some cases, the results of querying a data services layer are not just meant to be passed on to another service, but are meant to be interpreted. In such cases, a developer might want to code with a more familiar interface like JDBC/ODBC and process result sets (rather than parse and process XML results). In this case, you want to see this data cloud have a relational schema and you want to query it with SQL. This is one reason the Dual Core implementation from Ipedo makes sense in these use cases. You can see your data work through your favorite colored glasses as either a bunch of relational tables and views, or as a bunch of XML trees.

EII as a data integration technology also has a major role in BI solutions. Allowing BI tools like Business Objects' WEBI and Crystal Reports seamlessly access web services is important for Operational Business Intelligence. With these EII capabilities BI tools can now report off of an instance of an SOA. Phil Wainewright has written an article at Loosely Coupled that describes how EII is bringing BI into the SOA world.

If your SOA deals with a single data source, you might not need much of a Data Services layer. JDBC as an API provides you enough of an abstraction in that case. If you need to deal with multiple data sources that are all describing different aspects of similar entities, you definitely should look into implementing a Data Services layer implemented using EII technology.