EII Exposed: The 10 Hidden Complexities
Director of Product Management Peter Spielvogel weighs in on a recent article. -- T.M.
I recently read Tieu Luu’s article on “Ten Things Your Enterprise Information Integration Vendor May Not Tell You: The hidden complexities of implementing an EII solution.” He has captured many of the key implementation barriers that can trip up organizations with unrealistic expectations about EII, often fed by vendors who are either naïve (best case) or deceptive (worst case). Real-world EII deployments are often quite different from the sanitized success stories that appear in most articles on the subject.
Most companies we talk to realize this.
Ipedo’s ‘Explain Plan’ shows every step of the integration process, including the SQL that is passed to each data source. This allows administrators to trace individual queries and streamline the debugging process.
Ipedo addresses this by allowing users to pass vendor-specific SQL to the back-end database. Allowing proprietary extensions to standard SQL balances the advantages of a common interface with the power to leverage platform-specific features.
5. "Of course we support
write operations, but you may not actually be able to use them."
Luu’s advice here is perfect: “understand what type of distributed transaction support your enterprise will need.” In some cases, read-only access will be fine; in others the ability to update a single data source will suffice. Some organizations will need full XA support. Ipedo’s vast transaction processing experience allows us to help our customers design the right system architecture that exploits the capabilities of the EII layer and complements it with robust application code to address idiosyncrasies of individual back-end systems.
Supporting distributed high-availability environments is one of the challenges of deploying EII in production environments. As with any database cluster, when the primary server goes down, the application will need to switch over to the failover server. Ipedo has used many techniques, both in the product and in the operating environment, to address this problem. The key is to start with the specific customer needs and use both the EII layer and operating environment to create a workable solution. For reporting applications that use EII for read-only access to data sources, this is less of an issue.
Performance depends on the back-end data sources, types of queries you want to run, and the optimization algorithms the EII layer uses. Ipedo looks at the source data to determine its join algorithms (cost-based optimization) rather than relying only on pre-set formulas (rule-based optimization). Ipedo also uses caching to speed performance. Of course, caching is subject to the caveats Luu lists in item #10.
An EII solution must fit into the customer’s existing security model and not attempt to reorganize the entire data security model around its own needs. For example, some organizations want the EII layer to store and use access to remote data sources while others expressly prohibit any persistent storage of login information. Ipedo allows granular security of data sources, tables, and views by individual users and groups at the EII layer as well as connections to single sign-on systems, proprietary authentication systems, and whatever other safeguards were in place prior to the EII deployment.
The issues Luu raises are associated with all transactional databases. So, for transactional EII, they remain a significant issue. For analysis, reporting, portal, and dashboard applications, which are less transactional, most companies will not have to worry about this issue. If you are planning on using your EII layer to support OLTP (few of our customers do), you should be careful in your application to use the correct isolation level.
While EII generally promises data on-demand, caching is commonly used to boost performance. In the EII context, caching refers to storing only query results, and not moving entire data sets. To avoid the synchronization problems that Luu describes, Ipedo uses intelligent caching that automatically invalidates itself at pre-defined intervals. In addition, Ipedo allows applications to force a query to go against the original data source if the application requires “fresh” data.
Tieu Luu’s article demonstrates the growing sophistication in the EII market and the consultants, analysts, and journalists that follow it. Customers should ask difficult questions and demand to see proof points in their own environments. Only then will EII become another standard information integration tool that organizations rely on to make their operation business decisions.
Ipedo offers several resources to help individuals become more informed about EII and how to use it to drive business results.
EII Guide - provides a pragmatic approach to implementing enterprise information integration in your enterprise. It includes sections on: EII technology, applications where EII can add value, suggestions for starting an EII project.
Determining Financial Returns on EII Projects - How to quantify the potential cost savings of an EII project based on your specific information integration needs.
The annual Ipedo EII Survey polls a broad cross section of IT professionals to determine how they are using EII and their implementation plans for the coming year. To weigh in with your opinion (through the end of the year), please visit the survey start page. After that, the same web link will provide you an opportunity to download the survey results .
Comments