EII Exposed: The 10 Hidden Complexities

Director of Product Management Peter Spielvogel weighs in on a recent article. -- T.M.

I recently read Tieu Luu’s article on “Ten Things Your Enterprise Information Integration Vendor May Not Tell You: The hidden complexities of implementing an EII solution.” He has captured many of the key implementation barriers that can trip up organizations with unrealistic expectations about EII, often fed by vendors who are either naïve (best case) or deceptive (worst case). Real-world EII deployments are often quite different from the sanitized success stories that appear in most articles on the subject.

In his summary, he wisely suggests that companies considering EII take the time to thoroughly understand their current data environment, and application’s needs. I would add that they should work with their EII vendor(s) to create a proof of concept application that connects to a representative set of data sources, running queries comparable in complexity to what they would expect in a production environment.

While I’m sure some of the “surprises” he discusses do occur, in this blog I will address how Ipedo handles these challenging real-world EII implementation issues.

1. "It's probably a lot harder to implement an EII solution in your enterprise than our examples and case studies would tell you."

Most companies we talk to realize this.

For example, one organization’s VP of Development placed a prominent note on their proposed architecture diagram: “When source data is dirty and/or inconsistent, data access layer construction will encounter the same class of problems as the ETL steps of a data warehouse. There is no magic.” He clearly understands where EII will fit in his overall data integration solution.

Another customer originally expected an out-of the box solution to the foreign key across different database issue. After a brief discussion, they agreed that mapping these relationships manually made sense, especially since they only had to perform that process once.

2. "We isolate you from all of the complexities of accessing multiple data sources... including the ability to debug."

Ipedo’s ‘Explain Plan’ shows every step of the integration process, including the SQL that is passed to each data source. This allows administrators to trace individual queries and streamline the debugging process.

3. "Be prepared to lose some platform-specific functionality and flexibility."

Ipedo addresses this by allowing users to pass vendor-specific SQL to the back-end database. Allowing proprietary extensions to standard SQL balances the advantages of a common interface with the power to leverage platform-specific features.

4. "Sure we support transactions, but only if they go through a single data source."

5. "Of course we support write operations, but you may not actually be able to use them."

Luu’s advice here is perfect: “understand what type of distributed transaction support your enterprise will need.” In some cases, read-only access will be fine; in others the ability to update a single data source will suffice. Some organizations will need full XA support. Ipedo’s vast transaction processing experience allows us to help our customers design the right system architecture that exploits the capabilities of the EII layer and complements it with robust application code to address idiosyncrasies of individual back-end systems.

6. "Our product can be clustered, but that doesn't guarantee you'll have proper backups."

Supporting distributed high-availability environments is one of the challenges of deploying EII in production environments. As with any database cluster, when the primary server goes down, the application will need to switch over to the failover server. Ipedo has used many techniques, both in the product and in the operating environment, to address this problem. The key is to start with the specific customer needs and use both the EII layer and operating environment to create a workable solution. For reporting applications that use EII for read-only access to data sources, this is less of an issue.

7. "Tuning this thing for performance can be a nightmare."

Performance depends on the back-end data sources, types of queries you want to run, and the optimization algorithms the EII layer uses. Ipedo looks at the source data to determine its join algorithms (cost-based optimization) rather than relying only on pre-set formulas (rule-based optimization). Ipedo also uses caching to speed performance. Of course, caching is subject to the caveats Luu lists in item #10.

8. "Better make sure you set aside some time and money to build the proper security infrastructure if you really want to use this thing."

An EII solution must fit into the customer’s existing security model and not attempt to reorganize the entire data security model around its own needs. For example, some organizations want the EII layer to store and use access to remote data sources while others expressly prohibit any persistent storage of login information. Ipedo allows granular security of data sources, tables, and views by individual users and groups at the EII layer as well as connections to single sign-on systems, proprietary authentication systems, and whatever other safeguards were in place prior to the EII deployment.

9. "Say goodbye to the concurrent access behavior that you've been used to."

The issues Luu raises are associated with all transactional databases. So, for transactional EII, they remain a significant issue. For analysis, reporting, portal, and dashboard applications, which are less transactional, most companies will not have to worry about this issue. If you are planning on using your EII layer to support OLTP (few of our customers do), you should be careful in your application to use the correct isolation level.

10. "Unlike ETL products, we offer a federated approach that doesn't store any data, sort of..."

While EII generally promises data on-demand, caching is commonly used to boost performance. In the EII context, caching refers to storing only query results, and not moving entire data sets. To avoid the synchronization problems that Luu describes, Ipedo uses intelligent caching that automatically invalidates itself at pre-defined intervals. In addition, Ipedo allows applications to force a query to go against the original data source if the application requires “fresh” data.

Summary

Tieu Luu’s article demonstrates the growing sophistication in the EII market and the consultants, analysts, and journalists that follow it. Customers should ask difficult questions and demand to see proof points in their own environments. Only then will EII become another standard information integration tool that organizations rely on to make their operation business decisions.

Additional EII Resources

Ipedo offers several resources to help individuals become more informed about EII and how to use it to drive business results.

EII Guide - provides a pragmatic approach to implementing enterprise information integration in your enterprise. It includes sections on: EII technology, applications where EII can add value, suggestions for starting an EII project.

Determining Financial Returns on EII Projects - How to quantify the potential cost savings of an EII project based on your specific information integration needs.

The annual Ipedo EII Survey polls a broad cross section of IT professionals to determine how they are using EII and their implementation plans for the coming year. To weigh in with your opinion (through the end of the year), please visit the survey start page. After that, the same web link will provide you an opportunity to download the survey results .

Comments

[TicginiaNedia]

Anyone and every one should own at least one LED flashlight. More than one, even. You unequivocally beggary story in every household in suit of a blackout. How would you be navigating your fall down about the power box or even in sight of the house to determine broken if it's reasonable your house coating this climax or if your neighbors are in the condition too? I from a adherent who even keeps one LED flashlight per margin in the house. That includes the bedrooms, living room, larder, storeroom, and even the bathrooms! I put one's trust in she has as the case may be 10 of such an LED flashlight. Another important throw away victim of an LED flashlight is while you're traveling. You should always transport an LED flashlight with you in case your tourist house or wherever you are encounters a blackout. Moreover, shining the magnificent LED flashlight at one's desire be masterly to come down with people's notoriety to criticize to your release, wherever you are (unless you're beneath water, at which your LED flashlight probably won't be functioning).



So why an LED flashlight and not one of those general torchlights? Fount, honest like how the transportation lights these days don't reach flawed as much as in the old times, so will the lifespan of your LED flashlight. It is known to last you spirit longer than the flashlights of the past.



I bought the UltraFire WF-502B SSC-P7-CSXO 3-Mode 900-Lumen Respect LED Flashlight with Shorten (1*18650/1*17670) from the Ultrafire Lights conditioned by trust in on DealtoWorld.com, where I regularly do some online shopping. DealtoWorld.com carries varied gadgets and tools that fetch everyday elasticity much easier. It sure beats booming for everyone the marts and just discovering things at random and buying them almost on impulse. At least while I'm doing online shopping at DealtoWorld.com, I can forthwith research on the product and the power of it, and even-tempered announce reviews of it preceding I earn that buy decision. It sure makes me experience like I'm shopping crumble smarter than up front! Uninterrupted my husband is blithesome for me that I don't proper against as a service to clothes, but know how to get such advantageous tools fixed the internet! It's avenue more convenient, and you don't have to be wonderful savvy to remember how to do it. After buying this LED flashlight, I regular bought some birthday gifts for friends. The gifts are mostly functional things that they can press into service, and I also bought another LED flashlight as my shelter who still uses candles during a power outage.



The LED flashlight I bought from DealtoWorld.com allows me to on from three modes. There's strong, low, and strobe. Exalted is in the service of when I need loads of daybreak, like during a blackout, for instance. Switching it to squat enables me to protect batteries or when I don't call too much gleam, like when I trouble to descry the gas meter, on instance. Strobe is for when I desideratum some rescuer's distinction or to disorient an thief or barbaric animal.



Decent form month I was stuck in a jammed elevator and rogue was I tickled to partake of my trusty LED flashlight with me so I didn't sense so scared. As you can envisage from the pictures, the LED flashlight produces a to a certain extent intense slight, to sunrise up the zone when you most need it.

[Illighhip]

paraprotex