New ROI Guide

Over the past two years, one of our consistently most popular pieces of collateral has been our ROI guide.  So I'm happy to announce that we've released a revised and refreshed version, with a renewed focus on information virtualization and some tweaks from customers and partners who have used it.

The guide contains five different quantitative models for calculating returns, so organizations can choose the one that most closely matches their technical or business situation. In addition, the guide includes additional qualitative methods that show how information virtualization can provide upside impact to a business.

To get your very own copy, just go here.

Fusing Together Oracle Products with Data and Service Virtualization

We did a demo recently that I thought was worth sharing.  With all the talk of Oracle Fusion in the press recently, I thought it was worth showing how simply and easily you can create custom combinations of data using Ipedo's data and service virtualization techniques.

In the demo we fuse together data from some Oracle 10g databases, with Salesforce.com (ex-Oracle CEO,  so we threw it in) and data moved from SQL Server by Oracle DI.  We put it all into Oracle BI Publisher, run Ipedo on Oracle AS, use another Oracle 10g database as a cache, and lastly move it into Oracle BPEL.  Whew!  Our professional services guys are now almost Oracle experts.

Here's the demo.  It's 15 minutes long, but broken into sections so you can navigate around quickly.

New Release of Ipedo's Virtual Integration Server

I'm very pleased to announce the new version of Ipedo XIP.  In 4.3, we've added a bunch of features to make the product even stronger.   I'm especially excited about the new incremental caching capabilities.

If you missed the announcement, here are the highlights:

The new enhancements are designed to increase both the performance and flexibility for large deployments, as well as accommodate the growing use of open source databases in the enterprise.  Whether creating data services in an SOA, or new reports in business intelligence applications, customers can access combinations of business information using a simple templated approach, with performance levels that are transparently maintained.  New features include:

 

-- Incremental Caching -- Ipedo XIP now allows for its cache to be
    refreshed incrementally, significantly reducing network traffic when a
    cache refresh is required.  Working automatically and in the background,
    the new incremental caching capability watches remote data sources for
    changes and automatically keeps the cache in sync.

-- Parameterized Views -- Parameterized views allow developers to
    encapsulate complex queries as parameterized templates, thereby avoiding
    construction of complex ad-hoc query strings at execution time. They are
    defined using a syntax common to prepared statements at design time, and
    parameter values are passed into the view at run time. Ipedo XIP performs
    all the optimizations on the resulting composite query tree to ensure
    efficient execution of the user query.

-- MySQL Enterprise Server Support -- Ipedo XIP can now work seamlessly
    with MySQL Enterprise Server, both as a data source and as a results cache.
    Extensive testing on data type interoperability and performance was done to
    ensure optimal deployment.

The new parameterized views are quite powerful.  MySQL support will be a hit in several of our accounts.  You can see the full release here. 

Enterprise Information Virtualization and the "Information Fabric"

I have to say, I am liking the new coinages from Mike Gilpen and Noel Yuhanna at Forrester: Enterprise Information Virtualization and the Information Fabric.

What's interesting is that last year they called it Enterprise Data Virtualization, but have correctly broadened the definition to include content, thus the switch to "information."  Here's how they described it in last year's report, which is the only publicly available summary:

Enterprises are facing the growing challenges of using disparate sources of data managed by different applications, including problems with data integration, security, performance, availability, and quality. Business users want fast, real-time, and reliable information to make business decisions, while IT wants to lower costs, minimize complexity, and improve operational efficiency. New technology is emerging that Forrester has coined "information fabric," a term defined as a virtualized data layer that integrates heterogeneous data and content repositories in real time. This technology is provided via middleware components that deliver quality information — "the truth" — when and where it's needed. (Here's a link to this year's EIV report - subscr. req.)

I like the idea of Enterprise Information Virtualization being the category, with Information Fabric being the catchy rubric for the concept.  That leaves the technologies for implementing EIV: query federation, virtual views, metadata and distributed caching, which fall in place nicely.  Forrester adds enterprise search to the list of technologies, which could make sense, depending on the application.

I've been talking about virtualization of data sources for quite a while in this blog, so it's nice to see Forrester's work. 

I think we still have a few loose ends to take care of, however.  Since EII and EIV are essentially the same thing (this is my take, not Forrester's), then what to do with the term EII? Also, Forrester also has Information as a Service (IaaS) in the mix, which I get, but to me sounds like something more like a hosted integration service.  Gartner talks about "data services" (which is about more than virtualization, and lacks content, hence the 'data') and the newer "Information Infrastructure."

Stepping back, I think it can be said that, despite the overabundance of terms, the analysts are converging on the idea that a new kind of infrastructure is needed that can deliver the right combinations of data and content to the right users on demand.  This infrastructure must, by its nature, be distributed.  Centralization is not an option.  It's also not about sticking information on a bus - this is for applications, not people.  Sounds like the perfect fit for a combination of query federation (structured and unstructured), virtual views, metadata and caching, which is exactly what Ipedo's got - no matter what you call it.

Fusing SQL and XQuery to Combine XML and Relational Data for Reporting

There is a lot of talk these days about harnessing content for use in business intelligence.  This is something we've been enabling for some time at Ipedo, but it seems like more and more people are having the need/becoming aware of the possibilities.

Generally speaking, there are two types of content people are interested in - search results from textual documents and access to documents represented as XML.  I know, I know, there are more than that.  But if you look at what people are actually using, I contend they are looking to utilize search results on Web and/or Microsoft Office documents; or they want to search and extract information from semi-structured XML documents, which are usually forms, industry schemas (MISMO, FpML, ACORD et al), or transformed Microsoft Word or Excel.

So I want to show you something really cool.  Take a look at the picture below, which shows how you can use SQL to query semi-structured XML documents.  Perfect for BI.

 

The magic is our Dual-Core Query Engine that marries SQL and XQuery.  We have something called XML Tables that make XML documents look like relational tables.  So you can query a bunch of XML documents using SQL and manipulate it in a reporting tool like Crystal Reports.  Or you can query a combination of relational and XML sources using SQL and manipulate in Crystal et al.

If you want to see something live, you can also see how this same technology works against combinations of relational and Web Services data here.

Balancing Performance in an EII Server

After many years of explaining all of Ipedo's performance optimization functionality bullet by bullet, I finally put it into a picture.   The idea is that you always need to balance the user response time with the impact on the back end data sources.

So, that's all I'll say.  See if this picture does the trick in explaining it.

 

Teasing Out 'Integration 2.0'

Those of you who have read my stuff know that this '2.0' bug has been knocking around my brain recently.  It's not so much a cheap desire to gather more buzz for Ipedo (although, by way of disclaimer, I should point out that this is what I am paid to do for my day job), but rather something that keeps nagging me to ask if there's not something to it.

Tim O'Reilly is most famous for declaring Web 2.0 in his article What Is Web 2.0: Design Patterns and Business Models for the Next Generation of Software.   His article tried to define what was a very elusive and potentially out of control concept, on the way to becoming a meaningless buzzword.  Along the way he discovered a set of design patterns that were undeniable.  Not that everything was totally new - there were people doing 2.0 stuff in the 1.0 era - but overall there was a pattern.  I liked the following way of describing Web 2.0:

You can visualize Web 2.0 as a set of principles and practices that tie together a veritable solar system of sites that demonstrate some or all of those principles, at a varying distance from that core.

We are on the verge of 2.0 everything.  I see many of these as hollow marketing efforts.  On the other hand, I see 2.0 as a challenge - a new way of thinking.   Our new 'ng' or '.next.' 

So far, the integration vendors have not really done anything to define what the next generation of integration should look like.  Saying that it's SOA is a punt.  That's not an answer.  The question is whether there is something new, something that can be hosted, a service, collaborative, open that will cost less and yield more.

Stay tuned here as we ponder.

EII, SOA and Virtualization: Roundtable Podcast from Dana Gardner

I've been waiting to use the category tags EII, SOA and Virtualization together for some time now.  It's something readers here have heard me talk about before - the nexus of the SOA trend and the virtualization trend, and how EII nicely joins the two together when it comes to data.

The podcast is a lively roundtable format with Steve Garone, Joe McKendrick, Neil Macehiter, Tony Baer, Jim Kobielus and Dana Gardner - some pretty knowledgeable folks.  Though the podcast starts out with a debate on SOA stack vs. best of breed implementations, it gets more interesting toward the end when they delve into other subjects.  I must say, it's nice to see EII come up in the context of a general SOA discussion.  They definitely get the importance of data virtualization in an SOA in general, and how EII can handle data and master data nicely.

Listen to it here.

A Discussion on Virtualization with Stephen Hayward

Had a good conversation with Stephen Hayward of ProjectX around data virtualization.  Stephen did a good job summarizing the discussion and providing some background, which you can see in this entry  Or go right to the podcast.

What Does Data Temperature Mean, and What's It Got to do with Data Virtualization?

I was reading a post from Dan Linstedt from last year on the temperature of data.  This is a very interesting concept, and so intuitive I wondered why I had never heard it before.

You should read the whole of Temperature of Data for RDBMS, and DW 2.0, but here are Dan's definitions, summarized:

• Hot = accessed all the time, or extremely important data requiring sub-second response times
• Medium = is data accessed most of the time, but where response times can be anywhere from 1 second to 10 or 15 seconds
• Lukewarm Data = data that is accessed rarely; rarely may be (for example) once every 30 minutes or twice every 4 hours. 
• Cold Data = historical context that is hardly ever accessed, but when requested, must have a response time equal to that of a couple minutes

What's really interesting about these ratings is that they can serve as a guide to proper data placement, and hence can inform how one should use data virtualization. 

If you have data that's Hot, it should either be cached, or the application should hit the source directly.  If you have data that is Medium, then it need not be cached, since query federation can easily hit these time frames.  Likewise with Lukewarm data.  Cold Data might even be put off to tape or cheap storage with a low query priority and a never-cache setting.

As Dan points out, definitions will vary by company (and maybe even within companies), but the framework can be universally applied.  We'll continue to dig into this concept and show how it could work in practice.  Think about it, and I'm sure you'll warm up to the idea.