"Data Driving SOA." Amen, Brother.

I was pleased to see Ronan Bradley's post from the other day 'Data: The Heart of SOA.'  I liked it so much I almost stole the title.  In it, he talks about how data issues really are part and parcel of any SOA project.  Amen, brother.

Once SOA projects get beyond simple 'Hello world' examples, data becomes critical.  First, you need something to pump into the service.  Then, of course, you need to make sure the data is correct.  So while much of the talk has been on service-enabling applications, this is all meaningless unless you can service-enable the data.  People in the data integration space like to refer to these as 'Data Web Services' as opposed to 'Application Web Services,' and often just 'Data Services' for short.

The keys here are data models and data access.  Data models allow you to know what the data looks like, what it means, and who owns it.  Data access is just what it says it is, but involves performance and security (and maybe even governance) issues.

I recently wrote about how EII is perfect for creating data services from existing sources - it provides the mediation layer to address performance, security and governance concerns.  To Ronan's point, Ipedo's product also allows the dynamic creation of data services from existing data models:

What's even cooler with Ipedo's product is that all of this can be scripted.  As my colleague Peter Spielvogel wrote about in The EII Files, Ipedo can automatically import data models from popular modeling tools like ERwin and ER/Studio.  So you can import all of the data models you need in a matter of minutes.  Once in Ipedo, you can script which Views you would like to be made available as Web Services, and voila!

I think we'll see more and more companies looking for ways to leverage their existing data models, which are typically relational, and transform them to Data Services. 

Dave Linthicum in his post "'SOA is Data' Really?" takes issue with Ronan's emphasis on data over process.  While I don't dispute that process and functions are the focus of an SOA application, a few things seem clear.  First, data services can support application services (get me customer data, get me risk profile, get me consolidated sales figures, etc.).  Second, many Web Services really are just data services, allowing easy access to information.  Third, seems to me that RESTful Web Services, being gets and puts, are predominantly Data Services by their nature.

That said, I also think you'll see more and more use of EII within ESBs, where ESBs will call a Data Service that is mediated by an EII product.  EII servers are just better suited to this task.  So the process- or application-oriented bus products can do what they do best.

Can I get an Amen?

Mind if I SELECT * Your SOA?

Has anyone done a study of how DBAs feel about SOA?  If so, send it my way.  I don't know of one.  But if I had to wager a guess, I'd say the consensus would be somewhere from uneasy to defensive.

It's not just that database schemas look quite different from XML schemas.  It's that all of this 'service enablement' opens up a Pandora's box of unoptimized access patterns.  For all this talk of business agility, there's the small matter of database stability.  Let's see, what rhymes with flexibility?  How about 'fragility'?

You see, today's databases are typically designed to support a specific application.  Or, commonly they are designed to support reporting or business intelligence applications.  Unlike the Web boom, where databases were designed to support large scale and sometimes even wacky applications, enterprise databases have years of modeling and tuning in them.  But now Web Services are taking off inside enterprises, and the SOA architects are eying the databases that belong to the data architects.  Imagine the first time a Web Services developer launches a SELECT * query on the wrong database.  The fur will fly.

But, as with many things in technology, the SOA horse is out of the barn, and while the data architects may accuse the SOA architects of putting the cart before the horse, something needs to be done.  Enter EII - or data services if you like.

As I wrote last year in Web Services Journal, EII is the perfect mediator.  An EII server a) understands SQL, b) understands data models, c) has built-in query optimization, d) has built-in security and governance, and e) can output Web Services.  So what's not to like?  You allow the EII server to mediate SQL queries to back end databases, complete with monitoring, caching, query tuning, and - importantly - adjustable thresholds on the allowable load on the back-end database.  The EII server can likewise service-enable these EII Views, publish them to an SOA registry, and monitor their accessibility and performance.  Everyone's happy.

What's even cooler with Ipedo's product is that all of this can be scripted.  As my colleague Peter Spielvogel wrote about in The EII Files, Ipedo can automatically import data models from popular modeling tools like ERwin and ER/Studio.  So you can import all of the data models you need in a matter of minutes.  Once in Ipedo, you can script which Views you would like to be made available as Web Services, and voila!  A little testing and tuning later, and your SOA developer and your DBA can leave work early to have a beer together.