Jul
I’m new to the term “data federation.” How about you?
Michael Bergman, federated search luminary, just wrote on the subject, preferring the term “data mixing.” He explains the concept:
What is Data Mixing and Why is it So Hard?
As a new term there is no “official” definition of data mixing. However, I think we can consider it as generally equivalent to the older data federation concept.
Data federation is the bringing together of data from heterogeneous and often physically distributed data sources into a single, coherent view. Sometimes this is the result of searching across multiple sources, in which case it is called federated search. But it is not limited to search. Data federation is a key concept in business intelligence and data warehousing and a driver behind master data management (MDM).
Bergman explains that data federation was a hot research topic in the 1980s. Computers of different hardware, operating systems, databases, and other software were proliferating. Today’s robust and ubiquitous networking protocols were far from mature then. There were no dominant standards for data representation in the 80′s. Today we take interoperability for granted; if two systems don’t speak to one another directly we expect that someone has already developed software to bridge the gap. The whole Internet speaks TCP/IP. XML is everywhere.
So, we can say that it took the solving of some major data federation problems to lay the foundation for the Internet and the Web that we enjoy today.
Bergman further explains that the next major challenge is in semantics:
The Internet and its TCP/IP and Web HTTP protocols and XML standards in particular, have been major contributors to overcoming respective physical and syntactical and data exchange heterogeneities. The current challenge is to resolve differences in meaning, or semantics, between disparate data sources. Your “glad” may be someone else’s “happy” and you may organize the world into countries while others organize by regions or cultures.
I recommend Bergman’s article, especially if you have an interest in the Semantic Web. It’s moderately technical but it’s worth the read to understand where data federation fits into the Semantic Web.
Tags: federated search
One Response to "Michael Bergman on data federation"
July 7th, 2009 at 10:27 pm
Data federation and data mixing is very hot these days. Since the advent of RSS (essentially XML feeds of the data on a web page or database), people have focused on turning everything into XML. And if a website doesn’t provide it, solutions like Dapper.net come along that allow anyone to create an RSS feed visually from any site — just point and click. Also popular is the concept of Mashups, which is combining multiple RSS or XML feeds into a new UI or data source, and even creating an RSS feed from that source. Dapper uses what amount to a ‘screen scraper’ which is a very negative word, or has been in the past, due to instability and lawsuits over copyrights, which is the real blocking point to a true open Internet.