Kosmix is the search engine that produces “topic pages” on millions of subjects. Kosmix creates these topic pages by searching APIs of deep web sources (in real time). In other words, Kosmix relies heavily on federated search for the content of their topic pages. (Actually, Kosmix combines federated search with crawling technology. More about this later in this article.)
Kosmix co-founder, Anand Rajaraman, recently spoke at PARC, the prestigious Palo Alto Research Center. Rajaraman’s talk: “What lies beneath: harnessing the deep web.” A video of the hour-long talk is available at the PARC web-site. The slides are available at the Kosmix Blog.
Rajaraman has very impressive credentials. He is also co-founder of the VC firm Cambrian Ventures, he teaches a class for Stanford’s Computer Science department, and he is former Director of Technology at Amazon.com where:
he was responsible for technology strategy. Anand helped launch the transformation of Amazon.com from a retailer into a retail platform, enabling third-party retailers to sell on Amazon.com’s website. Third-party transactions now account for over 25% of all US transactions, and represent Amazon’s fastest-growing and most profitable business segment.
Here’s a nice video interview. John Meyer of Silicon Prairie News interviews Will Bushee, VP of Development of BrightPlanet. I also found a link to a recent text interview with Mr. Bushee by Stephen Arnold.
BrightPlanet is a Deep Web search company. They’re best known for the seminal work that federated search luminary and BrightPlanet co-founder Michael Bergman did in sizing the Deep Web. The video provides an understandable introduction to the Deep Web plus discusses, at a very high level, what BrightPlanet does.
OSTI (the US Department of Energy Office of Scientific and Technical Information) recently joined YouTube and released a couple of short videos pertaining to the Deep Web. Disclosure: I consult for OSTI.
Deep Web Video
To make the web work better for science, OSTI has developed state-of-the-art technologies and services including a deep web search capability. The deep web includes content in searchable databases available to web users but not accessible by popular search engines, such as Google. This video provides an introduction to the deep web search engine.
I recently discovered a video at the Federal Library & Information Center Committee (FLICC) web-site titled Federated Search Technologies. The video (divided into two pieces) is over two hours long. It includes three presentations from members of organizations who have implemented federated search solutions:
• Betty Day, University of Maryland—MetaLib (ExLibris)
• Karen Huffman, National Geographic Society—AGent (AutoGraphics)
• Tamas Doszkocs, National Library of Medicine—ToxSeek
Abe discovered “Searching the Deep Web” on YouTube yesterday. This is a nice professionally produced 6-minute introduction to the Deep Web made by the US Department of Energy Office of Scientific and Technical Information (OSTI). Deep Web Technologies is mentioned in the video as DWT has created the search technology for a number of major OSTI applications.
What’s very cool about this video hitting You Tube is that Abe and I think of You Tube as hosting very mainstream videos. We like the idea of the public being exposed to federated search in such a venue.
For your viewing pleasure, here’s the video. Make yourself some popcorn, relax, and enjoy the show!
