<i>WebCQ</i> -detecting and delivering information changes on the web

Abstract

WebCQ is a prototype system for large-scale Web information monitoring and delivery. It makes heavy use of the structure present i n h ypertext and the concept of continual queries. In this paper we discuss both mechanisms that We-bCQ uses to discover and detect changes to the World Wide Web (the Web) pages e ciently, and the methods to notify users of interesting changes with a personalized customization. The WebCQ system consists of four main components: a c hange detection robot that discovers and detects changes, a proxy cache service that reduces communication tra cs to the original information servers, a personalized presentation tool that highlights changes detected by W ebCQ sentinels, and a change noti cation service that delivers fresh information to the right users at the right time. A salient feature of our change detection robot is its ability to support various types of web page sentinels for detecting, presenting, and delivering interesting changes to web pages. This paper describes the WebCQ system with an emphasis on general issues in designing and engineering a large-scale information change monitoring system on the Web.