A novel programming interface (API) is introduced in this paper. The basic requirement is for multiple and asynchronous calling processes to capture a data stream without affecting each other. IP multicasting is the primary tool employed for this purpose. UDP is used at transport layer. The current implementation is based on the stream of web pages produced by a high performance web crawler. Along with the interface, a demo application (WordCount) is also implemented. Experiments have verified the robustness of the design, and preliminary measurement shows 1-3% data loss is observed, which is acceptable for many streaming data applications.
Reference
Related
Cited by
Get Citation
XIE Zheng-mao, ZHANG Fan, LI Xiao-ming. On the Design, Implementation and Application of a Multicast Interface for Streaming Data[J]. Journal of Integration Technology,2012,1(1):43-47