Peer-to-peer (P2P) systems have been widely deployed and used to provide different services, such as file sharing, video streaming, and voice-over-IP. These applications have contributed the largest portion of Internet traffic, and have unnecessarily wasted a significant amount of precious Internet bandwidth due to the mutual-blindness between the application layer (overlay) and its network layer (underlay). The lack of communication between these two layers has caused significantly inefficient Internet resource utilization. How to utilize Internet underlay information to optimize the quality of these overlay applications as well as Internet resource utilization is a fundamental and challenging issue that must be addressed for building Internet systems and applications on top of it. In this Ph.D. dissertation, we first investigate the frequently occurred Internet routing instabilities that can greatly degrade overlay application quality, and analyze their root causes. We propose several new techniques and their system implementation to discover routing instabilities by solely utilizing available user-level utilities. Our techniques rely on traceroute probes that can be easily launched from end hosts. We have implemented our techniques in a system facility called NetRecce, and have extensively tested and evaluated its performance by analyzing the routing instabilities for three top-tier ISPs over a 50-day period. We show that NetRecce is highly effective in discovering a large fraction of routing instabilities.
We then conduct intensive Internet measurement experiments to confirm the benefits gained by peer relays in VoIP, and to investigate the performance of the Skype system. We observe that Skype peer relay selection is suboptimal, takes long time, and generates a large amount of unnecessarily probing traffic. Our further analysis shows that two main reasons cause these problems. First, the peer selections do not take Autonomous System (AS) topology into consideration, and second, the complex communication relationships among peers are not well utilized. Motivated by our measurements and analysis, we propose an AS-aware peer-relay protocol called ASAP. Our intensive evaluation by trace-driven simulation shows ASAP is highly effective and easy to implement on the Internet for building large and scalable VoIP systems.
Finally, aiming at building an infrastructure-independent user-level facility, we present our design, implementation, and evaluation of a topology-aware BT system, called TopBT, to significantly improve the Internet resource utilization without degrading user downloading performance. A TopBT client actively discovers network proximities (to connected peers), and uses both proximities and transmission rates to maintain fast downloading while reducing the transmitting distance of the BT traffic and thus the Internet traffic. We have implemented TopBT based on widely used open-source BT client code base, and made the software publicly available. By deploying TopBT and other BitTorrent clients on hundreds of Internet hosts, we show that on average TopBT can save a significant amount of download traffic while achieving faster download speed compared to several prevalent BT clients. TopBT has attracted a large number of downloads for a wide usage across the world. It has also been integrated into a major BT system for testing and a potential adoption in mainstream BT systems.