Flash Crowds
[Niven 1973 (doi)]
Larry Niven,
“Flash Crowd” in “Flight of the Horse”,
published by Ballantine Books, September 1973, pp. 99–164
ResiliNets Keywords: Flash Crowd, Definition
Keywords: flash crowd, transfer booths
Notes: The first publication to introduce flash crowds. It's a sci-fi novel.
[Chen-Heidemann 2005 (doi) .]
Xuan Chen, John Heidemann,
“Flash crowd mitigation via adaptive admission control based on application-level observations”,
ACM Transactions on Internet Technology (TOIT),
vol.5 iss.3, August 2005, pp. 532–569
ResiliNets Keywords: flash crowd, detection, mitigation
Keywords: flash crowd, detection, mitigation, dynamic limiting rate, application level, EWMA
Abstract: “We design an adaptive admission control mechanism, network early warning system (NEWS), to protect servers and networks from flash crowds and maintain high performance for end-users. NEWS detects flash crowds from performance degradation in responses and mitigates flash crowds by admitting incoming requests adaptively. We evaluate NEWS performance with both simulations and testbed experiments. We first investigate a network-limited scenarion in simulations. We find that NEWS detects flash crowds within 20 seconds. By discarding 32% of incoming requests, NEWS protects the target server and networks from overloading, reducing the response packet drop rate from 25% to 2%. For admitted requests, NEWS increases their response rate by two times. This performance is similar to the best static rate limiter deployed in the same scenario. We also investigate the impact of detection intervals on NEWS performance, showing it affects both detection delay and false alarm rate. We further consider a server memory-limited scenario in testbed experiments, confirming that NEWS is also effective in this case. We also examine the runtime cost of NEWS traffic monitoring in practice and find that it consumes little CPU time and relatively small memory. Finally, we show NEWS effectively protects bystander traffic from flash crowds.”
Notes: Seems the only paper studying flash crowd detection, works on application level.
[Jung-Krishnamurthy-Rabinovich-2002 (doi) .]
Jaeyeon Jung, Balachander Krishnamurthy, Michael Rabinovich,
“Flash Crowds and Denial of Service Attacks: Characterization and Implications for CDNs and Web Sites”,
in ACM Proceedings of the 11th international conference on World Wide Web ,
Hawaii, USA, 7-11 May 2002, pp. 532–569
ResiliNets Keywords: flash crowd, characterization, mitigation
Keywords: content distribution network performance, denial of service attack, flash crowd, web workload characterization
Abstract: “The paper studies two types of events that often overload Web sites to a point when their services are degraded or disrupted entirely - flash events (FEs) and denial of service attacks (DoS). The former are created by legitimate requests and the latter contain malicious requests whose goal is to subvert the normal operation of the site. We study the properties of both types of events with a special attention to characteristics that distinguish the two. Identifying these characteristics allows a formulation of a strategy for Web sites to quickly discard malicious requests. We also show that some content distribution networks (CDNs) may not provide the desired level of protection to Web sites against flash events. We therefore propose an enhancement to CDNs that offers better protection and use trace-driven simulations to study the effect of our enhancement on CDNs and Web sites.”
Notes: Seems the only paper trying to differentiate flash crowd from DDoS, very good study, but limited by the data then.
[Patel-Gupta 2004 (doi) .]
Jay A. Patel, Indranil Gupta,
“Overhaul: Extending HTTP to Combat Flash Crowds”,
International Workshop Web Content Caching and Distribution, ,
Beijing, China, 18-20 October 2004, pp. 532–569
ResiliNets Keywords: flash crowd, remediation
Keywords: HTTP, Web Browser, P2P, content breaking and distribution
Abstract: “The increasing use of the web for serving http content, for database transactions, etc. can place heavy stress on servers. Flash crowds can occur at a server when there is a burst of a large number of clients attempting to access the service, and an unprepared server site could be overloaded or become unavailable. This paper discusses an extension to the http protocol that allows graceful performance at web servers under flash crowds. We implement our modifications for the Apache web server, and call the new system as Overhaul. In Overhaul mode, a server copes with a stampede by offloading file transfer duties to the clients. Overhaul enables servers to chunk each requested document into small sections and distribute these partial documents to clients. The clients then share the sections amongst themselves to form a complete document. Overhaul enables a web server to remain responsive to further requests from other clients and at the same time helps conserve the amount of bandwidth utilized by a flash crowd. We present detailed experimental results comparing the benefits of using Overhaul under flash crowds and under normal operating situations. Although we restrict our studies to static content, the Overhaul architecture is applicable to improving web services in general.”
Notes: requires big changes on web browser and http protocol.
[Arlitt-Jin 1999 (doi) .]
Martin Arlitt, Tai Jin,
“A workload characterization study of the 1998 world cup web site”,
IEEE Network,
vol.14, iss.3, May/June, 2000, pp.30-37
ResiliNets Keywords: flash crowd, characterization
Keywords: Workload characterization, flash crowd, worldcup98
Abstract: “The increasing use of the web for serving http content, for database transactions, etc. can place heavy stress on servers. Flash crowds can occur at a server when there is a burst of a large number of clients attempting to access the service, and an unprepared server site could be overloaded or become unavailable. This paper discusses an extension to the http protocol that allows graceful performance at web servers under flash crowds. We implement our modifications for the Apache web server, and call the new system as Overhaul. In Overhaul mode, a server copes with a stampede by offloading file transfer duties to the clients. Overhaul enables servers to chunk each requested document into small sections and distribute these partial documents to clients. The clients then share the sections amongst themselves to form a complete document. Overhaul enables a web server to remain responsive to further requests from other clients and at the same time helps conserve the amount of bandwidth utilized by a flash crowd. We present detailed experimental results comparing the benefits of using Overhaul under flash crowds and under normal operating situations. Although we restrict our studies to static content, the Overhaul architecture is applicable to improving web services in general.”
Notes: the first thorough study on flash crowd based on data of typical scenario.
[Stading-Maniatis-Baker 2002 (doi) .]
Tyron Stading, Petros Maniatis, Mary Baker,
“Peer-to-Peer Caching Schemes to Address Flash Crowds”,
International Workshop on Peer-to-Peer Systems,
Cambridge, MA, USA, 7-8 March, 2002
ResiliNets Keywords: flash crowd, remediation
Keywords: P2P, flash crowd, caching, file distribution
Abstract: “Flash crowds can cripple a web site's performance. Since they are infrequent and unpredictable, these oods do not justify the cost of traditional commercial solutions. We describe Backslash, a collaborative web mirroring system run by a collective of web sites that wish to protect themselves from ash crowds. Backslash is built on a distributed hash table overlay and uses the structure of the overlay to cache aggressively a resource that experiences an uncharacteristically high request load. By redirecting requests for that resource uniformly to the created caches, Backslash helps alleviate the e�ects of ash crowds. We explore cache di�usion techniques for use in such a system and �nd that probabilistic forwarding improves load distribution albeit not dramatically1.”
Notes: P2P caching scheme to mitigation the flash crowd requests pressure on web server.
[Crovella-Bestavros 1997 (doi) .]
Mark E. Crovella, Azer Bestavros,
“Self-Similarity in World Wide Web Traffic: Evidence and Possible Causes”,
IEEE/ACM Transactions on Networking,
vol.5, iss.6, December 1997, pp.835-846
ResiliNets Keywords: Web traffic, characterization
Keywords: Web traffic, Self-similarity, Heavy tails distribution, file sizes
Abstract: “Recently, the notion of self-similarity has been shown to apply to wide-area and local-area network traffic. In this paper, we show evidence that the subset of network traffic that is due to WorldWideWeb (WWW) transfers can show characteristics that are consistent with self-similarity, and we present a hypothesized explanation for that self-similarity. Using a set of traces of actual user executions of NCSA Mosaic, we examine the dependence structure of WWW traffic. First, we show evidence that WWW traffic exhibits behavior that is consistent with self-similar traffic models. Then we show that the self-similarity in such traffic can be explained based on the underlying distributions of WWW document sizes, the effects of caching and user preference in file transfer, the effect of user “think time,” and the superimposition of many such transfers in a local-area network. To do this, we rely on empirically measured distributions both from client traces and from data independently collected at WWW servers.”
Notes: study and evidence of web traffic characterization, especially the file size distribution.
[Xie-Smith-Hutchison-Banfield-Leopold-Jabbar-Sterbenz-2008 (doi) .]
L. Xie, P. Smith, D. Hutchison, M. Banfield, H. Leopold, A. Jabbar, and J.P.G. Sterbenz,
“From Detection to Remediation: A Self-Organized System for Addressing Flash Crowd Problems”,
IEEE International Conference on Communications, May 2008
ResiliNets Keywords: flash crowd
Keywords: HTTP request; ISP network; Web server; admission control; detection mechanism; flash crowd event; network traffic; push-back mechanism; self-organized system; service request; Internet; telecommunication traffic;
Abstract: “A flash crowd event can be characterised by a dramatic increase in requests for a service over a relatively short period of time. Often, these events lead to a loss of service because of the saturation of the target server and associated network resources. This paper presents a set of mechanisms that can be used to make Web servers and associated resources more resilient to flash crowd events. Specifically, we present a novel admission control mechanism that uses a detection mechanism we developed in earlier work to adjust the admission rate of HTTP requests to a Web server. We demonstrate, via simulations, that the admission control mechanism can be used to protect a Web server from the effects of a flash crowd event, protect the traffic of other services that are hosted on the same network as a targeted Web server, and in combination with a push-back mechanism reduce the effect of flash crowd traffic on an ISP's network that is serving the Web server. The mechanisms presented here are exemplars that fit within a resilience strategy we are developing - D2R2+DR - which is summarised here.”
Notes:
