The Fundamentals of Heavy Tails: Properties, Emergence, and Estimation

By Jayakrishnan Nair, Adam Wierman, and Bert Zwart

Order from Amazon today!

Abstract

Heavy tails–extreme events or values more common than expected–emerge everywhere: the economy, natural events, and social and information networks are just a few examples. Yet after decades of progress, they are still treated as mysterious, surprising, and even controversial, primarily because the necessary mathematical models and statistical methods are not widely known. This book, for the first time, provides a rigorous introduction to heavy-tailed distributions accessible to anyone who knows elementary probability. It tackles and tames the zoo of terminology for models and properties, demystifying topics such as the generalized central limit theorem and regular variation. It tracks the natural emergence of heavy-tailed distributions from a wide variety of general processes, building intuition. And it reveals the controversy surrounding heavy tails to be the result of flawed statistics, then equips readers to identify and estimate with confidence.

A pre-publication version of the book can be downloaded here.

A set of tutorial slides covering some of the main ideas in the book.

Reviews

‘Heavy-tailed distributions are ubiquitous in many disciplines which use probabilistic models. The book by Nair, Wierman and Zwart is a superb introduction to the topic and presents fundamental principles in a rigorous yet accessible manner. It is a must-read for researchers interested in understanding heavy tails.’ — R. Srikant, University of Illinois at Urbana-Champaign

‘As one of the people who keeps discovering heavy tails in computer systems, I’m thrilled to see a book that delves into the deeper foundations behind these ubiquitous distributions. This beautifully written book is both mathematically precise and also full of intuitions and examples which make it accessible to newcomers in the field.’ — Mor Harchol-Balter, Carnegie Mellon University

‘The book provides a fresh look at heavy-tailed probability distributions on the real line and their role in applied probability. The authors show that these distributions appear via natural algebraic operations. Their approach, towards understanding the properties of these distributions, combines the key mathematical ideas alongside informal explanations. Physical intuition is also provided, for example, the ‘catastrophe/big jump principle’ for heavy-tailed distributions versus the ‘conspiracy principle’ for light-tailed ones. The book is designed to help the practitioner and includes many interesting examples and exercises that may help the reader to adjust and enjoy its content.’ — Sergey Foss, Heriot-Watt University

Table of contents

1. Introduction
   • Defining heavy-tailed distributions
   • Examples of heavy-tailed distributions
   • What’s next?

Part I: Properties

2. Scale invariance, power laws, and regular variation
   • Scale invariance and power laws
   • Approximate scale invariance and regular variation
   • Analytic properties of regularly varying functions
   • An example: Closure properties of regularly varying distributions
   • An example: Branching processes
3. Catastrophes, conspiracies, and subexponential distributions
   • Conspiracies and catastrophes
   • Subexponential distributions
   • An example: Random Sums
   • An example: Conspiracies and catastrophes in random walks
4. Residual lives, hazard rates, and long tails
   • Residual lives and hazard rates
   • Heavy tails and residual lives
   • Long-tailed distributions
   • An example: Random extrema

Part II: Emergence

6. Additive processes
   • The central limit theorem
   • Generalizing the central limit theorem
   • Understanding stable distributions
   • The generalized central limit theorem
   • A variation: The emergence of heavy-tails in random walks
7. Multiplicative processes
   • The multiplicative central limit theorem
   • Variations on multiplicative processes
   • An example: Preferential attachment and Yule processes
8. Extremal processes
   • A limit theorem for maxima
   • Understanding max-stable distributions
   • The extremal central limit theorem
   • An example: Extremes of random walks
   • A variation: The time between record breaking events

Part III: Estimation

9. Estimating power-law distributions: Listen to the body
   • Parametric estimation of power-laws using linear regression
   • Maximum likelihood estimation of power-law distributions
   • Properties of the maximum likelihood estimator
   • Visualizing the MLE via regression
   • A recipe for parametric estimation of power-law distributions
10. Estimating power-law tails: Let the tail do the talking
    • The failure of parametric estimation
    • The Hill estimator
    • Properties of the Hill estimator
    • The Hill plot
    • Beyond Hill and regular variation
    • Where does the tail begin?
    • Guidelines for estimating heavy-tailed phenomena