johndcook-com

Online (one-pass) algorithms process data sequentially without storing the entire dataset. A canonical example is computing variance in a single pass, despite the formula requiring the mean—which itself depends on the full dataset—by using an algebraic rearrangement that updates running sums.

Kullback-Leibler divergence is not a metric due to asymmetry. Jeffreys divergence fixes symmetry by averaging K-L in both directions, and Jensen-Shannon divergence further uses the average distribution to produce a bounded measure whose square root is a true metric.

The post discusses how the Meta logo can be represented as a Besace curve, and explains the mathematical process for determining the curve parameters a and b to fit the logo.

This post examines how to compute the expected range of n samples drawn from a standard normal distribution N(0,1). The results are given in units of σ, so they can be scaled for other standard deviations.

A blog post examines the expected IQ spread among members of a jury, relating it to discussions about how large IQ differences can hinder communication between people.

The article discusses the Hilbert transform in the context of Fourier series, specifically how the transform of a function's Fourier series yields a related series. The author then explores interpreting this operation as an infinite matrix acting on Fourier coefficients, providing a linear algebra perspective on the Hilbert transform.

John D. Cook discusses notes based on Henry Baker's approach to implementing complex-variable functions using real-variable functions, decomposing f(x+iy) into real and imaginary parts u(x,y) and v(x,y).

The article explains how complex elementary functions, such as the sine and cosine of a complex number, can be computed using only real functions of real variables. While some functions require more complicated equations than the basic examples, this approach can be extended to all elementary functions.

The blog post introduces the concept of "couth" and "uncouth" function pairs, using circular and hyperbolic functions as examples. These functions are not naturally invertible due to many-to-one mapping, but mathematicians invert them by restricting domains, creating a pair of functions (a couth, invertible one and its uncouth counterpart).

The article explains that circular and hyperbolic functions are related by rotation in the complex plane. For instance, cosh(z) equals cos(iz), meaning a quarter-turn rotation transforms a circular function into its hyperbolic counterpart.

The article explores the complexity of defining √(z² − 1). While for non-negative real numbers x, √x is simply the non-negative square root, the definition becomes more subtle when considering complex numbers and how to choose a principal branch for expressions like √(z² − 1).

The article examines why a previously derived mathematical identity only holds for x > 1 and y > 1, using Mathematica to plot the condition and showing that the identity's validity is restricted to that domain.

Markov numbers are integer solutions to x² + y² + z² = 3xyz. Don Zagier studied them using the approximating equation x² + y² + z² = 3xyz + 4/9, which is equivalent to f(x) + f(y) = f(z) where f(t) = arccosh(3t/2).

The article describes how to reverse engineer the internal 128-bit state (four 32-bit integers) of the xorshift128 random number generator, following similar previous analyses of the Mersenne Twister and lehmer64 generators.

The blog post discusses how to initialize and print 128-bit integers in C, noting that while a 128-bit value can be initialized using a 64-bit value for simplicity, the language supports 128-bit integer types (such as __uint128_t in GCC) and also covers printing them using compiler-specific format specifiers or custom functions.

The post explains how to recover the internal state of the lehmer64 random number generator from its outputs, following a similar approach to a previous article on hacking the Mersenne Twister. The lehmer64 generator is noted for its simple implementation and high speed.

A blog post discusses the probability that a random matrix over a finite field is invertible, noting that this probability converges quickly as the matrix dimension n increases.

The post explores the concept of the inverse of a right shift operation on a binary sequence. While intuitively the inverse is a left shift, the author explains that this does not fully recover the original sequence because the bit that falls off the end is lost, making the operation not perfectly reversible without additional information.

The article discusses the probability that a random n × n binary matrix (filled with 0s and 1s) is invertible, exploring different approaches to calculating this probability depending on the underlying assumptions about the entries.

The post explains that bit-twiddling operations, such as the tempering step of the Mersenne Twister, can be expressed as matrix multiplication modulo 2. It notes that standard linear algebra theorems apply regardless of the field of scalars, not just over real or complex numbers.

The Mersenne Twister (MT) is a random number generator with strong statistical but weak cryptographic properties. This post demonstrates how to recover the internal state of an MT generator from its output using linear algebra, contrasting this approach with the usual bit twiddling method.

Curvature is conceptually simple but often difficult to compute. For a level set curve defined by f(x, y) = c, the formula for curvature κ can be complicated even when f is relatively simple.

The post discusses a triangular analog of the squircle, which is a unit circle in a p-norm (typically p around 4) that transitions from a Euclidean circle (p=2) to a square as p approaches infinity. It introduces three functions Li(x, y) whose level sets each form a smoothed polygon shape.

The blog post explores a mathematical shape that serves as a triangular analog to the squircle (a shape between a square and a circle). Unlike a simple triangle with rounded corners, this shape has continuously curved sides, directly extending the squircle concept to a triangle.

John D. Cook describes his approach to maintaining a consistent work environment across multiple computers by unifying configuration files, such as remapping keys so that the same key performs the same function on all devices, reducing unimportant differences between machines.

A blog post discusses category theory as a useful pattern language while cautioning against unrealistic expectations that it can deliver something for nothing. The author references a post by Qiaochu Yuan that echoes this sentiment about the overhyped promises sometimes attributed to category theory.

A blog post examines a claim that changing a hyphen to an en-dash in a PDF increases file size by about 10 bytes. The author initially suspected the reason might be that a hyphen is an ASCII character while an en-dash is not, which would affect encoding.

The blog post explores the mathematical shape defined by the equation (log x)² + (log y)² = 1, noting that while x² + y² = 1 produces a circle, taking the logarithm of the variables transforms the contour into a shape resembling a guitar pick. The post examines contours for different constant values on the right-hand side of the equation.

The article explores approximating even functions using powers of cosine, building on a previous post about approximating Bessel functions. A comment on Mathstodon pointed out that the earlier approximation relates to Bürmann's theorem, which the author discusses for constructing accurate approximations of even functions.

The article discusses three generalized derivative approaches for the ReLU function, which is not differentiable in the classical sense at zero. These generalizations are explored because ReLU is a common activation function in neural networks.