by Hari Balasubramanian

Optimization – the search for the best among many – is at the heart of the statistical and machine learning models that get used so extensively these days. Take the simple concept that underlies many of these models: fitting a mathematical curve to data points, better known as regression. In the simplest two-dimensional case, the curve is a line; in three dimensions, it is a plane, as seen in the figure. Among all possible planes – there are infinitely many of them, obtained by changing the angle and orientation: imagine rotating the plane every which way – we would like the one that passes ‘closest’ to as many of the data points (shown in red) as possible. Thus regression, often called the workhorse of machine learning, is really an optimization problem.
It turns out that optimization is fundamentally connected to even basic statistical quantities. Common statistics we we now reflexively use to summarize data – means, medians and percentiles – are themselves answers to certain optimization questions. We are not used to looking at them in this way.
Let’s take the average or sample mean, which, despite its obvious limitations, everyone turns to first. Suppose we have five numbers in a sample: 3,4,5,8 and 12. The sample mean is given by the straightforward calculation:
(3+4+5+8+12)/5 = 6.4
But there is another way to think of the sample mean: as the optimal answer to the following problem. We seek to find the x that minimizes (produces the lowest value of) the sum of the squared difference between x and each observation in the sample. Mathematically, we can write the function as:
(x-3)2 + (x-4)2 + (x-5)2 + (x-8)2 + (x-12)2
At what value of x does the function have its lowest value? In the figure below, the horizontal axis shows the values that x can take (I restricted myself to the range of the data points, 3 to 12). The vertical axis shows value of the sum of squares function corresponding to each value of x.

We see that the function follows a U-shape with the lowest/best value, x=6.4, occurring at the bottom of the U. Simple high school calculus – finding that point on the continuous curve at which the derivative is 0 – will yield the same answer. In fact, the formula for the mean, adding up all the sample values and dividing by the total number in the sample, can be derived using such calculus. Read more »


Why do people care about sport? With hundreds of millions of human beings (myself included) obsessively following the world cup that is being played out in Russia, it’s a good time to reflect once again on this perennially interesting question.



The ability to control electricity so that it arrives in measured doses where and when needed combined with simple steps to improve public sanitation, such as running water, indoor toilets and the removal of the uncountable tons of horse manure that marked major cities before the advent of the internal combustion engine that spurred roads and transportation networks that enabled frozen food to be enjoyed from coast to coast, wrought a step change in living standards unlikely to be repeated.
“It’s a long, long way from the Trump administration to an actual fascist dictatorship,” I said, “but it’s a straight line.”
It was all another day of Trump TV. Another day when all eyes were on Trump. Another day when headlines ran with his name splashed across the front page all over the world. Another day when memes were shared on Facebook and twitter. And another day people expressed feeling incredibly offended over and over again.
Give me a break I mutter. I text—and I text. Incessantly I text. Send money. Now. Send money. More money. You don’t reply. You will. It is Spring and I am young. Everyone around me on the beach is around my age or younger. We are young.
Political debates in Europe these days seem to have only one subject. At one point or another they all turn to the issue of migration, Islam, and a danger to “the West”, which are presented as essentially synonymous. Germany’s political future seems currently to 

