The scalloping loss of a window a(k) of length N is
The scalloping loss is the coherent gain of a frequency half a bin from a component on the discrete Fourier transform divided by the coherent gain of the window, as explained below.
Motivation for the scalloping loss
The following is a graph of the maximum value of the discrete Fourier transform of 40 points taken on simple sine waves with various frequencies between 20 Hz and 80 Hz, where the sampling frequency is 400 Hz.
The graph resembles a scallop (the decorative loops in curtains or tablecloths). The largest "loss", the places where the Fourier transform is least precise, occurs (approximately) in the middle between the transform components. At the sampling rate of 400 Hz, a transform of 40 points will have components 0 Hz, 20, Hz, 40, Hz, …, 380 Hz. The largest loss occurs at 30 Hz, 50 Hz, and so on. Thus, the scalloping loss is said to be the largest loss due to the frequency.
Scalloping loss and windows
Consider the following graph.
This is the discrete Fourier transform (DFT) of 20 components of the sine wave of 80 Hz, sampled at the sampling frequency 400 Hz. Rather than taking the DFT of the wave simply at the 20 DFT components (0 Hz, 20 Hz, 40 Hz, …, 380 Hz), however, the graph also shows the DFT at frequencies in between the DFT components. If the DFT was taken only at its components, then the graph will show only 0 dB at 80 Hz and practically very large negative numbers at all other components, since the DFT there is approximately zero. In between the components, however, there is spectral leakage. At half a component, at 0.5, we get -4.11 dB. This is the scalloping loss with no window (of the rectangular window).
Suppose that we apply the Hann window of 20 points to the sine wave of 80 Hz before computing the DFT. The result is shown below. The DFT after the Hann window is shown as a solid line. The DFT without the window is shown as a dashed line.
The DFT after the Hann window in this graph is adjusted for the coherent gain of the window. The scalloping loss with the Hann window is -1.28 dB. Thus, the scalloping loss is a measure of the shape of the main lobe of the DFT of the window.
This is, of course, a computation of the scalloping loss at half a component of the DFT after some randomly chosen frequency for a very short window. With the formula above and for larger windows, the scalloping loss of the Hann window would be approximately -1.42 dB (typically noted without the minus sign, as it is a "loss").
Scalloping loss for common windows
The following is the scalloping loss (in dB) of some common windows (with the window definitions on this site).