The phase vocoder can also be used to stretch or shrink time. The typical way of handling this involves
first setting the analysis and synthesis hop sizes to different values, as shown in Figure 20.14a. If the
synthesis (IFFT) hop size hs is larger than the analysis hop size ha then it means that we are taking a FFT
snapshot at a certain time, then pasting the IFFT result to a location later in the output timeline, forcing
the time to stretch out. The fact that the FFTs are done in chunks allows us to sew the new data to the old
data in an overlap-add fashion. A question you might have is “why bother with the FFT—why not just cut
and paste the time domain samples directly?” The answer lies with the phase of the transplanted signal:
you need to shift its phase to make it line up correctly with the output (this is the crux of the classic phase
vocoder algorithms where the two hop sizes are different). This is shown in Figure 20.14b and c where
the problem is evident when processing a single sinusoid input. If we don’t adjust the phase, the overlap
and mixing won’t work correctly—there will be phase cancellation or addition in the overlapped regions.
In order to accomplish the phase adjustment, we need to keep track of the running phase of each bin,
and estimate the bin frequency according to Equation 20.14. A new phase is calculated for each FFT bin,
then saved for the next frame. This maintains the horizontal phase coherence of the signal and ensures
that successive frames are glued together correctly—but it destroys the vertical phase coherence, which
is the phase relationship between the sinusoidal components within the FFT frame. Storing the old
phases and updating the new ones to maintain horizontal phase coherence is called phase propagation.
This brings up an interesting issue with time stretching and pitch bending with the FFT: if you try to
change the pitch but hold the time constant, you will necessarily need to alter the phase. The same is true
if you hold the frequency constant and try to adjust the time. In a sense, this is a game you can’t win.
FFT Processing 585
Figure 20.14: (a) Time stretching is done by transplanting the IFFT data to a location later in
the timeline. (b) Simply shifting the IFFT result without phase adjustment produces the incorrect
output signal (note black and white phase-dot locations). (c) Adjusting the phase of the bin gets
it to line up properly with the paste location.
For time shrinking you have to work in the opposite direction, transplanting the IFFT data to locations
earlier on the timeline, then overlap and mix the signals. A little thought will show that this isn’t going
to be easy because we are chewing our way through forward time on the analysis timeline. By the time
we are ready to make the transplant, the chance to do so has already passed by. To fix that, we would
need to pre-buffer the signal by enough time to accommodate the transplant-paste operation. For time
stretching, the opposite problem occurs in that we keep pasting information further and further down
the timeline in the output buffer, whose size must constantly grow to accept the new data. So we will
eventually run out of buffer space, or we will run out of time if we try to perform this in real time.
However, there are numerous offline approaches that work well—but of course not in real time.
For time shrinking you have to work in the opposite direction, transplanting the IFFT data to locations
earlier on the timeline, then overlap and mix the signals. A little thought will show that this isn’t going
to be easy because we are chewing our way through forward time on the analysis timeline. By the time
we are ready to make the transplant, the chance to do so has already passed by. To fix that, we would
need to pre-buffer the signal by enough time to accommodate the transplant-paste operation. For time
stretching, the opposite problem occurs in that we keep pasting information further and further down
the timeline in the output buffer, whose size must constantly grow to accept the new data. So we will
eventually run out of buffer space, or we will run out of time if we try to perform this in real time.
However, there are numerous offline approaches that work well—but of course not in real time.