Abstract:

In this paper, we propose to use transform-domain processing in the macro block coding mode decision of H.264/AVC such that an optimal mode decision is achieved with significantly reduced computational complexity. Specifically, we achieve the computational savings by calculating the distortion and the residual error in the transform-domain. We show that the distortion calculation can be performed efficiently in the transform-domain such that the inverse transform and reconstruction of the pixels can be omitted. We calculate the residual error in the transform-domain by taking advantage of the fact that the transform of several intra prediction signals (DC, Horizontal, and Vertical) can be very efficiently calculated, and their transform coefficients have few non-zero entries.