The present principles provide a method and an encoder for encoding an image frame. The method includes generating, with a processor, for each block in the image frame a prediction block based on at least a prediction method; generating a scaling factor and an offset for adjusting the prediction block; and transforming the prediction block into an adjusted prediction block based on the scaling factor and the offset. The encoder includes a processor configured to, for each block in the image frame, generate a prediction block based on at least a prediction method and generate a scaling factor and an offset for adjusting the prediction block and to transform the prediction block into an adjusted prediction block based on the scaling factor and the offset. The scaling factor and offset minimize error between the block and the adjusted prediction block.