Clustering patches of a degraded version of an image is such that the clusters are based on errors computed between patches of the degraded version processed with upgrade functions associated respectively with the clusters and patches of a full graded version of the image corresponding respectively to the patches of the degraded version. The source image is thus used to determine clusters for the image to restore. A restoration framework based on a clustering according to the present principles is also disclosed.