UltraSR: Spatial Encoding is a Missing Key for Implicit Image Function-based Arbitrary-Scale Super-Resolution

Abstract

The recent success of NeRF and other related implicit neural representation methods has opened a new path for continuous image representation, where pixel values no longer need to be looked up from stored discrete 2D arrays but can be inferred from neural network models on a continuous spatial domain. Although the recent work LIIF has demonstrated that such novel approaches can achieve good performance on the arbitrary-scale super-resolution task, their upscaled images frequently show structural distortion due to the inaccurate prediction of high-frequency textures. In this work, we propose UltraSR, a simple yet effective new network design based on implicit image functions in which we deeply integrated spatial coordinates and periodic encoding with the implicit neural representation. Through extensive experiments and ablation studies, we show that spatial encoding is a missing key toward the next-stage high-performing implicit image function. Our UltraSR sets new state-of-the-art performance on the DIV2K benchmark under all super-resolution scales compared to previous state-of-the-art methods. UltraSR also achieves superior performance on other standard benchmark datasets in which it outperforms prior works in almost all experiments.

References in corpus (4)

• NeRF++: Analyzing and Improving Neural Radiance Fields
• Implicit Neural Representations with Periodic Activation Functions
• SSD-GAN: Measuring the Realness in the Spatial and Spectral Domains
• NeRD: Neural Representation of Distribution for Medical Image Segmentation

Cited by in corpus (6)

• CIPS-3D: A 3D-Aware Generator of GANs Based on Conditionally-Independent Pixel Synthesis
• Continuous Remote Sensing Image Super-Resolution based on Context Interaction in Implicit Function Space
• Continuous Cross-resolution Remote Sensing Image Change Detection
• Spectral-wise Implicit Neural Representation for Hyperspectral Image Reconstruction
• Lightweight Stepless Super-Resolution of Remote Sensing Images via Saliency-Aware Dynamic Routing Strategy
• Arbitrary-Resolution and Arbitrary-Scale Face Super-Resolution with Implicit Representation Networks