# Inpainting

Inpainting is a conservation process where damaged, deteriorated, or missing parts of an artwork are filled in to present a complete image. It is used in both physical art restoration and digital image and video editing.

Inpainting is a conservation process where damaged, deteriorated, or missing parts of an artwork are filled in to present a complete image. This process is commonly used in image restoration. It can be applied to both physical and digital art mediums such as oil or acrylic paintings, chemical photographic prints, sculptures, or digital images and video. With its roots in physical artwork, such as painting and sculpture, traditional inpainting is performed by a trained art conservator who has carefully studied the artwork to determine the mediums and techniques used in the piece, potential risks of treatments, and ethical appropriateness of treatment.

In the context of [generative-ai](https://www.wikiprompt.org/wiki/generative-ai), inpainting has become a widely used technique where [machine-learning](https://www.wikiprompt.org/wiki/machine-learning) models, particularly [deep-learning](https://www.wikiprompt.org/wiki/deep-learning) [neural-network](https://www.wikiprompt.org/wiki/neural-network)s, fill in missing or corrupted regions of an image based on the surrounding context. This modern approach extends the historical practice into automated tools that can remove objects, restore old photographs, or edit visual content based on textual prompts.

## History

The modern use of inpainting can be traced back to Pietro Edwards (1744-1821), Director of the Restoration of the Public Pictures in Venice, Italy. Using a scientific approach, Edwards focused his restoration efforts on the intentions of the artist. It was during the 1930 International Conference for the Study of Scientific Methods for the Examination and Preservation of Works of Art that the modern approach to inpainting was established. Helmut Ruhemann (1891-1973), a German restorer and conservator, led the discussions on the use of inpainting in conservation. Ruhemann was a leading figure in modernizing restoration and conservation, and his greatest contribution was his insistence on following the methods of the original painter exactly and understanding the painter's artistic intention. After a career of over 40 years, he published *The Cleaning of Paintings: Problems & Potentialities* in 1968, describing methods where the surface of the fill should be slightly lower than the surrounding paint and the inpainting medium should not darken with age. Cesare Brandi (1906-1988) developed the *teoria del restauro*, an approach combining aesthetics and psychology, used primarily by Italian restorers, with terminology becoming widespread in the 1990s.

Technological advancements led to new applications. Since the mid-1990s, digital inpainting has evolved to include computer software relying on sophisticated algorithms to replace lost or corrupted parts of image data. This is more commonly known as image or video interpolation, a form of estimation.

## Ethics

To preserve the integrity of an original artwork, any inpainting technique or treatment applied to physical or digital work should be reversible or distinguishable from the original content. Prior to treatments, conservators proceed according to the American Institute of Conservation of Historical and Artistic Works. Ethical considerations rest mainly with authenticity, reversibility, and documentation. Any intervention to compensate for loss should be documented in treatment records and reports and should be detectable by common examination methods. Such compensation should be reversible and should not falsely modify the known aesthetic, conceptual, and physical characteristics of the cultural property, especially by removing or obscuring original material. New technologies and the aesthetic demand for perfect images without imperfections challenge conservators' ethical practices to protect the integrity of originals.

## Physical Inpainting

Inpainting is rooted in the conservation and restoration of paintings. It can aim to make a visual improvement to the artwork as a whole by repairing missing or damaged parts using methods and materials equivalent to the original artist's work. By studying the painting methods of various artists and the composition of paints used historically, conservators can restore works very closely to their original visual appearance. The picture as a whole determines how to fill in the gap. Ruhemann's techniques include procedures to preserve the quality of oil and tempera paintings.

## Digital Inpainting

Many programs can reconstruct missing or damaged areas of digital photographs and videos. Most widely known for use with digital images is Adobe Photoshop. Given the various abilities of the digital camera and the digitization of old photos, inpainting has become an automatic process. Techniques can be applied to object removal, text removal, and other automatic modifications of images and videos. In video special effects, inpainting is usually performed after video matting. It is also used in applications like image compression and super-resolution. In photography and cinema, it is used for film restoration to reverse, repair, or mitigate deterioration, such as physical damage like cracks, scratches, and dust spots, or chemical damage; this can include infrared cleaning. It can also remove red-eye, stamped dates, logos, or watermarks, and replace lost blocks in coding and transmission, for example in streaming video.

## Deep Learning Approaches

Deep learning neural network-based inpainting can be used for decensoring images. Deep image prior-based techniques can be used where a trained deep learning model is either unavailable or infeasible. Deep models for visual content generation, like text-to-image or text-to-video, learn complex priors over the distribution of visual content and can be used to inpaint missing parts. For example, videos can be separated into layers using a technique called omnimatte, which either pretrains an omnimatte model or uses an omnimatte-zero model without any training.

Three main groups of 2D image-inpainting algorithms exist in the literature: structural (or geometric) inpainting, texture inpainting, and a combination of the two. Structural methods focus on propagating geometric information, while texture methods replicate repetitive patterns.

## Use in Generative AI and Prompting

In [generative-ai](https://www.wikiprompt.org/wiki/generative-ai), inpainting is a core capability of many [large-language-model](https://www.wikiprompt.org/wiki/large-language-model)-based and diffusion-based image generation systems. Users provide a [prompt](https://www.wikiprompt.org/wiki/prompt) describing the desired content for a masked or selected region, and the model generates pixels that blend seamlessly with the surrounding image. This is distinct from outpainting, which extends the canvas beyond its original borders. Inpainting is commonly available through platforms like [replicate](https://www.wikiprompt.org/wiki/replicate), which host models for image editing, and is integrated into tools from companies such as [openai](https://www.wikiprompt.org/wiki/openai), [anthropic](https://www.wikiprompt.org/wiki/anthropic), and [google-deepmind](https://www.wikiprompt.org/wiki/google-deepmind) research efforts.

For prompt writers, effective inpainting requires specifying the subject, style, lighting, and perspective to match the existing image. Models often use [cross-attention](https://www.wikiprompt.org/wiki/cross-attention) mechanisms to align text prompts with image regions, and techniques like [top-k-sampling](https://www.wikiprompt.org/wiki/top-k-sampling) or [top-p-sampling](https://www.wikiprompt.org/wiki/top-p-sampling) may influence output diversity. The process typically involves an [encoder-decoder](https://www.wikiprompt.org/wiki/encoder-decoder) architecture, such as a [u-net](https://www.wikiprompt.org/wiki/u-net), which processes the masked image and generates the filled region.

## Practical Applications

Practical uses of inpainting include restoring historical photographs, removing unwanted objects or people from images, repairing damaged film footage, and creating seamless composites in advertising and entertainment. In medical imaging, inpainting can fill in missing data from scans. In autonomous driving, it can remove obstacles from training data to improve model robustness. The technique is also used in [audio-inpainting](https://www.wikiprompt.org/wiki/audio-inpainting) for filling gaps in sound recordings, though this is a separate domain from visual inpainting.

## Challenges and Limitations

Challenges in digital inpainting include maintaining consistency with the original image's lighting and texture, avoiding artifacts, and ensuring that generated content does not introduce false information, especially in documentary or forensic contexts. Deep learning models may hallucinate details that were not originally present, raising ethical concerns similar to those in physical conservation. As of the early 2020s, models continue to improve in handling complex scenes, but perfect reconstruction remains an open problem in [artificial-intelligence](https://www.wikiprompt.org/wiki/artificial-intelligence) research.

## External links

- [Wikipedia: Inpainting](https://en.wikipedia.org/wiki/Inpainting)

---
Source: https://www.wikiprompt.org/wiki/inpainting
License: CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0/)
Last updated: 2026-10-12T10:26:57.60484+00:00