The implementation of a Null-Text Inversion Pipeline #6313
Description
Model/Pipeline/Scheduler description
NullTextInversionPipeline
A pipeline implementation of Null-text Inversion for Editing Real Images using Guided Diffusion Models.
As I did not find it in Diffusers, I'd like to add this feature to Diffusers pipeline.
The NullTextInversionPipeline allows users to conduct DDIM-Inversion, Null-text Optimization, and DDIM reconstruction via w, w/o Null-text Optimization.
No prompt-to-prompt code is implemented as there is a Prompt2PromptPipeline.
https://github.com/huggingface/diffusers/blob/2d43094ffc9b1ee377651c6c8a358c81f0c96005/examples/community/pipeline_prompt2prompt.py#L44
Open source status
- The model implementation is available.
- The model weights are available (Only relevant if addition is not a scheduler).
Provide useful links for the implementation
The official implementation from Google is
https://github.com/google/prompt-to-prompt/blob/main/null_text_w_ptp.ipynb
Paper: Null-text Inversion for Editing Real Images using Guided Diffusion Models
http://openaccess.thecvf.com/content/CVPR2023/html/Mokady_NULL-Text_Inversion_for_Editing_Real_Images_Using_Guided_Diffusion_Models_CVPR_2023_paper.html
Null-text Inversion requires no additional model weights. The users may use personal photos to conduct the inversion process.
Null-text inversion enables intuitive text-based editing of real images with the Stable Diffusion model. We use an initial DDIM inversion as an anchor for our optimization which only tunes the null-text embedding used in classifier-free guidance.
@amirhertz @dmarx @johnnypeck @mbrukman @jikkuatwork