Official implementation of "Edge-Aware Image Manipulation via Diffusion Models with a Novel Structure-Preservation Loss"

SPL is a training-free method for pixel-level structure-preserving image editing using latent diffusion models. Unlike existing methods that only preserve coarse layouts, SPL maintains fine-grained edge structures while allowing creative semantic edits through text prompts.

Key Features:

• Pixel-Level Structure Preservation: Local Linear Model-based loss for maintaining edge details
• Training-Free: Plug-and-play with pre-trained diffusion models
• Versatile Editing: Supports relighting, tone adjustment, style transfer, season changes, background replacement, and more
• Precise Local Control: Cross-attention mask upsampling for targeted editing
• Interactive Interface: Easy-to-use Gradio web demo

1. Clone the repository:

git clone https://github.com/gongms00/SPL.git
cd SPL

2. Create a conda environment:

conda create -n spl python=3.10 -y
conda activate spl

3. Install PyTorch:

pip install torch==2.8.0 torchvision==0.23.0 torchaudio==2.8.0 --index-url https://download.pytorch.org/whl/cu128

4. Install remaining dependencies:

pip install -r requirements.txt

5. (Optional) Set environment variables for harmonization feature:

export OPENAI_API_KEY=your_openai_key

python app.py

Open your browser at http://localhost:7860

1. Upload a source image
2. Enter source prompt (describing the current image)
3. Enter edit prompt (describing the desired edit)
4. Adjust parameters if needed
5. Click "Run" to generate the edited image

SPL quantifies structural differences between source and edited images using Local Linear Models. For each local window, the loss enforces that the edited image is a linear transformation of the source. The bidirectional constraint ensures robust structure matching, even in flat regions.

Key Properties:

• Edge-preserving: image gradients are preserved through the local linear model
• Computed on grayscale (intensity) channel for uniform RGB updates
• Color Preservation Loss (CPL) handles chrominance preservation

At each denoising timestep, the predicted latent is decoded to image space, optimized with SPL using the source image as reference, then re-encoded back to latent space. A final post-processing step refines the decoded output to recover fine details lost during VAE decoding.

For localized editing:

1. Extract 16x16 attention map from cross-attention layers
2. Progressively upsample 2x with guided filter at each step
3. Source image guides edge-aligned boundary refinement
4. Result: high-resolution mask with sharp, accurate boundaries

These are the core parameters that control how much the edited image preserves the source image's structure and color.

When SPL or CPL is set to "Masked area" mode, these parameters control which regions of the image are preserved. Masks are extracted from cross-attention maps and upsampled using guided filtering.

This project builds upon:

• InfEdit (Xu et al., 2024): Coarse-structure-preserving editing via attention conditioning
• Prompt-to-Prompt (Hertz et al., 2023): Cross-attention control for text-driven editing

Our contribution is the Structure Preservation Loss (SPL) that adds pixel-level edge structure preservation to these coarse-structure methods.

@inproceedings{gong2026spl,
  title={Edge-Aware Image Manipulation via Diffusion Models with a Novel Structure-Preservation Loss},
  author={Gong, Minsu and Ryu, Nuri and Ok, Jungseul and Cho, Sunghyun},
  booktitle={Proceedings of the Winter Conference on Applications of Computer Vision (WACV)},
  year={2026}
}

• InfEdit for the base editing framework
• Prompt-to-Prompt for attention control
• LCM for efficient sampling
• Guided Filter for edge-aware processing

This project is licensed under the Apache License 2.0. See LICENSE.txt for details.