Skip to content

AttributeError: 'CLIPTextModel' object has no attribute 'peft_config' when using default train_dreambooth_lora_sdxl_advanced.py but with checkpointing_steps < max_train_steps #6444

@brandostrong

Description

@brandostrong

Describe the bug

When checkpointing at defined steps, such as every 50 steps I get a 'CLIPTextModel' object has no attribute 'peft_config' while trying to save the model. This only happens when checkpointing_steps < max_train_steps, as the code wouldn't run if it was higher than the train steps

Reproduction

python train_dreambooth_lora_sdxl_advanced.py --pretrained_model_name_or_path="stabilityai/stable-diffusion-xl-base-1.0" --pretrained_vae_model_name_or_path="madebyollin/sdxl-vae-fp16-fix" --dataset_name="./trainimages/" --caption_column=prompt --instance_prompt="a TOK person" --validation_prompt="a TOK person dressed as Yoda" --output_dir="huggy_v11" --caption_column="prompt" --mixed_precision="bf16" --resolution=1024 --train_batch_size=1 --repeats=1 --gradient_accumulation_steps=1 --gradient_checkpointing --train_text_encoder_ti --learning_rate=1 --text_encoder_lr=1 --lr_scheduler="constant" --snr_gamma=5.0 --lr_warmup_steps=0 --rank=32 --max_train_steps=1000 --checkpointing_steps=50 --seed="0" --optimizer="prodigy" --prodigy_safeguard_warmup=True --prodigy_use_bias_correction=True --adam_beta1=0.9 --adam_beta2=0.99 --adam_weight_decay=0.01

Logs

Traceback (most recent call last):
  File "C:\Users\\Documents\GitHub\\train_dreambooth_lora_sdxl_advanced.py", line 2100, in <module>
    main(args)
  File "C:\Users\\Documents\GitHub\\train_dreambooth_lora_sdxl_advanced.py", line 1895, in main
    accelerator.save_state(save_path)
  File "C:\Users\\Documents\GitHub\\venv\lib\site-packages\accelerate\accelerator.py", line 2793, in save_state
    hook(self._models, weights, output_dir)
  File "C:\Users\\Documents\GitHub\\train_dreambooth_lora_sdxl_advanced.py", line 1320, in save_model_hook
    get_peft_model_state_dict(model)
  File "C:\Users\\Documents\GitHub\\venv\lib\site-packages\peft\utils\save_and_load.py", line 64, in get_peft_model_state_dict
    config = model.peft_config[adapter_name]
  File "C:\Users\\Documents\GitHub\\venv\lib\site-packages\torch\nn\modules\module.py", line 1688, in __getattr__
    raise AttributeError(f"'{type(self).__name__}' object has no attribute '{name}'")
AttributeError: 'CLIPTextModel' object has no attribute 'peft_config'

System Info

  • diffusers version: 0.25.0.dev0
  • Platform: Windows-10-10.0.22621-SP0
  • Python version: 3.9.16
  • PyTorch version (GPU?): 2.3.0.dev20240102+cu121 (True)
  • Huggingface_hub version: 0.20.1
  • Transformers version: 4.37.0.dev0
  • Accelerate version: 0.23.0
  • xFormers version: 0.0.21
  • Using GPU in script?:
  • Using distributed or parallel set-up in script?:

Who can help?

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions