temp: remove all LoRAs except 1 for testing

MULTI_LORA_DOCUMENTATION.md

@@ -1,8 +1,8 @@
-# Multi-LoRA Image Editing Implementation
+# Multi-LoRA Image Editing Implementation (Simplified)
 
 ## Overview
 
-This implementation provides a comprehensive multi-LoRA (Low-Rank Adaptation) system for the Qwen-Image-Edit application, enabling dynamic switching between different LoRA adapters with specialized capabilities. The system follows the HuggingFace Spaces pattern for LoRA loading and fusion, with **Lightning LoRA always active as the base optimization**.
+This implementation provides a simplified multi-LoRA (Low-Rank Adaptation) system for the Qwen-Image-Edit application, focusing on **Lightning LoRA always active as the base optimization** with only **Object Remover** as an additional LoRA for testing.
 
 ## Architecture
 
@@ -17,12 +17,12 @@ This implementation provides a comprehensive multi-LoRA (Low-Rank Adaptation) sy
 2. **LoRA Configuration** (`app.py`)
    - Centralized `LORA_CONFIG` dictionary
    - Lightning LoRA configured as always-loaded base
-   - Support for different LoRA types and fusion methods
+   - Simplified to Object Remover for focused testing
 
 3. **Dynamic UI System** (`app.py`)
    - Conditional component visibility based on LoRA selection
    - Lightning LoRA status indication
-   - Type-specific UI adaptations (style vs edit)
+   - Type-specific UI adaptations (edit vs base)
    - Real-time interface updates
 
 ## ⚡ Lightning LoRA Always-On Architecture
@@ -33,7 +33,7 @@ This implementation provides a comprehensive multi-LoRA (Low-Rank Adaptation) sy
 
 - **Consistent Performance**: Always-on 4-step generation
 - **Enhanced Speed**: Lightning's optimization applies to all operations
-- **Multi-LoRA Fusion**: Combine Lightning speed with specialized LoRA capabilities
+- **Multi-LoRA Fusion**: Combine Lightning speed with Object Remover capabilities
 
 ### Implementation Details
 
@@ -66,7 +66,7 @@ lora_manager.fuse_lora(LIGHTNING_LORA_NAME)
 def load_and_fuse_additional_lora(lora_name):
     """
     Load an additional LoRA while keeping Lightning LoRA always active.
-    This enables combining Lightning's speed with other LoRA capabilities.
+    This enables combining Lightning's speed with Object Remover capabilities.
     """
     # Always keep Lightning LoRA loaded
     # Load additional LoRA without resetting to base state
@@ -103,33 +103,25 @@ def infer(lora_name, ...):
         pipe.disable_adapters()  # Disable additional LoRA but keep Lightning
 ```
 
-## LoRA Types and Capabilities
+## Simplified LoRA Configuration
 
-### Supported LoRA Adapters
+### Current Supported LoRAs
 
 | LoRA Name | Type | Method | Always-On | Description |
 |-----------|------|--------|-----------|-------------|
 | **⚡ Lightning (4-Step)** | base | standard | ✅ **Always** | Fast 4-step generation - always active |
-| **None** | edit | none | ❌ | Base model without additional LoRA |
-| **InStyle (Style Transfer)** | style | manual_fuse | ⚡ Lightning+ | Style transfer from reference image |
-| **InScene (In-Scene Editing)** | edit | standard | ⚡ Lightning+ | Object positioning and perspective changes |
-| **Face Segmentation** | edit | standard | ⚡ Lightning+ | Transform facial images to segmentation masks |
-| **Object Remover** | edit | standard | ⚡ Lightning+ | Remove objects while maintaining background |
+| **None** | edit | none | ❌ | Base model with Lightning optimization |
+| **Object Remover** | edit | standard | ⚡ Lightning+ | Removes objects from an image while maintaining background consistency |
 
-### Lightning + Other LoRA Combinations
+### Lightning + Object Remover Combination
 
-Every LoRA operation benefits from Lightning's 4-step generation speed:
-
-- **Lightning + Style Transfer**: Fast style application with 4-step generation
-- **Lightning + Object Removal**: Quick object removal with optimized inference
-- **Lightning + Face Segmentation**: Rapid segmentation with enhanced speed
-- **Lightning + In-Scene Editing**: Fast scene modifications with 4-step process
+**Lightning + Object Remover**: Fast object removal with 4-step generation optimization
 
 ### LoRA Type Classifications
 
 - **Base LoRA**: Lightning (always loaded, always active)
-- **Style LoRAs**: Require style reference images, use manual fusion
-- **Edit LoRAs**: Require input images, use standard fusion methods
+- **Edit LoRAs**: Object Remover (requires input images, uses standard fusion)
+- **None**: Base model with Lightning optimization
 
 ## Key Features
 
@@ -169,18 +161,16 @@ def on_lora_change(lora_name):
 }
 ```
 
-### 3. Multi-LoRA Fusion Methods
+### 3. Simplified Multi-LoRA Fusion
 
 - **Lightning Base**: Always loaded, always active
-- **Additional LoRAs**: Loaded alongside Lightning using:
-  - **Standard Fusion**: Combined adapter loading
-  - **Manual Fusion**: Custom implementation for specialized LoRAs
-  - **No Additional LoRA**: Lightning-only operation
+- **Object Remover**: Loaded alongside Lightning using standard fusion
+- **None**: Lightning-only operation
 
 ### 4. Memory Management with Lightning
 
 - Lightning LoRA remains loaded throughout session
-- Additional LoRAs loaded/unloaded as needed
+- Object Remover LoRA loaded/unloaded as needed
 - GPU memory optimized for Lightning + one additional LoRA
 - Automatic cleanup of non-Lightning adapters
 
@@ -189,10 +179,6 @@ def on_lora_change(lora_name):
 Each LoRA has a custom prompt template (Lightning provides base 4-step generation):
 
 ```python
-"InStyle (Style Transfer)": {
-    "prompt_template": "Make an image in this style of {prompt}",
-    "type": "style"
-},
 "Object Remover": {
     "prompt_template": "Remove {prompt}",
     "type": "edit"
@@ -204,14 +190,19 @@ Each LoRA has a custom prompt template (Lightning provides base 4-step generatio
 ### Basic Usage with Always-On Lightning
 
 1. **Lightning is Always Active**: No selection needed - Lightning runs all operations
-2. **Select Additional LoRA**: Choose optional LoRA to combine with Lightning
-3. **Upload Images**: 
-   - Style LoRAs: Upload style reference image
-   - Edit LoRAs: Upload input image to edit
-4. **Enter Prompt**: Describe the desired modification
+2. **Select Additional LoRA**: Choose "Object Remover" to combine with Lightning
+3. **Upload Images**: Upload input image to edit
+4. **Enter Prompt**: Describe the object to remove
 5. **Configure Settings**: Adjust advanced parameters (4-step generation always enabled)
 6. **Generate**: Click "Generate!" to process with Lightning optimization
 
+### Object Remover Usage
+
+1. **Select "Object Remover"** from the dropdown
+2. **Upload Input Image**: The image containing the object to remove
+3. **Enter Prompt**: Describe the object to remove (e.g., "person", "car", "tree")
+4. **Generate**: Lightning + Object Remover will remove the specified object
+
 ### Advanced Configuration
 
 #### Adding New LoRAs (with Lightning Always-On)
@@ -236,18 +227,6 @@ lora_manager.register_lora("Custom LoRA", lora_path, **config)
 
 3. **Lightning + Custom LoRA**: Automatically combines with always-on Lightning
 
-#### Custom UI Configuration
-
-```python
-ui_config = {
-    "description": "Custom LoRA description",
-    "ui_components": [
-        {"type": "slider", "name": "custom_param", "label": "Custom Parameter", "min": 0, "max": 1, "value": 0.5}
-    ]
-}
-lora_manager.configure_lora("Custom LoRA", ui_config)
-```
-
 ## Technical Implementation
 
 ### Lightning Always-On Process
@@ -255,7 +234,7 @@ lora_manager.configure_lora("Custom LoRA", ui_config)
 1. **Initialization**: Load Lightning LoRA first
 2. **Fusion**: Fuse Lightning weights permanently
 3. **Persistence**: Keep Lightning active throughout session
-4. **Combination**: Load additional LoRAs alongside Lightning
+4. **Combination**: Load Object Remover alongside Lightning
 5. **Preservation**: Never unload Lightning LoRA
 
 ### Lightning Loading Process
@@ -282,35 +261,6 @@ gc.collect()
 torch.cuda.empty_cache()
 ```
 
-### Manual Fusion with Lightning
-
-```python
-def fuse_lora_manual(transformer, lora_state_dict, alpha=1.0):
-    # Lightning is already fused into transformer
-    # Additional manual fusion on top of Lightning
-    key_mapping = {}
-    for key in lora_state_dict.keys():
-        base_key = key.replace('diffusion_model.', '').rsplit('.lora_', 1)[0]
-        if base_key not in key_mapping:
-            key_mapping[base_key] = {}
-        if 'lora_A' in key:
-            key_mapping[base_key]['down'] = lora_state_dict[key]
-        elif 'lora_B' in key:
-            key_mapping[base_key]['up'] = lora_state_dict[key]
-
-    for name, module in tqdm(transformer.named_modules(), desc="Fusing additional layers"):
-        if name in key_mapping and isinstance(module, torch.nn.Linear):
-            lora_weights = key_mapping[name]
-            if 'down' in lora_weights and 'up' in lora_weights:
-                device = module.weight.device
-                dtype = module.weight.dtype
-                lora_down = lora_weights['down'].to(device, dtype=dtype)
-                lora_up = lora_weights['up'].to(device, dtype=dtype)
-                merged_delta = lora_up @ lora_down
-                module.weight.data += alpha * merged_delta
-    return transformer
-```
-
 ## Testing and Validation
 
 ### Validation Scripts
@@ -328,26 +278,33 @@ def fuse_lora_manual(transformer, lora_state_dict, alpha=1.0):
 ✅ **UI indicates Lightning always active**  
 ✅ **Proper loading sequence implemented**  
 
+### Object Remover Testing
+
+✅ **Object Remover loads alongside Lightning**  
+✅ **Lightning + Object Remover combination works**  
+✅ **Prompt template "Remove {prompt}" functions correctly**  
+✅ **Memory management for Lightning + Object Remover**  
+
 ## Performance Considerations
 
 ### Lightning Always-On Benefits
 
 - **Consistent Speed**: All operations use 4-step generation
 - **Reduced Latency**: No loading time for Lightning between requests
-- **Enhanced Performance**: Lightning optimization applies to all LoRAs
-- **Memory Efficiency**: Lightning stays in memory, additional LoRAs loaded as needed
+- **Enhanced Performance**: Lightning optimization applies to Object Remover
+- **Memory Efficiency**: Lightning stays in memory, Object Remover loaded as needed
 
 ### Speed Optimization
 
 - **4-Step Generation**: Lightning provides ultra-fast inference
 - **AOT Compilation**: Ahead-of-time compilation with Lightning active
-- **Adapter Combination**: Lightning + specialized LoRA for optimal results
+- **Adapter Combination**: Lightning + Object Remover for optimal results
 - **Optimized Attention Processors**: FA3 attention with Lightning
 
 ### Memory Optimization
 
 - Lightning LoRA always in memory (base memory usage)
-- Additional LoRA loaded on-demand
+- Object Remover LoRA loaded on-demand
 - Efficient adapter switching
 - GPU memory management for multiple adapters
 
@@ -365,10 +322,10 @@ def fuse_lora_manual(transformer, lora_state_dict, alpha=1.0):
    - Verify adapter status: `pipe.get_active_adapters()`
    - Ensure Lightning is not being disabled
 
-3. **Multi-LoRA Issues**
-   - Check adapter combination: Lightning should always be in active adapters
-   - Verify additional LoRA loading without Lightning reset
-   - Monitor memory usage for multiple adapters
+3. **Object Remover Issues**
+   - Check Object Remover loading: Look for "Lightning + Object Remover now active"
+   - Verify prompt format: Should be "Remove {object}"
+   - Monitor memory usage for Lightning + Object Remover
 
 ### Debug Mode
 
@@ -386,10 +343,10 @@ print(f"All active adapters: {pipe.get_active_adapters()}")
 
 ### Planned Features
 
-1. **LoRA Blending**: Advanced blending of multiple LoRAs with Lightning
-2. **Lightning Optimization**: Dynamic Lightning parameter adjustment
-3. **Performance Monitoring**: Real-time Lightning performance metrics
-4. **Lightning Fine-tuning**: On-demand Lightning optimization
+1. **Additional LoRAs**: Add more LoRAs after successful Object Remover testing
+2. **LoRA Blending**: Advanced blending of multiple LoRAs with Lightning
+3. **Lightning Optimization**: Dynamic Lightning parameter adjustment
+4. **Performance Monitoring**: Real-time Lightning performance metrics
 5. **Batch Processing**: Process multiple images with Lightning always-on
 
 ### Extension Points
@@ -399,10 +356,27 @@ print(f"All active adapters: {pipe.get_active_adapters()}")
 - Advanced multi-LoRA combination algorithms
 - Lightning performance profiling
 
+## Simplified Configuration Benefits
+
+### Focused Testing
+
+- **Reduced Complexity**: Only Lightning + Object Remover to test
+- **Clear Validation**: Easy to verify Lightning always-on functionality
+- **Debugging**: Simplified troubleshooting with fewer variables
+- **Performance**: Clear performance benefits of Lightning always-on
+
+### Risk Mitigation
+
+- **Gradual Rollout**: Test one LoRA before adding more
+- **Validation**: Ensure Lightning + LoRA combination works correctly
+- **Memory Management**: Verify memory usage with Lightning + one LoRA
+- **User Experience**: Validate simplified UI with fewer options
+
 ## References
 
 - [Qwen-Image-Edit Model](https://huggingface.co/Qwen/Qwen-Image-Edit-2509)
 - [Lightning LoRA Repository](https://huggingface.co/lightx2v/Qwen-Image-Lightning)
+- [Object Remover LoRA Repository](https://huggingface.co/valiantcat/Qwen-Image-Edit-Remover-General-LoRA)
 - [Diffusers LoRA Documentation](https://huggingface.co/docs/diffusers/main/en/using-diffusers/loading_adapters)
 - [PEFT Library](https://github.com/huggingface/peft)
 - [HuggingFace Spaces Pattern](https://huggingface.co/spaces)

app.py

@@ -69,7 +69,7 @@ Please strictly follow the rewriting rules below:
   "Restore and colorize the old photo."  
 - Clearly specify the object to be modified. For example:  
     > Original: Modify the subject in Picture 1 to match the style of Picture 2.  
-    > Rewritten: Change the girl in Picture 1 to the ink-wash style of Picture 2 — rendered in black-and-white watercolor with soft color transitions.
+    > Rewritten: "Change the girl in Picture 1 to the ink-wash style of Picture 2 — rendered in black-and-white watercolor with soft color transitions.
 
 ### 5. Material Replacement
 - Clearly specify the object and the material. For example: "Change the material of the apple to papercut style."
@@ -174,7 +174,7 @@ def polish_prompt_hf(prompt, img_list):
         # Fallback to original prompt if enhancement fails
         return prompt
 
-# Define LoRA configurations with Lightning as always-loaded base
+# Define simplified LoRA configurations with Lightning as always-loaded base
 LORA_CONFIG = {
     "Lightning (4-Step)": {
         "repo_id": "lightx2v/Qwen-Image-Lightning",
@@ -191,31 +191,7 @@ LORA_CONFIG = {
         "type": "edit",
         "method": "none",
         "prompt_template": "{prompt}",
-        "description": "Use the base Qwen-Image-Edit model without any LoRA.",
-    },
-    "InStyle (Style Transfer)": {
-        "repo_id": "peteromallet/Qwen-Image-Edit-InStyle",
-        "filename": "InStyle-0.5.safetensors",
-        "type": "style",
-        "method": "manual_fuse",
-        "prompt_template": "Make an image in this style of {prompt}",
-        "description": "Transfers the style from a reference image to a new image described by the prompt.",
-    },
-    "InScene (In-Scene Editing)": {
-        "repo_id": "flymy-ai/qwen-image-edit-inscene-lora",
-        "filename": "flymy_qwen_image_edit_inscene_lora.safetensors",
-        "type": "edit",
-        "method": "standard",
-        "prompt_template": "{prompt}",
-        "description": "Improves in-scene editing, object positioning, and camera perspective changes.",
-    },
-    "Face Segmentation": {
-        "repo_id": "TsienDragon/qwen-image-edit-lora-face-segmentation",
-        "filename": "pytorch_lora_weights.safetensors",
-        "type": "edit",
-        "method": "standard",
-        "prompt_template": "change the face to face segmentation mask",
-        "description": "Transforms a facial image into a precise segmentation mask.",
+        "description": "Use the base Qwen-Image-Edit model with Lightning optimization.",
     },
     "Object Remover": {
         "repo_id": "valiantcat/Qwen-Image-Edit-Remover-General-LoRA",
@@ -286,7 +262,7 @@ lora_manager.configure_lora(LIGHTNING_LORA_NAME, {
 lora_manager.load_lora(LIGHTNING_LORA_NAME)
 lora_manager.fuse_lora(LIGHTNING_LORA_NAME)
 
-# Register other LoRAs
+# Register other LoRAs (only Object Remover for testing)
 for lora_name, config in LORA_CONFIG.items():
     if lora_name != LIGHTNING_LORA_NAME and config["repo_id"] is not None:
         lora_path = hf_hub_download(repo_id=config["repo_id"], filename=config["filename"])
@@ -389,7 +365,7 @@ def infer(
         if style_image is None:
             raise gr.Error("Style Transfer LoRA requires a Style Reference Image.")
         image_for_pipeline = style_image
-    else:  # 'edit'
+    else:  # 'edit' or 'base'
         if input_image is None:
             raise gr.Error("This LoRA requires an Input Image.")
         image_for_pipeline = input_image
@@ -451,7 +427,7 @@ with gr.Blocks(css="#col-container { margin: 0 auto; max-width: 1024px; }") as d
         gr.Markdown("""
         [Learn more](https://github.com/QwenLM/Qwen-Image) about the Qwen-Image series. 
         This demo uses the new [Qwen-Image-Edit-2509](https://huggingface.co/Qwen/Qwen-Image-Edit-2509) with support for multiple LoRA adapters.
-        **⚡ Lightning LoRA is always active for fast 4-step generation** - combine it with other LoRAs for optimized performance.
+        **⚡ Lightning LoRA is always active for fast 4-step generation** - combine it with Object Remover for optimized performance.
         Try on [Qwen Chat](https://chat.qwen.ai/), or [download model](https://huggingface.co/Qwen/Qwen-Image-Edit-2509) to run locally with ComfyUI or diffusers.
         """)
 
@@ -465,10 +441,10 @@ with gr.Blocks(css="#col-container { margin: 0 auto; max-width: 1024px; }") as d
                 )
                 lora_description = gr.Markdown(visible=False)
                 
-                input_image_box = gr.Image(label="Input Image", type="pil", visible=False)
+                input_image_box = gr.Image(label="Input Image", type="pil", visible=True)
                 style_image_box = gr.Image(label="Style Reference Image", type="pil", visible=False)
                 
-                prompt_box = gr.Textbox(label="Prompt", placeholder="Describe the content or object to remove...")
+                prompt_box = gr.Textbox(label="Prompt", placeholder="Describe the object to remove...")
                 
                 run_button = gr.Button("Generate!", variant="primary")