AP Workflow 9.0 for ComfyUI

• The AP Workflow now features two next-gen upscalers: CCSR, and the new SUPIR. Since one performs better than the other depending on the type of image you want to upscale, each one has a dedicated function.
  
  Additionally, the Upscaler (SUPIR) function can be used to perform Magnific AI-style creative upscaling.
• A new Image Generator (Dall-E) function allows you to generate an image with OpenAI Dall-E 3 instead of Stable Diffusion. This function should be used in conjunction with the Inpainter without Mask function to take advantage of Dall-E 3 superior capability to follow the user prompt and Stable Diffusion superior ecosystem of fine-tunes and LoRAs.

  You can also use this function in conjunction with the Image Generator (SD) function to simply compare how each model renders the same prompt.

• A new Advanced XYZ Plot function allows you to study the effect of ANY parameter change in ANY node inside the AP Workflow.
• A new Face Cloner function uses the InstantID technique to quickly change the style of any face in a Reference Image you upload via the Uploader function.
• A new Face Analyzer function allows you to evaluate a batch of generated images and automatically choose the ones that present facial landmarks very similar to the ones in a reference image you upload via the Uploader function. This function is especially useful in conjuction with the new Face Cloner function.
• A new Training Helper for Caption Generator function will allow you to use the Caption Generator function to automatically caption hundreds or thousands of images in a batch directory. This is useful for model training purposes.

  The Uploader function has a new Load Image Batch node to accomodate this new feature.

  To use this new capability you must activate both the Caption Generator and the Training Helper for Caption Generator functions in the Controller function.

• The AP Workflow now features a number of @rgthree Bookmark nodes to quickly recenter the workflow on the 10 most used functions. You can move the Bookmark nodes where you prefer to customize your hyperjumps.
• The AP Workflow now supports new @cubiq’s IPAdapter plus v2 nodes.
• The AP Workflow now supports the new PickScore nodes, used in the Aesthetic Score Predictor function.
• The Uploader function now allows you to upload both a source image and a reference image. The latter is used by the Face Cloner, the Face Swapper, and the IPAdapter functions.
• The Caption Generator function now offers the possibility to replace the user prompt with a caption automatically generated by Moondream v1 or v2 (local inference), GPT-4V (remote inference via OpenAI API), or LLaVA (local inference via LM Studio).
• The three Image Evaluators in the AP Workflow are now daisy chained for sophisticated image selection.

  First, the Face Analyzer (see below) automatically chooses the image/s with the face that most closely resembles the original. From there, the Aesthetic Score Predictor further ranks the quality of the images and automatically chooses the ones that match your criteria. Finally, the Image Chooser allows you to manually decide which image to further process via the image manipulator functions in the L2 of the pipeline.

  You have the choice to use only one of these Image Evaluators, or any combination of them, by enabling each one in the Controller function.

• The Prompt Enricher function has been greatly simplified and now it works again open access models served by LM Studio, Oobabooga, etc. thanks to @glibsonoran’s new Advanced Prompt Enhancer node.
• The Image Chooser function now can be activated from the Controller function with a dedicated switch, so you don’t have to navigate the workflow just to enable it.
• The LoRA Info node is now relocated inside the Prompt Builder function.
• The configuration parameters of various nodes in the Face Detailer function have been modified to (hopefully) produce much better results.
• The entire L2 pipeline layout has been reorganized so that each function can be muted instead of bypassed.
• The ReVision function is gone. Probably, nobody was using it.
• The Image Enhancer function is gone, too. You can obtain a creative upscaling of equal or better quality by reducing the strength of ControlNet in the SUPIR node.
• The StyleAligned function is gone, too. IPAdapter has become so powerful that there’s no need for it anymore.

• Support for Stable Diffusion 3
• [EA2] The Prompt Enricher function now supports the new version of Advanced Prompt Enhancer node, which allows you to use both Anthropic and Groq LLMs on top of ones offered by OpenAI and the open access ones you can serve with a local installation of LM Studio or OogaBooga.
• [EA2] The AP Workflow now features an always-on Prompt Tagger function, designed to simplify the addition of LoRA and embedding tags at the beginning or end of both positive and negative prompts. You can even insert the tags in the middle of the prompt.
  
  The Prompt Builder and the Prompt Enricher functions have been significantly revamped to accomodate the change. The LoRA Info node has been moved inside the Prompt Tagger function.
• [EA2] The Caption Generator function does not require you to manually define LoRA tags anymore. It will automatically use the ones defined in the new Prompt Tagger function.
• [EA2] The Image Generator (Dall-E) function does not require you to manually define the user prompt anymore. It will automatically use the one defined in the Prompt Builder function.
• [EA2] There's a simpler switch to activate an attention mask for the IPAdapter 1 function.
• [EA2] The ControlNet Previews and ControlNet + Control-LoRAs functions now use a Get Node to get their input. In this way, you can easily switch from the Source Image to the 1st Reference Image uploaded in the Uploader function, if you need to.
• [EA1] The AP Workflow now supports the new Perturbed-Attention Guidance (PAG)
• [EA1] The AP Workflow now features an IPAdapter 2 function. You can chain it together with the IPAdapter 1 function, for example, to influence the image generation with two different reference images.
• [EA1] The Uploader function now supports uploading a 2nd Reference Image, used exclusively by the new IPAdapter 2 function.
• [EA1] The AP Workflow now features a different Canny preprocessor to assist Canny ControlNet. The new preprocessor gives you more control on how many details from the source image should influence the generation.
• [EA1] The AP Workflow is now configured to use the DWPose preprocessor by default to assist OpenPose ControlNet.
• [EA1] The Perp Neg node is not supported anymore due to its new implementation incompatible with the workflow layout.
• [EA1] The Self-Attention Guidance node is gone. We have more modern and reliable ways to add details to generated images.

Early Access to 10.0

The entire workflow is embedded in the workflow picture itself. Click on it and the full version will open in a new tab. Right click on the full version image and download it. Drag it inside ComfyUI, and you’ll have the same workflow you see below.

Before you download the workflow, be sure you read “9.0” in the image. If not, you are looking at a cached version of the image.

Alternatively, you can download the JSON version of the AP Workflow and load it via ComfyUI Manager.

If you are interested in sponsoring the AP Workflow, reach out.

If, after loading the workflow, you see a lot of red boxes, you must install some custom node suites.

The AP Workflow depends on multiple custom nodes that you might not have installed. You should download and install ComfyUI Manager, and then install the required custom nodes suites to be sure you can run this workflow.

You have two options:

1. (Recommended option) Restore the AP Workflow 9.0 custom node suites snapshot.
2. Install the required custom node suites manually.

Restore the AP Workflow 9.0 custom node suites snapshot

The manual installation of the required custom node suites has been proven unreliable for many users. To improve your chances to run this workflow, you should create a clean install of ComfyUI, manually install the ComfyUI Manager, and then restore a snapshot of Alessandro’s ComfyUI working environment.

This will help you install a version of each custom node suite that is known to work with the AP Workflow 9.0.

Instructions:

1. Install ComfyUI in a new folder to create a clean, new environment.
2. Install ComfyUI Manager.
3. Shut down ComfyUI.
4. Download the snapshot.
5. Move/copy the snapshot to the /ComfyUI/custom_nodes/ComfyUI-Manager/snapshots folder.
6. Restart ComfyUI.
7. Open ComfyUI Manager and then the new Snapshot Manager.
8. Restore the AP Workflow 9.0 Custom Nodes Snapshot.
9. Restart ComfyUI.

Notice that the Snapshot Manager is a new, experimental feature and it might not work in every situation. If you encounter errors, check the documentation: https://github.com/ltdrdata/ComfyUI-Manager#snapshot-manager

Install the required custom node suites manually

Install the nodes listed below via ComfyUI Manager.

Notice that some of these nodes conflict with others that you might have already installed. This is why this option is discouraged and you should really consider the first option.

• \ComfyUI\custom_nodes\ComfyUI-N-Sidebar
• \ComfyUI\custom_nodes\websocket_image_save.py
• \ComfyUI\custom_nodes\masquerade-nodes-comfyui
• \ComfyUI\custom_nodes\comfyui-previewlatent
• \ComfyUI\custom_nodes\lora-info
• \ComfyUI\custom_nodes\cg-image-picker
• \ComfyUI\custom_nodes\ComfyUI_essentials
• \ComfyUI\custom_nodes\ComfyUI-post-processing-nodes
• \ComfyUI\custom_nodes\ComfyUI_InstantID
• \ComfyUI\custom_nodes\ComfyUI_experiments
• \ComfyUI\custom_nodes\ComfyUI_IPAdapter_plus
• \ComfyUI\custom_nodes\comfyui-inpaint-nodes
• \ComfyUI\custom_nodes\comfyui-browser
• \ComfyUI\custom_nodes\ComfyUI-Advanced-ControlNet
• \ComfyUI\custom_nodes\failfast-comfyui-extensions
• \ComfyUI\custom_nodes\ComfyUI-Custom-Scripts
• \ComfyUI\custom_nodes\ComfyUI-KJNodes
• \ComfyUI\custom_nodes\ComfyUI-PickScore-Nodes
• \ComfyUI\custom_nodes\rgthree-comfy
• \ComfyUI\custom_nodes\efficiency-nodes-comfyui
• \ComfyUI\custom_nodes\comfyui_segment_anything
• \ComfyUI\custom_nodes\comfyui_controlnet_aux
• \ComfyUI\custom_nodes\ComfyUI_Comfyroll_CustomNodes
• \ComfyUI\custom_nodes\ComfyUI-moondream
• \ComfyUI\custom_nodes\comfyui-prompt-reader-node
• \ComfyUI\custom_nodes\ComfyUI-TiledDiffusion
• \ComfyUI\custom_nodes\ComfyUI-Inspire-Pack
• \ComfyUI\custom_nodes\ComfyUI-VideoHelperSuite
• \ComfyUI\custom_nodes\ComfyUI-dnl13-seg
• \ComfyUI\custom_nodes\ComfyUI_FaceAnalysis
• \ComfyUI\custom_nodes\ComfyUI-SUPIR
• \ComfyUI\custom_nodes\ComfyUI-DDColor
• \ComfyUI\custom_nodes\Plush-for-ComfyUI
• \ComfyUI\custom_nodes\ComfyUI-Impact-Pack
• \ComfyUI\custom_nodes\comfyui-reactor-node
• \ComfyUI\custom_nodes\ComfyUI-Manager
• \ComfyUI\custom_nodes\ComfyUI_tinyterraNodes
• \ComfyUI\custom_nodes\was-node-suite-comfyui
• \ComfyUI\custom_nodes\ComfyUI-VoiceCraft
• \ComfyUI\custom_nodes\ComfyUI-CCSR
• \ComfyUI\custom_nodes\comfy_mtb
• \ComfyUI\custom_nodes\ComfyUI-Crystool

Many nodes used throughouth the AP Workflow require specific AI models to perform their task. While some nodes automatically download the required models, others require you to download them manually.

At today, there’s not an easy way to export the full list of models Alessandro is using in his ComfyUI environment.

The best way to know which models you need to download is by opening ComfyUI Manager and proceed to the Install Models section. Here you’ll find a list of all the models each node requires or recommends to download.

The AP Workflow is a large, moderately complex workflow. It can be difficult to navigate if you are new to ComfyUI.

Start from the Controller section of the workflow on the left, and proceed to the right by configuring each section relevant to you: Uploader or Prompt Builder, Prompt Enricher, Prompt Selector and, finally, Configurator.

You should not change any additional setting in other areas of the workflow unless you have to tweak how a certain function behaves.

The AP Workflow come pre-configured to generate images with the SDXL 1.0 Base + Refiner models. After you download the workflow, you have to do nothing in particular but queue a generation with the prompt already configured in the Prompt Builder function.

Check the outcome of your image generation/manipulation in the Magic section on the bottom-left of the workflow.

The AP Workflow allows you to generate images from text instructions written in natural language (text-to-image. txt2img, or t2i), or to upload existing images for further manipulation (image-to-image, img2img, or Uploader).

Image Generator (SD)

You can use the Image Generator (SD) function to generate images with:

SDXL 1.0 Base + Refiner models

By default, The AP Workflow is configured to generated images with the SDXL 1.0 Base model used in conjunction with the SDXL 1.0 Refiner model.

When you define the total number of diffusion steps you want the system to perform, the workflow will automatically allocate a certain number of those steps to each model, according to the refiner_start parameter in the Configurator function.

Further guidance is provided in a Note in the Configurator function.

Here’s an example of the images that the SDXL 1.0 Base + Refiner models can generate:

SDXL 1.0 Base model only or Fine-Tuned SDXL models

If you prefer, you can disable the Refiner function. That is useful, for example, when you want to generate images with fine-tuned SDXL models that require no Refiner.

If you don’t want to use the Refiner, you must disable it in the Controller function, and then set the refiner_start parameter to 1 in the Configurator section.

Here’s an example of the images that a fine-tuned SDXL model can generate:

SD 1.5 Base model or Fine-Tuned SD 1.5 models

The AP Workflow supports image generation with Stable Diffusion 1.5 models.

However, to reconfigure the workflow to generate images with those models, you must follow a number of steps defined in the Support section of this document.

Image Generator (Dall-E)

You can use the Image Generator (Dall-E) function to generate images OpenAI Dall-E 3 instead of Stable Diffusion.

This function is designed to take advantage of Dall-E 3 superior capability to follow the user prompt compared to Stable Diffusion 1.5 and XL.

Once you generate a Dall-E 3 image that respects the composition you described in your prompt, you can further modify its aesthetic with the Inpainter without Mask function and take full advantage of Stable Diffusion’s superior ecosystem of fine-tunes and LoRAs.

Notice that the Image Generator (Dall-E) function is NOT fully integrated with the Prompt Builder function. You must enter the prompt in a dedicated node inside the function itself:

Uploader

You can use the Uploader function to upload individual images and as well as entire directories of images, processed as a batch.

The Source Image/s you upload can be used by Image Conditioners like the ControlNet + Control-LoRAs function.

The Source Image/s you upload can also be further modified by various Image Manipulators like the Hand Detailer, Face Detailer, Object Swapper, Face Swapper, and Upscaler functions.

Finally, the Source Image/s you upload can be used as source for the Video Generator, the Colorizer or the Watermarker functions.

Notice that an uploaded Source Image can be used by Image Conditioners, Image Manipulators, Auxiliary Functions, the Video Generator, or all of them in a chain.

The Uploader function also allows you to define a Reference Image, which will be used by some Image Conditioners like the IPAdapter function.

The AP Workflow is designed to greatly improve the quality of your generated images in a number of ways. One of the most effective ways to do so is by modifying or rewriting the prompt you use to generate images.

Prompt Generators can automatically invoke large language models (LLM) and visual language models (VLM) to generate prompts and captions thanks to the following functions.

Prompt Builder

The AP Workflow features a visual and flexible prompt builder.

You can use it to quickly switch between frequently used types and styles of image for the positive prompt, and frequently used negative prompts.

Prompt Enricher

The Prompt Enricher function enriches your positive prompt with additional text generated by a large language model.

AP Workflow allows you to use either OpenAI models (GPT-3.5-Turbo, GPT-4, or GPT-4-Turbo) or open access models (e.g., LLaMA, Mistral, etc.) installed locally.

The use of OpenAI models requires an OpenAI API key. To setup your OpenAI API key, follow these instructions: https://github.com/glibsonoran/Plush-for-ComfyUI?tab=readme-ov-file#use-environment-variables-in-place-of-your-api-key.

You will be charged every time the Prompt Enricher function is enabled and a new queue is processed.

The use of local open access models requires the separate installation of an AI system like LM Studio or Oobabooga WebUI.

Alessandro highly recommends the use of LM Studio and the AP Workflow is configured to use it by default.
Additional details are provided in the Setup for prompt enrichment with LM Studio section of this document.

If you don’t want to rely on an advanced AI system like LM Studio, but you still want the flexibility to serve any open LLM you like, Alessandro recommends the use of llamafile.

The Prompt Enricher function features three example prompts that you can use to enrich their ComfyUI positive prompts: a generic one, one focused on film still generation (AI Cinema), and one focused on collage art images. Multiple switches are in place to choose the preferred system prompt and the preferred AI system to process system prompt and user prompt.

Caption Generator

The AP Workflow offers a wide range of options to automatically caption any image image uploaded via the Uploader function with a Visual Language Model (VLM).

You can use Moondream v1 or v2, OpenAI GPT-4V, or any other VLM you have installed locally and serve via LM Studio or an alternative AI sytstem.

The Caption Generator function replaces any positive prompt you have written with the generated caption. To avoid losing LoRA tags in the process, you can manually define them in the Optional LoRA keywords node. A handy LoRA Info node, located in the Prompt Builder function, can retrieve details for each LoRA in your system from the Civitai database.

This approach is designed to improve the quality of the images generated by the Inpainter without Mask function and others.

Notice that, just like for the Prompt Enricher function, the use of OpenAI models requires an OpenAI API key. To setup your OpenAI API key, follow these instructions: https://github.com/glibsonoran/Plush-for-ComfyUI?tab=readme-ov-file#use-environment-variables-in-place-of-your-api-key.

Training Helper for Caption Generator

The AP Workflow allows you to automatically caption all images in a folder and save the captions in text files. This capability is useful to create a training dataset if you don’t want to use third party solutions like koyha_ss.

To use this capability, you need to activate the Training Helper for Caption Generator function in the Controller function and modify the Single Image or Batch? node in the Uploader function to choose batch images instead of single images.

Once that is done, crucially, you’ll have to queue as many generations as the number of images in the folder you want to caption. To do so, check the Extra Options box in the Queue menu.

The Training Helper for Caption Generator function will generate a caption for each image in the folder you specified, and save the caption in a file named after the image, but with the .txt extension.

By default these caption files are saved in the same folder where the images are located, but you can specify a different folder.

Images generated with the Image Generator (SD) function can be further optimized via a number of advanced and experimental functions.

Free Lunch (v1 and v2)

AI researchers have discovered an optimization technique for Stable Diffusion models that improves the quality of the generated images. The technique has been named “Free Lunch”. Further refinements of this technique have led to the availability of a FreeUv2 node.

For more information, read: https://arxiv.org/abs/2309.11497

You can enable either the FreeUv1 (default) or the FreeUv2 node in the Free Lunch function. Both have been set up following the guidance of @seb_caway, who did extensive testing to establish the best possible configuration.

Notice that the FreeU nodes are not optimized for MPS and DirectML devices. On these systems, the nodes force the image generation to use the CPU rather than the MPS or DirectML devices, considerably slowing down the process.

Kohya Deep Shrink

Deep Shrink is an optimization technique alternative to HighRes Fix, developed by @kohya, promising more consistent and faster results when the target image resolution is outside the training dataset for the choosen diffusion model.

Self-Attention Guidance

Self-Attention Guidance is an optimization technique that adversarially blurs only the regions that diffusion models attend to at each iteration and guides them accordingly.

For more information, read: https://arxiv.org/abs/2210.00939

Perp-Neg

Perp-Neg is a new optimization technique to process the negative prompt, helping the generation of 2D side images, and 3D images.

For more information, read: https://perp-neg.github.io

When you upload an image instead of generating one, you can use it as a conditioning source for a number of Image Conditioners. Each can be activated/deactivated in the Controller section of the workflow.

LoRAs

You can condition the image generation performed with the Image Generator (SD) function thanks to a number of LoRAs.

Each LoRA must be activated in the LoRAs section of the workflow via the LoRA selector in the Efficient Loader node and/or via the additional LoRA Stacker node.

ControlNet and Control-LoRAs

You can further condition the image generation performed with the Image Generator (SD) function thanks to a number of ControlNet and Control-LoRA models. Some of these models are compatible with SDXL models while others are compatible with SD 1.5 models.

The AP Workflow supports the configuration of up to six concurrent ControlNet models, but you can add additional Multi-ControlNet Stack nodes easily if they wish so. Additionally, you can further increase the efficacy of ControlNet models by activating up to six preprocessors.

Each ControlNet model and the optional preprocessor must be defined and manually activated in the ControlNet + Control-LoRAs function.

If you want to see how each ControlNet preprocessor captures the details of the source image, you can use the ControlNet Previews function to visualize up to twelve previews. The ControlNet Previews function can be activated from the Controller function.

IPAdapter

This function enables the use of the IPAdapter technique, to generate variants of the reference image uploaded via the Uploader function.

People use this technique to generate consistent characters in different poses, or to apply a certain style to new subjects.

For more information information on how to use this technique, Alessandro recommends reading the documention of @cubiq’s IPAdapter Plus custom node suite: https://github.com/cubiq/ComfyUI_IPAdapter_plus.

The AP Workflow allows you to specify an attention mask that the IPAdapter should focus on.

The attention mask must be defined in the Uploader function, via the ComfyUI Mask Editor, for the reference image (not the source image).

To force the IPAdapter to consider the attention mask, you must change the switch in the Control Bridge node, inside the IPAdapter function, from Mute/Bypass to Active.

The AP Workflow is capable of generating images at an industrial scale. To help you choose the best images among hundreds or thousands, you can leverage up to three Image Evaluators.

Two of them choose automatically, based on criteria you define, while the third one allows you to manually decide which image must be further processed in the L2 pipeline of the workflow.

Each Image Evaluator can be activated/deactivated in the Controller section of the workflow.

Face Analyzer

The Face Analyzer function allows you to evaluate a batch of generated images and automatically choose the ones that present facial landmarks very similar to the ones in a Reference Image you upload via the Uploader function.

This function is especially useful in conjuction with the new Face Cloner function.

Aesthetic Score Predictor

The AP Workflow features an Aesthetic Score Predictor function, capable of rearranging a batch of generated images based on their aesthetic score.

The aesthetic score is calculated starting from the prompt you have defined in the Prompt Builder function.

The Aesthetic Score Predictor function can be reconfigured to automatically exclude images below a certain aesthetic score. This approach is particularly useful when used in conjunction with the Image Chooser function to automatically filter a number a large number of generated images.

The Aesthetic Score Predictor function is not perfect. You might disagree with the score assigned to the images. In his tests, Alessandro found that the AI model used by this function does a reasonable good job at identifying the top two images in a batch.

Image Chooser

When you generate a batch of images with the Image Generator (SD) function, or you upload a series of images via the Uploader function, you might want to pause the AP Workflow execution and choose a specific image from the batch to further process it with the Image Manipulators.

This is possible thanks to the Image Chooser node. By default, it’s configured in Pass through mode, which doesn’t pause the execution of the workflow.

Change the node mode from Pass through to Always pause or Only pause if batch to enforce a pause and choose the image.

After you generate an image with the Image Generator (SD) function or upload an image with the Uploader function, you can use pass that image through a series of Image Manipulators. Each can be activated/deactivated in the Controller function.

Notice that you can activate multiple Image Manipulators in sequence, creating an image enhancement pipeline.

If every Image Enhancer is activated, the image will be passed through the following functions, in the specified order: Hand Detailer, Face Detailer, Object Swapper, Face Swapper, and finally Upscaler.

SDXL Refiner

This function enables the use of the SDXL 1.0 Refiner model, designed to improve the quality of images generated with the SDXL 1.0 Base model.

It’s useful exclusively in conjunction with the SDXL 1.0 Base model.

Face Detailer

The Face Detailer function identifies small and large faces in the source image, and attempts to improve their aesthetics according to two independent configurations: large faces require a different treatment than small faces.

The Face Detailer function will generate an image after processing small faces and another after also processing large faces.

Hand Detailer

The Hand Detailer function identifies hands in the source image, and attempts to improve their anatomy through two consecutive passes, generating an image after each pass.

Notice that the Hand Detailer function uses dedicated models based on Stable Diffusion 1.5. They work even if your source image has been generated with an SDXL 1.0 base model or a Fine-Tuned SDXL model. The reason for this design choice is that the Hand Detailer function seems to perform better (so far) with a particular model not yet available in the SDXL variant.

The Hand Detailer function uses a new Mesh Graphormer Depth preprocessor node and a new SD1.5 Inpaint Depth Hand ControlNet model.

However, notice that Mesh Graphormer Depth preprocessor node occasionally struggles to identify hands in non-photographic images, so you have the option to revert to the old DW preprocessor node.

Object Swapper

THe Object Swapper function is capable of identifying a wide range of objects and features in the source image thanks to the GroundingDINO technique.

You can describe the feature/s to be found in the source image with natural language.

Once an object/feature has been identified, it will be modified according to the prompt you defined in the Object Swapper function.

Notice that the Object Swapper function uses dedicated ControlNet and T2I models based on Stable Diffusion 1.5. They work even if your source image has been generated with an SDXL 1.0 base model or a Fine-Tuned SDXL model.

The reason for this design choice is that the Object Swapper function seems to perform better (so far) with a particular model not yet available in the SDXL variant.

Notice that the Object Swapper function can be used also to modify the physical aspect of the subjects in the source image.

One of the most requested use case for the Object Swapper function is eyes inpainting.

Face Swapper

The Face Swapper function identifies the face of one or more subjects in the source image, and swaps them with a face of choice. If your source image has multiple faces, you can target the desired one via an index value.

You must upload an image of the face to be swapped in the Face Swapper section of the workflow.

Upscaler (CCSR)

The AP Workflow abandons traditional upscaling approaches to embrace next-generation upscalers. The first one uses the Content Consistent Super-Resolution (CCSR) technique.

This node is easier to configure and it generates exceptional upscaling results, on par or superior to the ones you can obtain with Magnific AI or Topaz Gigapixel.

Upscaler (SUPIR)

The AP Workflow abandons traditional upscaling approaches to embrace next-generation upscalers. The second one uses the SUPIR technique.

Differently from the Upscaler (CCSR) function, the Upscaler SUPIR function allows you to condition the image upscaling process with one or more LoRAs, and it allows to perform “creative upscaling” similar to the one offered by Magnific AI, by lowering the strength of the control_scale_end parameter.

Just like for the Upscaler (CCSR) function, the Upscaler SUPIR function generates exceptional upscaling results, on par or superior to the ones you can obtain with Magnific AI or Topaz Gigapixel.

The AP Workflow offers the capability to inpaint and outpaint a source image loaded via the Uploader function with the inpainting model developed by @lllyasviel for the Fooocus project, and ported to ComfyUI by @acly.

This model generates superior inpainting results and it works with any model (rather than the only the ones that have been specifically trained for inpainting).

The inpainting process can be further conditioned by activating the ControlNet and Control-LoRAs function.

Notice that this form of inpainting is different from the one automatically performed by image manipulators function like Hand Detailer, Face Detailer, Object Swapper, and Face Swapper.

Inpainter without Mask

When no inpainting or outpainting mask is defined, the function will inpaint the entire source image, performing an operation known as img2img.

This approach is useful to reuse the same pose and setting of the source image while changing the subject and environment completely. To achieve that goal, you should set the value of the denoise parameter in the Inpainter node quite high (for example: 0.85).

The Inpainter without Mask function can also be used to add details to a source image without altering its subject and setting. If that’s your goal, you should set the value of the denoise parameter in the Inpainter node to a very low value (for example: 0.20).

Inpainter with Mask

This function allows you to define a mask to only inpaint a specific area of the source image.

The value of the denoise parameter in the Inpainter node should be set low (for example: 0.20) if you want to keep the inpainted area as close as possible to the original.

The inpainting mask must be defined manually in the Uploader function, via the ComfyUI Mask Editor.

Outpainting Mask

This function allows you to define an outer mask to be used by the Inpainter with Mask function. This approach is useful when you want to extend the source image in one or more directions.

When the Outpainting Mask function is active, the value of the denoise parameter in the Inpainter with Mask function must be set to 1.0.

The outpainting mask must be defined manually in the Outpainting Mask function, by configuring the number of pixels to add to the image in every direction.

HighRes Fix

This function enables the use of the HighRes Fix technique, useful to upscale an image while avoiding duplicate subjects when the target resolution is outside of the training dataset for the selected diffusion model.

It’s mainly useful in conjunction with Stable Diffusion 1.5 base and fine-tuned models. Some people use it with SDXL models, too.

The AP Workflow supports the new Stable Diffusion Video model.

This function turns images generated via the Image Generator function, or uploaded via the Uploader function, into animated GIFs and PNGs.

Notice that to use this function on an Apple system, it’s mandatory that you download and install the PyTorch nightly build in the Python virtual environment created by ComfyUI. Follow the instructions provided by Apple.

The AP Workflow includes the following auxiliary functions:

Face Cloner

The Face Cloner function uses the InstantID technique to quickly change the style of any face in a Reference Image you upload via the Uploader function.

To measure the likeness of the generated face to the original one, and select the most similar in a batch of generated faces, you can use the Face Analyzer function.

Colorizer

The AP Workflow includes a Colorizer function, able to colorize a monochrome image uploaded via the Uploader function.

While the AP Workflow allows you to colorize an image in other ways, for example via the Inpainting without Mask function, the Colorizer function is more accurate and significantly faster.

Watermarker

This function generates a copy of the image/s you are processing with the AP Workflow with a text of your choice in the position of your choice.

You can add your fonts to the /ComfyUI/custom_nodes/ComfyUI_Comfyroll_CustomNodes/fonts folder.

Notice that the text of the watermark is not the same text you define in the Add a Note to Generation node in the Configurator function.

Also notice that the Watermarker function is not yet capable of watermarking generated videos.

The AP Workflow was originally designed to study the compounding effect of multiple nodes on the quality of the generated images.

To help you understand the impact of each node and each parameter on the final image, you can use the following functions to conduct sophisticated experiments.

ControlNet Previews

This function allows you to visualize how twelve different ControlNet models captures the details of the image you uploaded via the Uploader function.

Each preprocessor can be configured to load a different ControlNet model, so you are not constrained by the twelve models selected as defaults for the AP Workflow.

The ControlNet Previews function is useful to decide what models to use in the ControlNet + Control-LoRAs function before you commit for a long image generation run. It’s recommended that you activate this function only once, to see the previews, and then deactivate it.

XY Plot

The XY Plot function generates a series of images permutating different generation parameters, according to the configuration you defined.

For maximum flexibility, The AP Workflow features a Manual XY Entry node that can be configured according to the values displayed in the Manual XY Entry Info node. This node is disconnected by default.

Advanced XYZ Plot

In addition to the XY Plot function, which only works with Efficient KSampler Advanced nodes, the AP Workflow includes an Advanced XYZ Plot function, capable of generating a series of images by permutating any parameters in any node of the workflow.

The Advanced XYZ Plot function doesn’t have to be connected to any node in the workflow. Just activate it in the Controller function and configure the node according to your needs.

The AP Workflow includes the following debug capabilities:

Image Comparer

The always-on Image Comparer function is capable of comparing the source image, either uploaded via the Uploader function or generated via the Image Generator (SD) function, with the final image generated by the workflow.

Additional Image Comparer nodes are scattered across the workflow to help you see the incremental changes made to the image throughtout the image generation pipeline.

Image Metadata

All images generated or manipulated with the AP Workflow include metadata about the prompts and the generation parameters used by ComfyUI. That metadata should be readable by A1111 WebUI, Vladmandic SD.Next, SD Prompt Reader, and other applications.

Only the videos generated with the AP Workflow don’t embed the metadata.

Prompt and Parameters Print

Both negative and positive prompt, together with a number of parameters, are printed in the terminal to provide more information about the ongoing image generation.

You can also write a note about the next generation. The note will be printed in the terminal as soon as the ComfyUI queue is processed.

This information can be further saved in a file by adding the appropriate node, if you wish so.

For a more advanced debugging, you might consider sending the output of the node you want to debug to a Beautify node, available as part of the 0246 custom node suite.

The Beautify node is not compatible with every node of every custom node suite and could generate problems of various nature. For this reason, no Beautify node is present in AP Workflow. You have to manually add it where relevant.

Occasionally, AP Workflow users can’t install or import a custom node suite necessary to run the AP Workflow. This happens when you try to use the AP Workflow in a pre-existing ComfyUI environment that you have installed loooooooooong time ago.

If you have a similar problem, be sure to:

1. Have all your packages up to date in your Python virtual environment for ComfyUI.
   
   To do so:
   1. Terminate ComfyUI.
   2. Manually activate its Python virtual environment with source /venv/bin/activate.
   3. Run pip install -U pip to upgrade pip.
   4. Run pip install -U setuptools to upgrade setuptools.
   5. Run pip install -U wheel to upgrade wheel.
   6. Run pip install -U -r requirements.txt to upgrade all the packages in the virtual environment.
2. Check the installation instructions of the custom node suite you are having troubles with.

If you have installed ComfyUI in a new environment and you still fail to install or import a custom node suite, open an issue on the GitHub repository of the author.

By default, the AP Workflow is configured to use SDXL models for image generation. To switch to SD 1.5 models, you must follow these steps:

1. In the Controller function, enable the Image Generator (SD) function and disable the Refiner (for SDXL) function.
2. In the Configurator function, change the ckpt_name to an SD1.5 model, change model_version to SDv1 512px, set refiner_start to 1, change the aspect_ratio to 1:1.
3. In the LoRAs function, change the vae_name to vae-ft-mse-840000-ema-pruned.safesensors, or any other VAE model optimized for Stable Diffusion 1.5

The AP Workflow allows you to enrich their positive prompt with additional text generated by a locally-installed open access large language model.

The AP Workflow supports this feature through the integration with LM Studio.

Any model supported by LM Studio can be used by the AP Workflow, including all models at the top of the HuggingFace LLM Leaderboard.

Guidance on how to install and configure LM Studio is beyond the scope of this document and you should refer to the product documentation for more information.

Once LM Studio is installed and configured, you must load the LLM of choice, assign to it the appropriate preset, and activate the Local Inference Server.

Alessandro usually works with LLaMA 2 fine-tuned models and the Meta AI LLaMA 2 Chat preset.

Are you trying to replicate A1111 WebUI, Vladmandic SD.Next, or Invoke AI?

Alessandro never intended to recreate those UIs in ComfyUI and has no plan to do so in the future.
While the AP Workflow enables some of the capabilities offered by those UIs, its philosophy and goals are very different. Read below.

Why are you using ComfyUI instead of easier-to-maintain solutions like A1111 WebUI, Vladmandic SD.Next, or Invoke AI?

1. Alessandro wanted to learn, and help others learn, the SDXL architecture, understanding what goes where and how different building blocks influence the generation. A1111 WebUI and similar tools makes it harder. With ComfyUI, you knows exactly what’s happening. So AP Workflow is, first and foremost, a learning tool.
2. While Alessandro considers A1111 WebUi and similar toos invaluable, and he’s grateful for their gift to the community, he finds their interfaces chaotic. He wanted to explore alternative design layouts. Many people might argue that ComfyUI is even more chaotic than A1111 WebUI or that AP Workflow, in particular, is more chaotic than A1111 WebUI.
   Ultimately, different brains process information in different ways, and some people prefer one approach over the other. Some people find node systems easier to work with than more standard UIs.
3. Alessandro served in the enterprise IT industry for over two decades. ComfyUI allowed him to demostrate how AI models paired with automation can be used to create complex image generation pipelines useful in certain industrial applications. This is not currently possible with A1111 WebUI and similar solutions.
4. The most sophisticated AI systems we have today (Midjourney, ChatGPT, etc.) don’t generate images or text by simply processing the user prompt. They depend on complex pipelines and/or Mixture of Experts (MoE) which enrich the user prompt and process it in many different ways. Alessandro’s long-term goal is to use ComfyUI to create multi-modal pipelines that can produce digital outputs as good as the ones from the AI systems mentioned above, without human intervention. AP Workflow 5.0 was the first step in that direction. The goal is not attainable with A1111 WebUI and similar solutions as they are implemented today.

The AP Workflow is provided as is, for research and education purposes only.

However, if your company wants to build commercial solutions on top of ComfyUI and you need help with this workflow, you could work with Alessandro on your specific challenge.

The AP Workflow wouldn’t exist without the dozen of custom nodes created by very generous members of the AI community.

In particular, special thanks to:

@rgthree:

• His Reroute nodes are the most flexible reroute node you can find among custom node suites.
• His Context Big and Context Switch nodes are the best custom nodes available today to branch out an expansive workflow.
• His Fast Groups Muter/Bypasser nodes offer the ultimate flexibility in creating customized control switches.
• His Image Comparer node is an exceptional help in inspecting the image generation.

@receyuki:

• He evolved his SD Parameter Generator node to support the many needs of the AP Workflow, working above and beyond to deliver the ultimate control panel for complex ComfyUI workflows.

@kijai:

• His Set and Get nodes allow the removal of most wires in the workflow without sacrificing the capability to understand the flow of information across the workflow.
• His CCSR and SUPIR nodes makes exceptional upscaling possible.
• His DDColor node powers the Colorizer function of the workflow.
• His MoondreamQuery node powers a key part of the Caption Generator function of the workflow.

@cubiq:

• His implementation of the IP Adapter technique allows all of us to do exceptional things with Stable Diffusion.

@ltdrdata:

• The nodes in his Impact Pack power many Image Manipulators functions in the workflow.
• His ComfyUI Manager is critical to manage the myriad of package and model dependencies in the workflow.

@glibsonoran:

• He evolved his Plush custom node suite to support the many needs of the AP Workflow and now his nodes power the Prompt Enricher and the Caption Generator functions.

@acly:

• His nodes to support the Fooocus inpaint model power the Inpainting with Mask function of this workflow.

@talesofai:

• His XYZ Plot node is the most elegant and powerful implementation of a plot function to date.

@jags111:

• His fork of LucianoCirino’s nodes allows AP Workflow to keep offering a great XY Plot function.

@LucianoCirino:

• His XY Plot function is the very reason why Alessandro started working on this workflow.

Thanks to all of you, and to all other custom node creators for their help in debugging and enhancing their great nodes.