Flux AI is the best open-source AI image generator you can run locally on your PC (As of August 2024). However, the 12-billion parameter model requires high VRAM to run. Don’t have a beefy GPU card? Don’t worry. You can now run Flux AI with a GPU as low as 6 GB VRAM.
You will need to use SD Forge WebUI.
What is Forge?
SD Forge is a fork of the popular AUTOMATIC1111 Stable Diffusion WebUI. The backend was rewritten to optimize speed and GPU VRAM consumption. If you are familiar with A1111, it is easy to switch to using Forge.
See the Guide for installing Forge.
If you are new to Stable Diffusion, check out the Quick Start Guide.
Take the Stable Diffusion course if you want to build solid skills and understanding.
What is the low VRAM NF4 Flux model?
The NF4 Flux uses a sophisticated 4-bit quantization method to achieve a smaller memory footprint and faster speed. It is based on the QLoRA method developed for large language models like ChatGPT. The speed-up is more significant in low VRAM machines.
Use Flux AI NF4 model on Forge
Step 1: Update Flux
The support for Flux is relatively new. You will need to update Forge before using it.
If you use the standalone installation package, double-click the file update.bat in the Forge installation folder webui_forge_cuXXX_torchXXX.
Step 2: Download the Flux AI model
There are two download options:
- Flux1 dev FP8 – This checkpoint file is the same as the one for ComfyUI.
- Flux1 dev NF4 – This version is smaller and faster if you have a low VRAM machine (6GB/8GB/12GB)
Download one of them and put it in the folder webui_forge_cuXXX_torchXXX > webui > models > Stable-diffusion.
Tips: If you have already downloaded the FP8 model for ComfyUI and is happy with the VRAM usage, you don’t need to download the NF4 model.
Step 3: Generate an image
In SD Forge WebUI, select Flux in UI. Select a flux model in the Checkpoint dropdown menu.
Enter a prompt, e.g.
A beautiful witch with white wings like angel, empty hall, victorian building, large window, dramatic light, looking at viewer
Set the image size:
- 1024 x 1024 (Square)
- 1216 x 832 (3:2)
- 1344 x 768 (16:9)
Click Generate to generate an image.
Note: Negative prompts are not supported in the Flux model. Set the CFG scale to 1.
Tips
You can reduce the VRAM usage even more by generating smaller, SD 1.5 size images.
- 512 x 512 (1:1)
- 512 x 768 (2:3)
Useful resources
Flux: Loading T5, CLIP + new VAE UI in SD Forge
[Major Update] BitsandBytes Guidelines and Flux
NF4 Checkpoint: flux1-dev-bnb-nf4-v2.safetensors
FP8 Checkpoint: flux1-dev-fp8.safetensors