Flux vs SDXL for LoRA Training: Which Base Model Should You Use?

Flux.1 and SDXL are the two base models most character LoRAs are trained on today. Flux is newer, larger, and better at following prompts. SDXL is faster, cheaper to train, and sits on a much bigger ecosystem of checkpoints and ControlNets. Neither is the right answer for everyone. This comparison lays out the differences that actually change your results: architecture, hardware needs, dataset and caption requirements, training time, output quality, and licensing.

Flux vs SDXL at a glance

Architecture and what it means for LoRAs

SDXL is a latent diffusion model with a convolutional UNet. LoRA adapters attach to the attention layers of that UNet and optionally to the two CLIP text encoders. Flux replaces the UNet with a transformer trained as a rectified-flow model, and its LoRAs attach to the transformer’s attention and projection layers. In practice this means Flux LoRAs learn faster and generalize better from fewer images, but each training step pushes far more parameters and needs more memory.

Flux’s T5-XXL encoder is the other major difference. It reads full sentences with grammar, which is why Flux follows detailed prompts so well and why Flux training captions should be written as sentences rather than tag lists. See the captioning guide for formats per model.

Hardware requirements

SDXL LoRA training fits on a 12 GB consumer card with default settings and on 8 GB with gradient checkpointing and 8-bit optimizers. Flux at bf16 precision wants 24 GB. Community trainers such as ai-toolkit and kohya sd-scripts bring Flux down to 12–16 GB by loading the transformer in fp8 or with quantization, at some cost in speed and fine detail. If you rent GPUs, an SDXL character LoRA typically trains in 20–40 minutes on an A10 or 3090-class card, while Flux runs closer to 1–2 hours for a comparable dataset.

Dataset and caption differences

Because Flux learns fast, it also learns your mistakes fast. One inconsistent image in a set of twelve has more influence than one in thirty. Curate aggressively and confirm every image agrees with the reference; our guide to identity drift lists what to check. For the count ranges behind both columns, see how many images a LoRA needs.

Starting training settings

Output quality and flexibility

On photorealistic faces, Flux LoRAs are generally more faithful and hold identity across a wider range of prompts, especially scenes with multiple subjects or readable text. SDXL LoRAs remain excellent for stylized and illustrated characters, where its huge library of fine-tuned checkpoints lets you retarget the same LoRA to anime, painterly, or 3D looks. SDXL also has mature ControlNet and IP-Adapter support for pose and composition control, while Flux’s control tooling is newer.

Licensing for commercial work

This is where many creators get surprised. Flux.1 dev is released under the FLUX.1 [dev] Non-Commercial License, which restricts commercial use of the model and its outputs without a separate agreement from Black Forest Labs. Flux.1 schnell is Apache 2.0 and free for commercial use, but it is a distilled four-step model that is harder to fine-tune well. SDXL uses the CreativeML Open RAIL++-M license, which permits commercial use subject to its use restrictions. If you plan to sell outputs or ship a product, verify the current license text before choosing a base.

Which should you choose?

One dataset, both models

You do not have to pick before you build the dataset. A well-curated 20–30 image set at 1024 px with trigger-first captions trains on both. Write captions as sentences; SDXL handles them fine and Flux prefers them. LoRA Dataset exports exactly that format: identity-consistent variations of one reference at 1024 px, each with a natural-language caption starting with your trigger token, so you can train a Flux version and an SDXL version from the same folder and compare.