LoRA Overfitting vs Undertraining: Signs, Checkpoint Grids, What to Change First

Mid-run, the hard question is not “is my loss going down?” It is whether the adapter baked your training photos, never learned the concept, or is drifting for a different reason. This guide owns that diagnosis: a fixed-seed checkpoint grid, a symptom table, and a change-first order so you turn one knob at a time. If likeness wanders seed to seed, stop here and open fixing identity drift—that failure is not overtraining.

What this guide covers (and what it does not)

Scope is mid-training diagnosis for character LoRAs (with a brief nod to style): symptom recognition, checkpoint grids, and a recovery decision tree. Out of scope: folder packing, anime tag taxonomy, full base-model picking, and a deep steps-formula essay. You will leave with three-way definitions, a symptom table, a copyable grid recipe, and a cheap-to-expensive change order.

For steps math see how many images to train a LoRA. For packing after you retrain, see Kohya / Civitai packaging. Anime tag specifics live on the WD14 / Danbooru guide.

Overfitting vs undertraining vs identity drift

Symptom table (what you are actually seeing)

High inference weight can mimic overtraining. Always weight-grid before declaring the run burned. Community recovery order: lower weight → earlier save → then retrain knobs.

Do not trust loss alone

Kohya / sd-scripts support periodic sample generation, and current sd-scripts also document a validation split (--validation_split, --validate_every_n_steps / --validate_every_n_epochs). Diffusion training loss is noisy across timesteps and batches. A falling train loss with worse held-out samples is a warning, not a paradox. Deciding evidence is images on held-out / novel prompts across checkpoints.

Diagnostic grid: how to sample checkpoints

Save often enough that a good middle exists

Use --save_every_n_steps or --save_every_n_epochs (Kohya GUI: Save every N steps / Save every N epochs). Example from the docs: 1000 total steps with a save every 200 yields files at 200/400/600/800. Community habit is often save every 1 epoch so you can pick before bake—practice, not law. The file named final / last epoch has no special claim to quality.

Generate fixed samples during training (optional but powerful)

From sd-scripts sample output during training: --sample_every_n_steps / --sample_every_n_epochs (epochs win if both are set), --sample_at_first for a pre-train baseline, and a --sample_prompts file. One prompt per line; flags include --n negative, --w / --h size, --d seed, --l CFG, --s steps. Community examples sample every ~300 steps or every epoch—cite as an example, not a magic interval. In-train samples can look better than real use; still useful for trends, then re-grid offline.

# sample_prompts.txt — placeholder trigger, fixed seeds
mychar woman, standing in a rainy city street, medium shot --d 42 --s 28 --l 5
mychar woman, full body, red dress, studio lighting --d 42 --s 28 --l 5
a woman in a kitchen, blue hoodie, looking at camera --d 7 --s 28 --l 5

The offline checkpoint grid (required recipe)

1. Same base checkpoint the LoRA was trained for (wrong family ≠ overfit).
2. Disable other LoRAs / IP-Adapter / face restore / prompt expanders for the test.
3. Fixed seeds across all rows.
4. Prompt set (minimum five): near-training portrait; novel environment; change a repeated attribute (outfit or bg); difficult angle / full-body if the set was mostly close-ups; control with no trigger (leakage check).
5. Columns: no-LoRA baseline | early ckpt | mid | late/final.
6. Rows: at least two adapter weights (moderate and high). Scales differ by network type / rank / loader—verify on your stack; do not prescribe “always 1.0.”
7. Read the grid: later file more rigid / leaky without better novel-prompt control → overfit; all files weak → undertrain / data / wrong arch; likeness wanders at every ckpt → drift.

                    | no LoRA | early | mid | late
weight moderate     |   …     |  …    |  …  |  …
weight high         |   …     |  …    |  …  |  …
(prompts × seeds fixed down the page)

What to change first (decision tree)

Cheap → expensive. One change per retrain.

1. 01

   Correct model family + trigger + LoRA loaded?

   Wrong architecture or a missing adapter looks like undertraining. Fix that before touching LR.
2. 02

   Lower inference weight (weight grid)

   Looks fixed? Document the recommended weight and stop. High weight often mimics bake.
3. 03

   Compare early / mid / late on the fixed grid

   Mid better than late on novel prompts? Ship the mid file. Final has no special claim.
4. 04

   Bias present from the first useful checkpoint?

   DATA / CAPTIONS first: duplicates, one framing, uncaptioned outfit or background. Training longer will not invent missing variety.
5. 05

   Concept appears then rigidifies over time?

   Reduce training pressure: stop earlier → lower LR → fewer repeats/epochs → lower network_dim if memorizing a tiny set → reconsider aggressive text-encoder training.
6. 06

   Concept never appears?

   Verify captions load, architecture, and resolution match. Then cautiously increase exposure or LR. Do not auto-double steps. Too-few-image symptoms are covered in the image-count guide.
7. 07

   Likeness wanders seed-to-seed?

   Exit this article → identity drift.

Data and captions (usually the real baked bg/pose fix)

Near-duplicates and single-framing sets teach pose and background as part of the concept. Caption what should vary (pose, outfit, background, camera) so they do not glue to the trigger—see the captioning guide. One-reference expansion with varied poses and backgrounds reduces bake risk; see creating a dataset from one image.

Weak (bg/outfit bind to trigger):
mychar woman

Stronger (name what should stay controllable):
mychar woman, standing in a kitchen, blue hoodie, looking at camera, medium shot

Learning rate (starting ranges — verify)

• Kohya wiki defaults often cited: main/UNet LR 1e-4; text encoder LR 5e-5. TE is usually lower because TE updates hit the whole model and overfit easily.
• Too high → harsh/rigid quickly. Too low → looks undertrained for ages.
• Optimizer pairing matters: a rate from AdamW may not translate to Adafactor / DAdaptation / Prodigy (those often ignore or reinterpret the LR field—check your optimizer docs).
• Larger batch with the same LR can underfeed versus batch-1 long runs; many trainers scale LR when scaling batch—treat as a hypothesis to test, not a formula to worship.
• Effective strength scales with network_alpha / network_rank (alpha 16 / rank 32 ≈ 0.5×). Equal alpha = rank turns that scaling off. Alpha should not exceed rank (wiki warning).

Steps / repeats / epochs

Reuse images × repeats × epochs ÷ batch from the image-count post—do not invent a “train N steps per image” law. If every checkpoint is weak, inspect data, captions, architecture, and LR before doubling steps. Stop when later checkpoints gain training-photo resemblance but lose flexibility (repeated bg/pose, baked clothes, harsh texture, prompt ignore).

Network rank (dim) and capacity

Higher dim = more capacity, larger file, and more risk of storing unnecessary / memorized detail. Kohya wiki: often set up to ~128; reports that 32 can be enough; trial starts from ~2–8; default 8. Community character starts often land in an 8–32 band; go higher only for broad/complex concepts, and verify on your set. Lower rank if a small set memorizes; too-low rank can underfit complex concepts. Guides disagree on face-specific ranks—prefer checkpoint evidence over doctrine.

Text encoder training and stop-TE

TE training strongly affects the whole net and easily overfits. Stop text encoder training at a percentage of total steps is an explicit anti-overfit lever while UNet continues. Aggressive TE can bind the trigger tightly but hurt editability—change only after data / weight / checkpoint checks.

Flux vs SDXL: over-training can look different

Community discussions often associate Flux failure modes with waxy / plastic skin and brittle adapters (guidance, sampler, and dataset quality confounders too—do not blame the LoRA alone). SDXL overcook talk more often mentions oversharpen / harsh microcontrast / burned detail. Settings do not copy 1:1 across families (Flux transformer vs SDXL UNet; different loaders). Verify on the exact trainer doc for that family. Full base-choice framing: Flux vs SDXL LoRA training.

Worked mini-scenarios

1. Baked bedroom wall. Weight OK; late ckpt worse than mid on novel rooms → ship mid. Captions never mentioned background → retrain with varied bgs and captioned scenes.
2. Weak face at 1.0, improves through all epochs without rigidity. Undertrain lane: confirm captions load, then modest more exposure—not a rank spike first.
3. Face morphs every seed; outfits stay flexible. Not overfit → identity drift.