Instanced Gaussian Splat Sunflower Field in Houdini

Training

Capture

My target is one plant in a corner of the field. There was still a lot in the way, and the flower faced into another patch I couldn’t get into and take photo from.

128 photoson my phone, orbiting the plant a few rounds, exposure and focus locked.
No proper gear, no controlled space, plus sun and wind. The quality is so-so. Let’s see how it goes.

I didn’t want to pull the plant out for the scan. Also once out of water from soil, the leaves start drooping fast during the capture. Inconsistency between photos is far more damaging in gsplats training.

Camera Alignment

Into RealityScan Desktop and a quick Align Images.
100 pics landed in the main component with valid poses. Exported as COLMAP.

Staged Training

In Houdini TOPs, trained in three passes: data downscale 4 / 2 / 1, for 7,500 / 15,000 / 10,000 steps.
The third pass wasn’t necessary. The first two took under 30min together. The third ran at full resolution, ate a ton of memory, took almost two hours, and didn’t look any distinctly better.

Gsplats Viewer

Cleanup Gsplats

Raw Import

Raw Gaussian splats imported into Houdini

The raw splats right after import. The sunflower is hiding in that clump in the middle.
The orientation needs to be corrected, and everything around it has to go.

Reorient and Clip

Turned upright, then clipped everything outside the bounding box.

Prune Disconnected Pieces

Splats before pruning disconnected pieces — Before

Only the largest connected piece kept — N=1

vex vex

int n = chi("n");
int counts[], classes[];

for (int i = 0; i < npoints(0); i++) {
    int c = point(1, "class", i);
    int f = find(classes, c);
    if (f < 0) { append(classes, c); append(counts, 1); }
    else counts[f] += 1;
}

int order[] = reverse(argsort(counts));
int keep[];
for (int i = 0; i < min(n, len(order)); i++)
    append(keep, classes[order[i]]);

for (int i = 0; i < npoints(0); i++)
    if (find(keep, point(1, "class", i)) < 0)
        removepoint(0, i);

Pruned the floating bits that aren’t connected to the N biggest pieces by point count.

Alpha Prune

Dropped every splat with GS_Alpha under 0.1. That alone gets rid of a huge chunk of them.

Density and Alpha

vex vex

int nei = neighbourcount(1, @ptnum);
if (nei < `ch("../connectadjacentpieces1/maxsearchpoints")`
            *chf('conn_threshold')
          && @GS_Alpha < chf('alpha_threshold'))
    removepoint(0, @ptnum);

Density and alpha together, for finer control. A splat only goes if it has few neighbours and low alpha, so the look mostly holds up.

Stem and Leaves

Split the flower head from the stem and leaves, then ran one more density cull on just the stem and leaves.

Flower Disc Touch-up

Touched up the shape and color of the flower disc by hand to hide the flaws.

Variation

Leaves

Leaf groups painted with GroupPaint — Painted a group for each main leaf with Group Paint node.

Per-leaf random rotation — For each leaf, I find the lowest point (the root) and the point farthest from it (the tip), then build a random rotation matrix.
Roll and pitch are weighted down so it’s mostly yaw. Otherwise the leaves end up sticking straight up in weird poses.

PDG Variations

Ten sunflower variations generated with PDG

The flower head just gets a random ±20° rotation as a whole.

10 variations via PDG, each tagged with a different ___pdg attribute.
That’s the attribute instancing uses later to tell them apart.

Scatter

Foreground, midground and background instance points

Points for the foreground (the four flowers blocking the camera), midground (the hero flowers facing the camera) and background (the other ~3,000 scattered plants). Each one gets a random yaw, a random scale, and a random ___pdg attribute that picks its variation.

TIP

Show the instances as bounding boxes, or the Vulkan viewport starts to lag.