r/computervision • u/Last-Luck-6077 • 2h ago
Discussion Re-randomizing object placement every frame to build a detection dataset. The labels come from the engine, so re-annotation is free, but I'm not sure the variety is worth what it costs.
Same camera position in all five frames of the first clip. The vehicles, containers and people are re-scattered before every render, and the time of day moves with them. The second clip shows the same frames beside their per-instance identity images: labels are derived from the renderer's instance buffer, so a new arrangement costs nothing to annotate.
That's the appeal. Here's what bothers me about it.
Re-placing everything per frame means consecutive frames are no longer a sequence. Nothing is tracked, nothing persists, so the corpus is useless for anything temporal. And frames from the same scene still look alike enough that near-duplicates across a train/val split will flatter your validation score.
I trained a person detector on eight captures like this and nothing else. Held-out synthetic tiles: 0.767 recall. 120 frames of real drone footage: 0.350. The split wasn't careless, held-out camera zones, so that gap isn't leakage.
So, for people doing this seriously:
- Do you re-randomize per frame, or place once and move the camera? What made you pick?
- How do you check for near-duplicates before trusting a validation number?
- Anyone got synthetic-only recall meaningfully above 0.4 on real footage? What did it?
2
u/_d0s_ 1h ago
Before explaining your approach you should define your goal. Is this a dataset to detect persons? I assume it's forsym to real transfer. (Transfer to what scenes exactly?) You need to evaluate what works on your target distribution. In my experience diversity beats reality. You could compare to training on https://unity-technologies.github.io/PeopleSansPeople/
Your environment probably has very limited variety in textures, poses and body shapes. In comparison to public datasets you probably won't gain much, because millions of images with real humans and annotated bounding boxes exist.
Again, specify your goals.