Your own deduction seems to suggest it’s not a scam. On reading the original post, I thought, they said it was dirty, asked for a refund to compensate, but still stayed the night.
If they left right away, what did they gain from this? Especially, as you say, dragging the kids along and everything.
The only thing that makes this look like a scam is this:
It’s probably a bit late now, but the only way they can be found out is to report this behavior to Airbnb and get them to check if this guest has a left a pattern with previous hosts (who you don’t know of because of the mutual non-review deal) but some of whom Airbnb, if they get to it, will have to contact to find out.
Still, this is all far-fetched, it was probably the fussy wife who didn’t like the color of your walls.