With the advice of a professional beekeeper, we grabbed our tools and went back to look into what was going on with the fourth hive. Literally, we needed to look inside that queen cup. Again, here is what it looked like from the outside (larger cell at the bottom of the frame) and now what we found it looks like inside:
If there had been a larva inside, we would have had to take some action to prevent a swarm. There was nothing inside, so we don't need to do anything immediately. We will be watching this queen and her brood pattern, to see if she is healthy enough, though. I suspect we will need to replace her but will not jump to that conclusion just yet. We saw her again - can you find her in the photo below?