We know that Joseph was more than a bit concerned about his future with Mary. We know that when she was conceived they were not yet married -- and that the baby wasn't his. We know from the gospel narratives that he considered divorcing her, as was his right under Jewish law. It is immediately clear from these circumstances, that Jesus began his life - even in Mary's womb, in difficult and dangerous circumstances.
We might want to object to this idea. We may say that the Holy Family was special because of their special mission and vocation-- and that the rules don't apply to them -- that their case was miraculous and so does not compare with our own. And so it was. BUT the purpose of God's miracles in our midst is always to show us something about the way God works in the world. None of us is the blessed Mary nor the righteous Joseph. But by showing us these people in the proper context of their world, a difficult one as ours is in its own way, holy scripture challenges us to live up to the miracles that we see in their pages. This is how the "spirit" of Christmas becomes real.