Preheat oven to 350℉
Cut the bread into cubes and lay them on a baking sheet so they are not touching one another. Add the bread to the oven and toast for 4 minutes or until it is ever so crisp. Do not over-toast the bread. Remove from and oven and set aside.
In a large bowl, combine the heavy cream, egg yolks, sugar, vanilla, cinnamon, and nutmeg.
Add the lightly toasted bread to the mixture and using a spoon make sure all the pieces are covered. Cover with foil and let it sit for 30 minutes.
In a greased 9x13 pan, add half of the bread mixture and evenly across the pan. Scoop the sweet potato puree on top and evenly spread it out, Ad the remaining bread on top and spread evenly so there are essentially 3 layers to the pan (bread, sweet potato, bread)
Cover and bake for 30 minutes