In a saucepan, combine sugar, cocoa powder, salt, and water. Stir together over medium-low heat and bring to a boil.
Cook for 2 minutes and then stir in the milk.
Bring the mixture to serving temperature, but not boiling.
Remove from heat and stir in vanilla extract.
Whisk until frothy and pour into serving glasses or mugs.
Garnish, if desired, with miniature marshmallows, whipped cream, a cinnamon stick, or a candy cane.