Preheat oven to 375 degrees. In a medium bowl, whisk together flour, cocoa powder, baking soda, and salt; set aside.

In the bowl of an electric mixer fitted with the paddle attachment, cream together butter and sugar until light and fluffy, about 2 minutes. Add egg, milk, and vanilla, and beat until well combined. Add reserved flour mixture; mix on low speed until combined.

Using a tablespoon or 1 3/4-inch ice cream scoop, drop dough onto ungreased baking sheets, about 2 inches apart. Bake until cookies begin to spread and become firm, 10 to 12 minutes.

Remove baking sheets from oven, and place a marshmallow, cut-side down, in the center of each cookie, pressing down slightly. Return to oven, and continue baking until marshmallows begin to melt, 2 to 2 1/2 minutes. Transfer cookies to a wire rack to cool completely before frosting.

Spread about 1 tablespoon of frosting over each marshmallow, starting in the center and continuing outward until marshmallow is covered.