Preheat your oven to 350°F (175°C) and line a baking sheet with parchment paper.
In a medium bowl, whisk together the flour, baking powder, and salt. Set aside.
In a large bowl, cream the softened butter and brown sugar together until light and fluffy, about 2-3 minutes.
Add the egg, vanilla extract, and cinnamon to the butter mixture, and mix until well combined.
Gradually add the dry ingredients to the wet mixture, alternating with the milk, and mix until just combined.
Divide the dough in half and shape each half into a rectangle. Wrap in plastic wrap and refrigerate for at least 30 minutes.
Once chilled, roll out one rectangle of dough on a floured surface to about 1/4 inch thick. Cut into rectangles (about 3x4 inches).
Place a teaspoon of brown sugar filling in the center of half the rectangles. Top with another rectangle and seal the edges with a fork.
Repeat with the remaining dough and filling.
Place the cookies on the prepared baking sheet and bake for 12-15 minutes, or until lightly golden.
Allow the cookies to cool completely on a wire rack.
For the icing, mix powdered sugar with milk until smooth. Drizzle over cooled cookies and add sprinkles if desired.