Preheat your oven to 350°F (175°C). Grease and flour a 9x13-inch baking pan.
In a medium bowl, whisk together the flour, baking powder, baking soda, salt, and cinnamon. Set aside.
In a large mixing bowl, cream together the softened butter, granulated sugar, and brown sugar until light and fluffy. Add the eggs one at a time, mixing well after each addition. Stir in the vanilla extract.
Gradually add the flour mixture to the butter mixture, alternating with the buttermilk. Begin and end with the flour mixture. Gently fold in the diced peaches until evenly distributed.
Pour the batter into the prepared baking pan and smooth the top. Bake for 30-35 minutes, or until a toothpick inserted in the center comes out clean. Allow the cake to cool completely in the pan on a wire rack.
While the cake is cooling, prepare the brown sugar frosting. In a saucepan over medium heat, melt the butter and brown sugar together, stirring constantly until the mixture comes to a boil. Remove from heat and stir in the milk. Let it cool for about 10 minutes.
Gradually whisk in the powdered sugar and vanilla extract until smooth and spreadable.
Once the cake is completely cool, spread the brown sugar frosting evenly over the top.