Preheat your oven to 325°F (163°C) and prepare a 9x5 inch loaf pan by greasing and flouring it lightly.

Gently warm the whole milk in a saucepan over medium heat, adding the dried lavender buds until simmering. Remove from heat and let infuse for about 10 minutes.

Cream together the softened butter and white sugar until smooth and fluffy, then beat in the eggs one at a time.

In a separate bowl, whisk together flour, baking powder, and salt. Set aside.

Add the lavender-infused milk to the butter-sugar blend, then mix in the dry ingredients just until combined.

Pour the batter into the prepared loaf pan and bake for 45-50 minutes until a toothpick comes out clean.

Let the bread cool in the pan for about 10 minutes, then transfer it to a wire rack.

Prepare the lemon glaze by whisking together powdered sugar, lemon juice, vanilla extract, and milk until smooth.

Drizzle the lemon glaze over the warm bread, allowing it to set before slicing.