Preheat oven to 350°F. Lightly oil a bundt pan, making sure to get into all the nooks and crannies.
Combine sugar, olive oil, eggs, buttermilk, and vanilla in the bowl of a stand mixer fitted with the whisk attachment. Blend until combined.
Add dry ingredients to the stand mixer bowl (flour, baking soda, cinnamon, nutmeg, cardamom, and ginger) and whisk to combine the batter. Pour the batter into the prepared bundt pan and place in the oven.
Bake for 35-45 minutes or until the cake is cooked through.
While the cake is baking, combine sugar and vinegar in a heavy bottom saucepan. Stir mixture and bring to a boil over medium-high heat. Cook sugar for 10 minutes and remove from the heat.
To the caramel sauce, add butter and butter milk (be careful, the mixture will bubble) and whish to combine. Set aside to cool (just to room temperature).
Once the cake is done, let cool for 30 minutes.
Drizzle caramel over the cake and top with pecans before serving.