In a small mixing bowl, whisk together the vinegar, honey, and 2 Tbsp. canola oil. Gently fold in the plum, shallot, capers, and 1/4 cup pickled peppers.
Preheat a grill or grill pan to medium-high. Toss the mushrooms with the remaining 2 Tbsp. canola oil, and season with salt and pepper. Transfer mushrooms to the grill, and cook, turning once, until browned on the outside and just tender, about 3 min. per side.
Transfer the mushrooms to a serving platter, and drizzle all over with the plum mixture. Sprinkle with the feta and basil, and serve with remaining pickled peppers.
In a medium saucepan over medium-high heat, bring 2/3 cup champagne vinegar, 1/4 cup sugar, and 2 Tbsp. salt to a boil over medium-high heat. Put 1 banana pepper and 1 Hungarian pepper (both stemmed, seeded, and sliced into thin rings) into a heatproof storage container with a fitted lid. Pour the hot vinegar mixture over the peppers, fully submerging the peppers in liquid. Let cool completely at room temperature, then cover and refrigerate for up to 1 month.
Recipe by Stephanie Izard, the owner of Girl & the Goat restaurant in Chicago, and the creator of This Little Goat cooking sauces and spices.