Boil the potatoes. Start by bringing a large pot of salted water to a boil over high heat. Carefully add the petite white or red potatoes and cook for about 15 minutes, or until they are fork-tender but not falling apart. Once cooked, drain the potatoes in a colander and allow them to cool slightly.

Prepare the dressing. While the potatoes are cooking, grab a medium jar with a tight-fitting lid. Combine the extra-virgin olive oil, red wine vinegar, minced garlic, dry mustard, dried thyme, dried oregano, a pinch of salt, and black pepper. Seal the jar and shake vigorously for about 30 seconds, until the dressing is well emulsified and fragrant. Set it aside.

Toss the warm potatoes. Once the potatoes are slightly cooled, transfer them back into the pot or a large mixing bowl. Drizzle the reserved caper brine over the warm potatoes and gently toss them to coat.

Combine the ingredients. Add the Kalamata olives, sun-dried tomatoes, capers, red onion, and fresh dill to the potatoes in the bowl. Gently fold all the ingredients together using a spatula or wooden spoon.

Dress the salad. Drizzle the prepared dressing over the potato mixture and toss gently until everything is well combined and evenly coated. Letting the salad rest for about 15 minutes will enhance the flavors.

Add the feta and serve. Just before serving, sprinkle the creamy feta cheese on top of the salad. Serve immediately or chill it in the refrigerator for up to 4 days.