Preheat the oven to 425°F.
On a large baking sheet, toss the potatoes with 2 tablespoons olive oil, season with salt and black pepper. Add the lemon halves and roast for 20 minutes, or until tender.
In a bowl, combine 2 tablespoons olive oil with the chicken, dried oregano, paprika, shallot, salt, pepper, and chili flakes.
Remove the potatoes from the oven, nestle the chicken among them, and arrange the onions, garlic, and lemon slices around. Place a slice of butter on each chicken piece. Return to the oven for 20-25 minutes, until the chicken is cooked through.
For the dressing, mix the remaining 1/3 cup olive oil with red wine vinegar, herbs, pepperoncini, and olives. Finely chop the lemon slices, discarding seeds, and mash the garlic into a paste. Stir both into the dressing. Season with red pepper and salt.
Pour the dressing over the cooked chicken and potatoes, break the feta over the top, and garnish with fresh herbs.