Place the oven rack in the lower third of the oven and set the temperature to 325 degrees. Spray a 9-inch springform pan with gluten-free cooking spray and place on a foil lined, rimed baking sheet.
Place apple slices on a microwave safe plate or pie dish, cover and microwave on high for 3 minutes, or until apples are slightly tender and pliable. Drizzle with brandy and lemon juice and let cool for 15 minutes.
Place 3/4 cup of gluten-free flour blend, almond flour, 1 cup of sugar, baking powder, salt and xanthan gum in a large mixing bowl. Whisk to combine. Add one egg, oil, milk and vanilla extract and whisk until smooth. Scoop out one cup of the batter and place in another medium bowl.
Add egg yolks to the larger portion of the batter and whisk to combine. Fold in the cooled apple slices and pour mixture into the prepared pan, smoothing the top with an offset (or rubber) spatula.
Whisk remaining 2 tablespoons of gluten-free flour blend into the remaining batter and smooth batter evenly over the apple mixture in the pan, taking care to spread it all the way to the edges.
Sprinkle the top with the remaining 2 tablespoons of sugar and bake until the center of the cake is set, about 1 hour and 15 minutes.
Transfer pan to a wire rack and allow to cool for 5 minutes. Run a knife around the edge of the cake along the sides of the pan and let cool completely, about 2 hours more (if you can). Cut into wedges and serve.