Combine the gelatin, milk, and vanilla in a small bowl, stirring with a fork to thoroughly combine. You’ll also need a 4-cup capacity container and a mesh sieve on hand.
Put the egg yolks in a medium sized bowl, then whisk in the brown sugar a little at a time (if you add it all at once, it may be very difficult to stir). Whisk in the cinnamon, salt, and ginger too.
Meanwhile, bring the milk to a simmer in a 2 quart stainless steel pot. Once hot, shut off the heat to prevent the milk from scorching while you temper the yolks. Ladle a bit of hot milk into the yolks, whisking gently to combine. It’ll be crazy thick at first, but loosen as the milk incorporates. Keep adding more milk until the yolks feel warm and fluid. Pour the yolk-mixture into the pot, whisking to combine it with the remaining milk.
Cook over medium-low heat, stirring constantly with a flexible spatula, until the custard thickens noticeably, about 4 minutes.
Shut off the heat and whisk in the pumpkin puree. Strain the custard into the prepared container, then add the vanilla gelatin. Stir with a fork until the gelatin has completely melted and incorporated into the custard. Cover with plastic and refrigerate at least 4 hours (or up to 10 days) before proceeding.