Whisk the cornstarch with a little bit of the milk until very smooth and the cornstarch is completely dissolved. Add this mixture back to the rest of the milk, and set aside.
In a medium, heavy bottomed saucepan, add the brown sugar along with a couple tablespoons water and melt the mixture until it starts to bubble.
Add the milk and the salt to the saucepan, and whisk everything together. It's ok if the sugar seizes up, it will melt again as the pudding cooks.
Over medium heat, continue to cook the mixture, stirring constantly, until it just comes to a boil.
Remove from the heat and whisk in the peanut butter and the vanilla extract until completely smooth.
Pour the pudding into your pudding cups, and allow to chill in the fridge for a few hours, or up to overnight. Serve with whipped cream and crushed peanuts, if desired.