OK, I got this working, let me summarize:
Add a reference to 'system.configuration' to your app
Create an app.config file, add an appsettings section:
<appSettings>
<add key="test" value="mykey" />
</appSettings>
add code in an event handler to read the config file:
Dim str As String
str = Configuration.ConfigurationManager.AppSettings("test")
Here is the key:
After publishing your gpaddins code, rename app.config to dynamics.config, and put it in the dynamics/gp directory.