Actually windows does fine unloading dllls from memory, when they are not used anymore. Do you have access to the source code of the plugins? If so check that all libraries are unloaded on exit.

Are you sure that the paths are as you expect? Try loading the library with full path instead of relying on windows to search for it.

Note that dlls are always loaded from the working dir of the executable, not from the dir where the plugin.dll file maybe located.