Clean up more thoroughly when shutting down server connections.