When the audioconvert starts, it emits 2 ports, but when we set the PortConfig,
it removes them and re-creates them. Previously, the stream class would not
remove the old port proxies from the list and therefore they existed twice.

It is also necessary here to store the proxies earlier, when they are added,
instead of when they are augmented, so that we can ensure they are removed.
Previously we would hit an issue where:
 - port proxy is added, augmented
 - augment completes but the GTask wants to complete asynchronously:
   it stores a ref on the proxy and adds an idle source
 - server removes the proxy, we delete it from the core's list
 - the GTask now calls the augment callback, which stores the (removed)
   proxy on the stream's port_proxies list...