Otherwise, if the object manager is destroyed while a sync is in progress,
we get an invalid 'self' pointer on the callback later, which is being
called regardless

There is a bit more work that should be done in the core to avoid leaking
this ref in case pipewire disconnects before the sync is completed