Backport assert and unregistration fix from recursive branch

After multiple changes, the submit should be removed as it is done after a wait, so we want to unregister the data at the moment we do it and not later.