Copied from my reply - just to make it visible :)
There is a bit of an issue here it seems.
If your user has lets say ID 5 - then when u run through the users addresses, then the address not in AccessUserAddress will when u call GetValue("UserManagement.User.UserAddress.ID") return the user ID instead of address ID - which then means if the user also have 5 alternative addresses then you have 2 values listed out where the ID will be the same.
Which will end up causing an error on different implementations like fx the rapido one, where when u want to delete an address it checks if it has the same ID as the user.