Paper_xrlhf_socc26