I recently implemented this. Set it to expire after a few minutes. Think thats quite safe.

But yeah I think the biggest security risk is if the user gets their email hacked.... But that's for anything really. You can even go via "Forgot Password" to get access to an account if you have your inbox compromised.