Once bitten, twice shy.
Are you seeing an OT/PT for the wrist? Baby steps. Is there anywhere non-technical you can go to get your confidence back and that doesn't require as much from a recovering wrist?
Let's commiserate. My knee won't do over 30 miles on the lower-geared CX bike (with the compact double that doesn't tweak it as much). Joints are stupid.