Thanks Renee! That is exactly the part I was searching for. Now that I know it's called a washer...

Trying to put an image here:



If the image shows up, you can see (on the edge) that it's tapered. When installed between the derailleur and frame, this taper can correct an angled front derailleur.