I've had to overhaul my FD on 2 bikes now. First one just was never right. Trips to the shop just made things worse. Second bike, seemed like it was just installed wrong.

I printed out the ParkTools page and started from scratch with the second bike (didn't have to go this far with the first one as the limits were correct, just positioning of the cage wrong).

I started by checking height with the penny test. Then I looked at angle of the cage. Once the clamp was set properly, then I started on indexing. Start with the high limit (usually screw farthest from the frame), with no tension on the cable. Set it so that there is a very small gap between the chain and the derailleur when the chain is in big ring little cog. Don't forget to make adjustments when the chain is in the small ring to take tension off the cable. Then shift back up and check for position. Look on the site for where you should look on the cage to check for rubbing. Rubbing more forward on the cage could indicate a problem with height or angle, meaning you didn't do step 1 properly. (However, I've noticed with my compact that I'm going to either get some rub or the cage will be too high--I decided to just deal with rubbing since I hardly use that cog, because the improved shifting outweighed the damage from rubbing IMO).

The lower limit screw is typically the one closest to the frame. Set this index next and check for rubbing on the inner part of the cage from the small ring. Again, you want a very small gap.

Once the limits are set, check shifting speed and then adjust indexing (cable tension) using the barrel adjuster.

Sounds like there may have been a little more wrong with your initial setup that has come out when you tried to tighten the cable at the clamp.

It took me a long time, but I was able to figure it out. I felt I really had no choice, because when I noticed problems with the FD after getting on the new bike for a fit, the shop "fixed" it, and by that I mean made it worse.

It helps to start this process with clean and well lubed chain. I'm sure you can do it. Just if you do move the clamp (if your derailleur is a clamp-on style) that you don't overtighten it and crack your seat tube.