Limiting screw on the rear derailleur?
Did this start when you changed something? Can you see whether the jockey wheel is too far toward the hub, or too far toward the drop-out, in that gear? Try turning the high-gear limiting screw accordingly, in small increments (like 1/4 turn, paying attention to how much you turn it each time so that you can go back if needed).
A work stand is a huge help in making derailleur adjustments. Alternately, if you have exposed joists or a ceiling hook in your garage or basement, you can use tie-downs to hang your rear wheel off the ground; some car racks allow the cranks and rear wheel to turn; or recruit a friend to hold the rear of the bike off the ground while you shift, turn the crank, and adjust.
Shimano's website has adjustment instructions for all their products, not sure about SRAM.
Other thoughts:
If the derailleur is very old and/or dirty, maybe the spring isn't pulling it all the way out? Even a small rock caught in the spring I suppose?
If the chain is old, maybe a very slightly sticky link that's only noticeable when it's under the least tension?
I'm assuming that your 12 tooth cog itself is in reasonably good condition since you said you don't use it often.