Steel is not going "downward." New steel bikes are very light and ride very smoothly. They are not like the old steel bikes of yesterday. There is a huge number of people on this forum who have very expensive, custom steel bikes!
If you can afford the carbon, go for it. You'll end up not riding the alu bike if it hurts. It is not silly to buy carbon, if it fits right, you can afford it, and you want it! My first road bike was alu; I had it for about a year. It had crappy components and that's what did me in, more than the alu. Then I bought a carbon bike, and traded it 2 years later for another one.
My favorite saying is "you get what you pay for." Be aware of the level of components, the service you get at the shop, etc.