I'd say keep the Matrix. If you are the type to drive a car until it dies, it will last alot longer than the Prius (no unknown battery issues). Plus, it seems like it'll do everything you want it to (cars, bikes), and you'll have $4k in your pocket, which you can use towards paying off the Matrix, or put in a special gasoline account--even at these prices, it'll take a while to use up $4k in gas. Plus, I bet you could sell the Prius in a heartbeat.

If your town has car sharing, like Zipcar or Flexcar (I think they bought each other), having one car among two people gets easier.

All that being said, however, I found that sharing a car to be quite cumbersome, even though I did it for five or six years. My ex (who was then not my ex) monopolized the car even though he worked from home and I had a 14 mile commute each way, which I did by bike several times a week and by public transit the other days. I could use the car as long as he didn't need it, but he always "needed" it. I often could not go on club rides on weekends because he needed the car. Not the reason we are exes, but it was perhaps a symptom of much larger issues.

I think if you have a plan and are both willing to take other forms of transit, then it can work. But if one is not willing to do that, it can mean alot of inconvenience for the person who is willing. I spent way too much of my time waiting to be picked up, waiting for a bus, etc.