I've only been down to Portland a few times but a few things I really liked
The Japanese Garden at the arboretum is awesome! December might not be its best season, but it is really beautiful. The classical Chinese garden is very nice too.
If you like books there's a great used book store called Powells - its enormous http://www.powells.com/
I'll agree that the Portland street people aren't very agressive pan handlers, but there are a lot of them. I've heard that Oregon's very liberal support system and relatively mild weather attracts them, and some,especially young people, specifically travel there. My husband is also from New England (and a small town at that) where you just don't see a lot of people living out on the streets. It kind of freaks him out. I never felt threatened by them.