You totally can. Map your route to go around the "biggest hills". Topography maps available on Google maps and such.
You get more torque to the rear wheel on a Single Speed / Fixed. It will make hill-climbing easier. A lot easier.
I bust up hills on my SS so much faster than on any of my geared bikes.
It's nice to have the gears when you need em' though, like on long rides and super duper hilly terrains in the countryside.
"Things look different from the seat of a bike carrying a sleeping bag with a cold beer tucked inside." ~Jim Malusa
2009 Trek 520-Brooks B-17 Special in Antique Brown
2010 Surly Long Haul Trucker-Brooks B-17 Standard in Black
1983 Fuji Espree Single Speed-Brooks B17 British Racing Green