I've had that problem on a column on my front porch. When the birds fledged, I pulled down the nest and put a couple of bricks up there to fill the space.