I find it handles better lowered. It's probably just the slightly lower center of gravity I feel. regardless, I certainly didn't find any drawback to it.
I lowered the front as much as possible, which isn't too much past an inch. I was concerned since the front couldn't be lowered enough to match the rear, that it might slow up the turn in, or feel less stable, but if anything I think the opposite is true. It turns in really nice and feels quite stable.
If you ride really aggressively, the reduced peg clearance could be an issue, but it hasn't been for me.
'79 Yamaha XS650