If you bring food to a potluck and more than half of it is still in your pan, is it alright to bring the extras home?

Really, I've always wondered about the etiquette of leftovers at potlucks.