Yes, but there's another facet to this. Suppose someone was born in Albania, then came to the US, had a political career, and was elected President. His/her parents and many relatives were still living in Albania. Well, then there's a military coup in Albania, and a dictator is installed. He starts making anti-US threats. He claims to have a nuke. The President having family in Albania--and being from there--could certainly impact his/her ability to react in this situation.
Okay, yes, this is a farfetched scenario, but there are all sorts of ways someone born (and possibly raised) in another country might--deliberately or inadvertently--show favoritism to that country: economically, militarily... . I'm not saying this ought to necessitate maintaining the barrier on a foreign-born President, but it is something to think about.