I think it's more correct to think of the "guilt" as being a socially manifested thing. Affirmative Action has been a response to that "guilt."

Personally, I think Affirmative Action is a great effort, but is more of a "band-aid" approach. Dramatically improved funding for our nation's poorest of educational systems is the only way to lastingly improve the poverty/power gap.