I haven't been myself but have always wanted to check out thre relatively small town of Lake Placid, NY, home of the 1980 Winter Olympics. Visitors can ride down the actual luge run used in that Olympics. I'm sure there are some other "remnants" visitors can look-up on-line & check out in person.

Lake Placid is a rather nice place to visit - at least it was in the 90's when I was there. Yes the ski jump is still there, but aside from that you wouldn't know it held a major event like the Winter Olympics.