What Olympians hide in San Francisco’s quietest heart—minutes from the action, hours from the chaos