I think I've discovered a small bug/loophole with this option at the moment. If you delete an entire league, the players from it (and unsigned draft picks who've returned to school) appear to ignore this setting, and will sign with team in leagues that have been specifically set to not allow outside free agents. I found this out when I retired the American Association after 1891, and then noticed some of the (white) guys from it had turned up in the Negro Leagues.
