For sigma-delta, noise shaping would be used. This of course would introduce a entirely different type of linearity errors, but given how sigma-delta works, the overall error would be less.
....
Now how does this correlate to what we hear? Those of you who have heard enough DACs, quality DACs that is of both kinds, know that resistor ladder and sigma-delta sound different and have different strengths. The best sigma-delta implementations sound highly detailed with sharp attacks. The best resistor ladder DACs sound smooth with natural timbre.
....
Maybe greater linearity = more resolving (it's true mathematically, but is it true subjectively?) Maybe linearity error plots with certain more predictable patterns = smoother more natural sound?

Very interesting!  I know I'm quite late to the party, but it just struck me how this proposition is somewhat analogous to Nelson Pass' comments about negative feedback in amplifiers - that negative feedback decreases total THD, and improves other standard audio measurements, but at the cost of adding new complex distortion components/nonlinear distortions, particularly of high order, that many people perceive as unnatural.  https://passlabs.com/articles/audio-distortion-and-feedback