Find the right transducer for you. Then start building a system around it.
It might be that you can spend more on a headphone than the amp, but it's only because headphones are fundamentally over-priced at the moment.
Why would Sennheiser develop a proper HD650 successor with the best sound they could possibly put into a 500$ package and sell that to Europeans/Americans when they might as well produce a 3000$ unit with the same sound quality and sell to people for whom even that amount warrants no overnight consideration?
Ultra-high-end headphones? Many audiophiles have been taking it in the ass and loving it for years, that won't stop. The hype train ain't going to derail any time soon.
But getting the right headphone for you (maybe with an O2 as a starter amp until you figure out what you want there) is the place to start.