I found that sometimes aluminum does not like the black or brown compound....the compound itself actually scratched it. Since you've seeming sanded each step well, and up to 2000 grit, go straight ot the white rouge on a loose cotton wheel.
I wetsanded to 1500 and went straight to the white and achieved results like this on my wheels.
I'd give it another sanding with some 1500 or 2000 and try bypassing the "coarser" compounds, see if it helps.