[thesound]

These tiny specks you're seeing in the finish are called inclusions. This text gives a pretty good idea of what they are and how they get there:

Effects of Inclusions. In addition to oxides, a number of additional compounds can be considered inclusions in cast structures. All aluminum contains aluminum carbide (Al4C3) formed during reduction. Borides may also be present. By agglomeration, borides can assume sufficient size to represent a significant factor in the metal structure, with especially adverse effects in machining.

Under all conditions, inclusions whether in film or particle form are damaging to mechanical properties. The gross effect of inclusions is to reduce the effective cross section of metal under load. The more devastating effect on properties is that of stress concentration when inclusions appear at or near the surface of parts or specimens. Fatigue performance is reduced under the latter condition by the notch effect. Ultimate and yield strengths are typically lower, and ductility may be substantially reduced when inclusions are present.

Hard particle inclusions are frequently found in association with film-type oxides. Borides, carbides, oxides, and nonmetallic particles in the melt are scavenged and then concentrated in localized regions within the cast structure.

I discovered this phenomenon while in the midst of polishing a cast iron engine block. The block was cast as a "high nickel" block, which by default designated all the nickel in the parent iron as an inclusion. The consequence of having nickel, which is a pretty soft metal, dispersed evenly throughout the harder iron casting in the form of tiny specks, was that when I got to the buffing stage after sanding to 600 grit, I discovered that the buffing compound was rubbing away the tiny nickel speckles quicker than it's harder iron counterpart. This basically left me with a continuously progressive "orange peel" effect that never smoothed out no matter how much compound I used or for how long.
This of course gave me fits, but it also forced me to be experimental. I exhausted pretty much every form of buffing wheel using different wheels in combination with different compounds, and had a marginally higher degree of success with certain combos but nothing that actually amounted to the problem being solved. Due to the rules of this board, I have no choice but to PM the details about the product I used to the originator of this thread, but what I can say is that this stuff made the difference between the project being a success and a failure, and I'm pretty confident that it will help with the problem here.