If you have sanded the trim pieces you could have left sanding scratches in them. From your description it sounds like that is probably not the case.
The very fine, almost invisible, scratches that remain after a final buffing with a white finishing compound are caused by grit. Where does the grit live? It can be in your buffing wheels, in micro fiber polishing rags, on the sticks of compound. Cleanliness is essential for the finishing process. I found that micro fiber rags straight from the package sometimes contain grit. I wash my buffing wheels and polishing rags frequently in hot soapy water, especially when new, and store them in screw top plastic jars.