Never used AG paint cleaner before mate so I'm not familiar with it I'm afraid.
People say it isn't impossible to remove swirls by hand but I believe if you want to keep your arm then it is. You're best off getting a machine polisher and then using either Meguires Swirl remover or Dodo Juice Lime Prime. Don't just jump straight in though, there are several guides over the internet on how to do it properly, just search google.

As for getting rid of polish in places like that, I use a soft detailing brush whilst washing. Just work it around the areas that have polish in, not too fast, not to slow, a good steady pace