I'd stick with Large. I got the green polo and it fits very similar to ALD tees I have. It's a little longer than other polos, so also something to consider.
Saks also has a pair of 550s for preorder -- limited sizes left
https://www.saksfifthavenue.com/product/new-balance-men-s-550-mixed-media-sneakers-0400013066037.html?dwvar_0400013066037_color=WHITE%20BLACK