Sweats are fine. I have stopped at the store many times after working out, in my sweat pants and shirt.
But i’m talking straight up PJ’s. Like the PJ pants with little sheep or w/e on them, an oversized shirt and all the hair on your head pushed to one side because all you did was get out of bed, put your shoes on and went on a shopping spree.