I wouldn’t clean it with anything either unless something was obviously on it. Break in is most important in a new belt and since we are on the topic make sure to break in your spare belt before you need it. Break it in then switch belts so the spare you carry is ready when you need it.
I installed belt after cleaning sleeve with scotch brite and blowing out the clutches I did the break in as per description on belt cover drove in low with varying speed for 17 miles
There used to be a recommendation that all new belts were washed with soap and water (in the sink or dishwasher with a good rinse to remove the soap) to remove any residue left behind from the moulding process. I still do this with all new belts but recently saw a video from Ultimax that said that washing was no longer required. He definitely recommended the break in procedure to allow the rubber to set before going full power.
Cleaning the sheave surfaces to remove all belt dust and residue with a scotchbrite pad is something I do whenever I have the belt guard off followed by blowing out the belt dust with air and wiping the sheave surface with a clean cloth that has break clean or something similar on it to remove any remaining belt residue.