Rashi on Numbers 22:7/רש"י על במדבר כ״ב:ז׳

1וקסמים בידם AND DIVINATIONS WERE IN THEIR HANDS — all kinds of divination, in order that he should not say, “I have not got my tools with me”. — Another explanation is: this omen (קסם) the elders of Midyan took with them: — they said, “If he comes with us this time, there is something substantial in him, but if he puts us off there is no use of him”. — Consequently when he told them, “Stay here tonight”, they said, There is no hope in him”; they left him and went away, as it is said, (v. 8) And the princes of Moab stayed with Balaam”, but the princes of Midyan went away (Midrash Tanchuma, Balak 5).