Sforno on Genesis 44:20/ספורנו על בראשית מ״ד:כ׳

1ואביו אהבו, more than he loves all (any) of us combined. Therefore he did not allow him to come to Egypt the first time we came here in order to buy food for our families. The reason was not that we came on a spying mission, as you appear to have thought