The Spurs boss has regularly singled the club's record signing out for criticism over his fitness and performance levels since taking over at the club.
Both of Ndombele's Spurs goals came under Mauricio Pochettino and the 23-year-old has only started six games in all competitions since Mourinho was appointed in north London.
The former Manchester United manager turned the criticism up a notch following Tottenham's 1-1 draw at Burnley earlier this month when - having hooked Ndombele at half time - he said the midfielder had been in the Premier League long enough to adapt and start performing better.
Petit, who made 140 Premier League appearances during his career, says Ndombele has the quality to be a top player, but that Mourinho's criticism is correct.
"Is he overweight? He looks a little bit," he told The Sun. "But his body language does not help him. He needs to improve physically.
"He has quality but I agree with Mourinho in that Ndombele could do much more.
"When he is very motivated, fit and ready, with his mind and body, and when he has the ambition to perform well, he becomes a top player."