ok....
in book 4 its states that 50 years ago the Riddles were killed....
that means voldemort did it straight after he left school so hes still young so his father must still be fairly young
it then says
Elderly Mr and Mrs Riddle had been rich, snobbish and rude, and their grown-up son, Tom, had been even more so.
well that suggests that mr grown-up son Tom is voldemort's father Tom who must have gone back to live with his rich parents after discovering his wife was a witch and dumping her