Chasing 189 to win, Rajasthan were 161/2 in the 18th over when Starc dismissed Nitish Rana (51) to change the course of the match.
Rajasthan needed nine runs in the final over with six wickets in hand but Starc kept bowling yorkers which put the visitors under pressure and the match went into a super over.
The 35-year-old was Delhi's natural choice to bowl the Super Over and Rajasthan managed 11 runs from five balls before losing both their wickets.
Delhi needed just four balls to get over the line and register their fifth win in six matches to top the table.