He represented the Netherlands at the Eurovision Song Contest 1960 singing "Wat een geluk" (What luck). He finished 12th out of 13 scoring just 2 points.

His own show "The Rudi Carrell Show" was a huge success in Germany from the 1960s to the 1990s. The show included a similar concept to "Star Search" or "Pop Idol" and brought many well-known German pop stars and actors to prominence, such as Alexis or Mark Keller. It also featured comedy sketches.

In 1987, he famously caused a diplomatic rift between Germany and Iran with a sketch in which veiled women threw their undergarments at someone dressed like Iran's Supreme Leader Ayatollah Khomeini. The outraged Iranian government responded by expelling two German diplomats and closing the Goethe Institute in Tehran. Another controversial sketch used clever editing to show the then Chancellor Helmut Kohl and other prominent German politicians apparently consorting with prostitutes.