The analysis of 5,700 elderly men in Norway showed those doing three hours of exercise a week lived around five years longer than the sedentary.
The authors, writing in the British Journal of Sports Medicine, called for campaigns to encourage fitness in older people.
The study comes as a charity warns about low levels of exercise.
In the study – conducted by Oslo University Hospital – found both light and vigorous exercise extended life expectancy.
Official advice in the UK recommends 150 minutes of moderate exercise per week in the over-65s.
The trial tracking 68 to 77 year olds found that doing less than an hour a week of light exercise had no impact.
But overall those putting in the equivalent of six, 30-minute sessions of any intensity, were 40% less likely to have died during the 11-year study.
The report said: “Even when men were 73 years of age on average at start of follow-up, active persons had five years longer expected lifetime than the sedentary.”
It added that physical activity was as “beneficial as smoking cessation” at reducing deaths.
“Public health strategies in elderly men should include efforts to increase physical activity in line with efforts to reduce smoking behaviour.”
The report did not look at how active people were earlier in their lives.
However, the study comes as the British Heart Foundation publishes a report warning people are getting too little exercise.
Its analysis shows that the percentage of adults doing no moderate exercise is:
69% in Portugal
55% in Poland
46% in France
44% in the UK
34% in Croatia
26% in Germany
with Netherlands doing relatively well at 14%
Julie Ward, from the charity, said: “Regular physical activity, whatever your age, is beneficial for your heart health and ultimately can help you live longer.
“However, our latest statistics show that nearly half of people in the UK do no moderate exercise whatsoever – a rate higher than many European countries.
“Our message is that every 10 minutes counts and that making simple, more active changes to your daily routine can set you on a path to improved heart health.”