Tankless on-demand water heaters are in vogue right now, along with most other things that purport to save money on energy costs. I think they're a silly waste of money for most people, especially folks who are trying to improve the energy efficiency of their older home. May I blather at you? It's the internet, right?
I'm a fan of a basic electric water heater. No moving parts means not much to go wrong, there's no risk of carbon monoxide poisoning, they're easy to install, and cheap. They're also nearly 100% efficient. All the electrical energy consumed goes right into heating the water. If the tank is well-insulated, the losses to the air around the water heater are small. They're especially small if the tank lives inside the thermal envelope of your home, since the heat flow is a function of the difference in temperature across the insulation. And if you live in a heating climate, the heat lost just keeps your home warmer and reduces the load on your heating system by some tiny amount during most of the year. For many people in older houses, the water heater lives in the basement, and this situation applies.
So why would you want a tankless? Natural gas is cheap... For now. I'm not sure that's a good long-term gamble. Big families that need a lot of hot water in a short period of time might want a tankless. The most compelling argument for me is the essentially unlimited flow of hot water you can get out of on-demand units. But I view this as a luxury upgrade, not an energy saver.
What happens if the power goes out? A tank heater will stay hot for a very long time, and you can use the hot water in it as long as gravity keeps working and pushing water into your pipes from the storage tank you likely rely on.
Many of the statistics you can easily look up are misleading, in my opinion. Everybody focuses on the energy factor of the unit, but it's a lot more complicated. The waste heat from a gas appliance goes out the flue. The waste heat from an electric appliance stays in the building, and there's no vent to leak air in and out of your house, so even if you could get a gas unit with the same energy factor, the real impact is different.