Sorry for the long answer but not knowing the specific light or use I just kept thinking of more things to cover. Lastly if a light is safety rated that rating is based on using only the recommended batteries,usually alkalines. Not a big deal since most bulbs are easy to replace but if it has a specific bulb you want to keep it should be avoided. 'Overdriving' a bulb often produces better light but shortens bulb life and can 'instaflash' a bulb sometimes.

It probably will be fine for most LED applications but will 'overdrive' an incan bulb since you are using four in a series. Lithium Energizers,also far superior to alkaline, are the reverse and usually have a slightly higher voltage than alkalines. NiMH is generally superior to alkaline IMO but output could be reduced in a light designed to run on alkalines. So unless your in Alaska, certainly all night. If your drawing 500mah you should get something like 40-50 hours of burn time on bright depending on the efficiency of the converter. I think the f cells are around 26 ah in capacity (26000 mah). You can use NiMH but they are only 1.2V compared to 1.5V in a fresh alkaline. a 6 volt lantern battery is really just 4 'f' cells in a case. In short having the wrong polarity will ruin the LED unless it has polarity protection.

Finding LED drop-ins for old incan 6V and Railroad lanterns has been an obsession of mine and it takes some searching but there are dual-polarity drop-ins available that will work.If you are powering a light that is LED from the manufacturer then it is most likely not dual polarity and may not have polarity protection. Like Lynx Arc posted it doesn't matter with an incan bulb but if your lantern is LED or you are using an LED drop in it must be the correct polarity. The majority of 6V lanterns are negative polarity where common in-line battery lights are positive. Dorcy still offers a 4D to 6V adapter and D to AA adapters are readily available online.

The simplest method would be to use a 4D to 6V adapter with D to AA adapters or the AAs fixed in the 6V adapter another way. AAs will work fine power wise but obviously don't offer the run time you get with the 4D or 4F cells that you find in a 6V lantern battery.

Lantern lamps 6 volt battery