As Elusive says there isn't a clear answer as it can depend on so many things. At some point, hamsters do have to go outside and leave the hosue - if they have to go to the vets for example, they need carrying from A to B in a pet carrier. Even if it's just from a house to a car. When I have taken our hamsters out this way, they have seemed curious about the "fresh air" smell and sniffed a lot through the top of the pet carrier, so I don't think it's an unpleasant experience.
There are risks to having them outside in a garden really - most of which are common sense. If you live near a busy road there could be traffic fumes and noise for example. If it's too hot or too cold it's bad for them.
If you have a garden and some grass and don't use any chemicals on the grass and it's a quiet area, and early evening on a warm summer's day (but not too hot), I can understand why people might wish to let their hamsters have a little bit of time outside in a garden, in a secure area. It is something I have done myself in the past.
However that was with a confident, tame Syrian who trusted me, and in a secure area. There is a risk that, however tame, they could run away, so a playpen or large cage top might be necessary. There is also the risk that a bird of prey might swoop on them. I have seen people use a large cage top as a playpen area to protect from that, or have a picnic rug on the ground eg.
Personally I have mixed feelings about it. On the one hand, it's your pet and you may want them to experience certain pleasures in their short lives, with precautions. On the other hand there are risks involved.
There are risks to a hamster free roaming in a room in the house as well but that can be good for them providing a lot of care is taken.
So there is no hard and fast rule except for caution with the environment and the risk of other animals that could be around.
It is a very personal decision, as Elusive says, and weighing things up. It would be interesting to see what a vet would say about it.
But when you say "from time to time" I would say - definitely not as a regular thing - they need their home environment and routine.
