Hi. So annoying. I hear you - hamsters don't like full cage cleans and being moved somewhere temporary. However, it may have to come to that. The bugs won't do any harm to your hamster but they could start to multiply. They can come from hay, sprays, or even hatch out from invisible eggs in hamster food or treats. So I freeze absolutely everything edible and even some hyacinth and grass things or wood bark items and hay as well. Freeze them for a week. Anything wipe clean like smooth wood items or ceramic is fine. Although I have seen reports of some niteangel houses coming with bugs living in the corners of them - possibly from bad storage. And once had a barkwood house with woodworm! Which I sent back straight away.
But houses are usually fine.
You could wait and see, just squashing them as you see them and see if that solves it. If they start to increase in number though, you may well have to do a full cage clean as bug eggs could end up in every item in the cage and in little crevices. It's a horrible big job though.
I did once find a half eaten, rotten blueberry in a cage that was crawling with bugs, but once I chucked the blueberry I had no issues with bugs in the cage.