A cage move needs to be done carefully, so he settles. Three weeks isn't too soon, but the best way is to add new bedding to the new cage and press it down (because it's bigger) and then when you come to move him over, also move all the old bedding across (resisting the temptation to clean anything) and spread the old bedding on top of the new. And also move everything else across and keep it in the same position as much as possible. Eg house front right, wheel centre back, shelf left - or whatever. Even keeping sprays and small hides in the same location. Keep any hoard or nesting area and move that across and put it back in the same place.
Without doing these things, a cage move can be very stressful for a couple of weeks. With doing them he is more likely to settle in 2 or 3 days. He might be a bit skitty the first day. This would be the same whether you do it now or at a later time.
As well as that, what really helps, I find, is a transition day - if you can possibly have both cages in the room at the same time (ie space wise). With that you let the hamster have a play and explore of the new cage, the day before moving him in/moving everything over. So obviously the new cage wouldn't be fully set up if he's still sleeping in his old one, so for the transition day you would just put new bedding in the new cage (which you'd be doing anyway), and add the odd new item you're thinking of using, and maybe take one family smelling item (not the wheel) - eg a tunnel or smaller hide - and put that in the new cage. So there are a few things in there and it's not too exposed. If you don't have any new things to add to it, then just put two or three things from his old cage in there.
Then let him have some time in it that evening - like out of cage time. They tend to look excited and explore and it helps familiarise them and means it's a bit scent marked before they move in. Some people think - oh he likes this I'll just leave him here and move everything across now, but I would recommend not doing that, as they can freak out later when they realise they're not "going home".
So after he's had some time - maybe half an hour or even longer - you can be there and talk to him as well - just pop him back in his old cage again for the night and move the items back in the old cage where they were before.
Then next day, moving day, pop him in a pet carrier in another room (where he can't hear things and smell things being done to his cage), and move everything over from his old cage to his new cage and get it all set up. This often takes longer than you think! But he'll probably just go to sleep in the pet carrier.
Final thing, once everything is set up, is to remove the old cage from the room entirely and maybe have a clean up around that area (eg hoover and the shelf) - unless the new cage is going to be sitting in the same location then it's not necessary.
The reason for that is, if they can still smell the old cage, they may not settle in the new one. So then pop him into the new cage. And then it's a bit like bringing them home for the first time - give him a day or two on his own, but you can still talk to him through the bars.
Last time I did this (with a hamster I'd had longer), he settled straight away because everything was the same inside the cage - the only thing that seemed to bother him a bit was the bars smelled different obviously - and he kept sniffing and scent marking the bars and looking a bit confused, but otherwise he was fine and happy within 2 or 3 days.
It's quite an important topic - the first time I did a cage move it was a disaster - cold turkey - incredibly stressful afterwards and we didn't think the hamster would survive. But he did. That was when I knew nothing about hamsters.