Rangnick is expected to be appointed as Ole Gunnar Solskjaer's short-term successor for the remainder of the campaign, after which he will move into a two-year consultancy role.
News of the 63-year-old's appointment has been met with widespread excitement from the Man United fanbase, but Merson is worried that his lack of experience in England could hinder his chances of success at Old Trafford.
Speaking to Sky Sports News, Merson said: "I'm hearing people going how great he is - but he's never managed in England, he's never managed one of the top teams in the world. Why is that? Why has no one else come in for him when managers have been up?
"I hear great things about him, people saying he's going to press and he's hard-working and he's turned lesser teams into real contenders. But that's a lot different, getting a Schalke or a Hoffenheim to a certain stage.
"When you're asked to go to one of the biggest clubs in the world and they already have superstars and you've got to make them play. It's all right at Schalke and Hoffenheim because they're going to listen and they want to get there. These [United] players are there and you have got to get them to go again.
"I think this is going to be difficult, this is going to be a lot harder than what people think. What I'm hearing is it's like a gimme, he'll come in and that will be it. That Man Utd are lucky. I'm not sure about that."
Rangnick's first game in charge of Man United could come against Merson's former club Arsenal on December 2. body check tags ::