Both players were withdrawn during the 3-2 defeat against West Ham United, Havertz at half time and Alonso with 18 minutes remaining.
The Blues are already having to contend with the absences of several key players, and Tuchel has acknowledged that the situation worsened during the setback at the London Stadium.
Speaking at his post-match press conference, Tuchel said: "Kai has huge pain. Marcos has back pain.
"I have no idea when he [N'Golo Kante] will be back and we are not only missing N'Golo, we are missing Mateo Kovacic for four, five, six weeks. Jorgi [Jorginho] is playing with hip pain for many matchdays and you can see it."