He notices the shivers and has to wonder if it's the effect of sickness, exhaustion or fear--he sincerely doubts it's due to the cold.

Bucky continues to follow, relying on looking fairly casual to keep himself from being noticed, and he continues at his own pace, apparently in no hurry. Although he's been all around the city, he can't say he's been inside every building, and he isn't sure what could be in the one the figure is heading towards. Not that that would stop him.