"Not by much, I'd still have to roll it off my shoulders --"

Even the slight movements that unbuttoning would take come from his shoulders, from the play of his muscles, and they're all sore. Then he'd still have to push it back off. He takes a deep breath once he's free of it, lets it out in a sigh and drops himself backwards over a chair so he can lean his weight against it.