Childermass doesn't particularly care whether he's bothering the other man or interrupting anything in particular. He's got a discussion to have with the man that he's already (clearly) held off for far too long already.

"How much further must this go?" he asks the other man, looking him straight in the face, watching for Norrell's reaction, his own expression blank, eyes piercing. He does not bother to explain himself. He's fairly certain that Norrell knows what he's talking about.