It might be mildly poisonous, but the smoke doesn't really go inside your lungs or anything, it'll probably kill a few cells or so in your head, but going into a smoking facility or a polluted area is probably much, much worse.
EDIT: Actually, my bad, it probably does go into your lungs, but...