Semantically, the typeface of the punctuation should be determined by its degree of association with the preceding word. In the example given for the Chicago Manual of Style, the exclamation mark after Banzai! belongs to the word itself, rather than marking the end of the enclosing sentence, so it should clearly be italicized, but that doesn't mean that all punctuation marks following words should be treated thus semantically. If text were highlighted by changing the background color, an effect which is available in markdown but not very visible on this site, it would be strange to have the punctuation of a sentence highlighted when most of the sentence wasn't.
Typographically, however, things are more complicated. The difficulty is that even when semantics would suggest that a punctuation mark shouldn't be italicized, attempting to render it upright can sometimes look ugly! Semantically, putting the punctuation mark in italic when the sentence isn't is semantically nasty! Upgright punctuation is semantically more correct, and if spaced properly, should be visually fine as well. Unfortunately, getting the spacing right is often difficult.
It should be noted, btw, that the Chicago Manual of Style is directed at people writing for newspapers and other hastily-typeset publications. Many of the practices advocated therein are intended not to yield the best results, but rather to yield consistent results which are not reliant upon the judgment of people setting text. If the typeface of a mark following an italicized word is varied according to semantics, then it's possible for a typesetter who misinterprets the sentence to get it wrong. If the typestyle is determined by rule, then it says nothing semantically and thus can't be semantically wrong.
I would suggest that when taking control of document typography, one should use very different rules from what the Chicago Manual of Style would advocate, but be prepared to hand-tweak a lot of things.
