The yellow zone is simply a "safe margin."
As you know, it means don't put anything important there because there might be variation in the trim.
For example: some designers add borders around their design; there's a minimum of thickness for that kind of concept otherwise some borders could look thinner than others because of the variation in the trim. That zone is not only important for the texts but also for some elements of the design like the border I mentioned above. If there's a variation in the trim, it will be a bit less obvious if there's more white space between the safe area and the edge of the paper because the edge of the paper serves as a visual guide that will make the asymmetry more noticeable.
When there's a print project done, sheets need to be "shaken' on a special machine, aerated, and with the help of gravity they get in almost perfect piles. Then they will be fed to the press, stacked on pallets, and moved again to the trimmer. The guy on the machine will then take small piles of these huge sheets, add some measurements to the machine and cut. Some trim machines are not digital; if the operator isn't precise, there's a few millimeters here and there that can disappear on different piles. Most of the time, the sheets need to be shaken again before that operation. So as you guess, there's a lot of manipulation along the way that can cause some variation at every phase, depending on how careful the operators are. And then, there's the binding or other operations that also demand a certain safe zone.
There's also the quality of the paper, how it was stored, the humidity level, how "fresh" the stock is, coated or uncoated (coated can be more sticky), what's the size and quality of the press, etc.
With digital printing, it's even worse; the prints often have a small 1-2 degree angle misaligned and nothing is truly straight.
Pretty much everything is done manually and as anything manually, there's variations and higher risks of mistakes too. The safe zone makes the job of all these operators faster and easier; because technically, they can print and trim very precise design but that also requires more time. So it's not true nothing is ever perfect. But perfection has a cost that not many people are willing to pay :)
That graphic the printer gave you is confusing. some little arrows to point at the real trim and bleed line would have made a better guide.
Like this, for example:

By he way, that safe margin should ideally be minimum 1/16", ideally 1/8" and more from the edge. And some small press might even require more on one side because of the "gripper" zone.
If your printer requires a safe margin of 1/2", that can be because of the type of binding (eg. books, magazines), that can be because he's lazy and/or more economical, that can be because he's outsourcing, that can be because his machines are old and can't retain their alignment, that can be because of the kind of press, etc.
Related:
How can I determine how much bleed to use?