I think your idea of the book icon vs power button isn't bad. You could probably find something a bit simpler and easy to recognize at small size using the same concept.
My first idea was to use a battery for offline, even though it's used for power. On a website or app, it could make sense since everybody knows the laptop or device is unplugged when they see the battery icon. So maybe you can use that idea too.
And as other examples, I found these icons:

You could use a laptop icon and maybe move the connection symbol in and out of it to mean it's online of offline. Or similar symbol and simply add the well known connection symbol or a electric plug symbol.
You could use different colors code as well to add more emphasis on the connection part of your icon; for example the connection part could be pale gray when offline, and colored when online.
People get used to these symbols, no matter what you'll choose. As long it's visible at small sizes.
PS: I didn't draw these icons, you'll need to purchase them or do your own version if you want to use them. Most of them are low resolution files from stock image sites, and some were modified slightly.