I am not sure how much of your original question was about the collaboration process and how much was about working on a server. I can't offer advice about collaboration, but I can share my experience working across a server.
First, I just want to preface my comments by saying that I have been working on InDesign files off of local servers for at least the past 14 years, and have never had any real issues to report. As long as the server and your computer are correctly configured, you shouldn't really even notice a difference most of the time. And since most the of the firms where I worked didn't back up individual hard drives, I have always been a stickler about not using my local drive for anything.
If you are having trouble, and your server is located near you (like in another part of your office)...then you need to discuss things with your IT folks because something is set up poorly. Don't let them blame InDesign for that issue...
BUT...a previous employer stored all of our shared documents and images on a main server in Charlotte, NC. My office was in Los Angeles. And I did have some challenges with this particular workflow. Whenever I had to work on a document, I would usually copy it from the main server to a local location, and then open it and save it there. If I tried opening directly from the remote server, there was a noticeable lag time (5-10 minutes to open the file), and frequent crashes. Any images or linked files that I had to use, I would do the same thing, copy them to a local drive and then work with them there.
If I had to collaborate with someone in a different office, I would send them the document (or section) directly--either embedding linked items, packaging the file or telling my coworker to ignore errors and work without the links.
Once the entire document was complete, part of my "close-out" process for that project was to upload the final version of the document to the main server, and relink to images stored on that remote server. As long as I retained the image names for the images that I had been storing locally, then relinking wasn't that difficult a procedure (we made sure that our image names were somewhat descriptive so that they were fairly simple to locate).
As a firm, we developed a number of templates for different types of content (like portfolios for our projects, resumes, etc.). Then we created pieces of content to fit within those templates and exported those sections of content as snippets (.idms files). The snippets were saved on the main server, and our entire team was very careful about regularly updating those snippets, so that our entire marketing team felt confident that the snippets were up to date and accurate.
Nowadays, shared CC Library files would probably serve the same function, only even better because then you would be notified if one of the snippets had been updated since you placed it.
Last, but not least, I use the InDesign book feature a lot. I started using it back in the days when the help manual (which was actually a printed book that was included in the box along with the 16 floppy discs that you used to install the program) said to limit the number of pages in one InDesign document. For the most part, that is no longer an issue, but I have found that working with large documents, especially when they have a lot of linked files (most of my documents run around 100-200 pages with lots of images, and file sizes are frequently 50-75 Mb) it really does make a difference to break your document into smaller pieces.
Anyway, I don't know this is of any help to you...but that is how I have been able to work successfully with large documents and remote computer servers.