As I mention, the difference it production speed is only about 30%, so in the end, if a rebuild would otherwise have taken "ages", it will still take ages. As I mention, some people will need to rebuild so often that it's necessary to have higher levels. The same logic applies to market place. Most people don't need more than level 10. However, those who have to shovel resources out of their city might want more if they don't have time to shift resources around. Even thought I strictly stick to level 10 market place, there have been many moments where I really would have liked a higher level.
So, really, I don't think our reasoning is that different. In the event that someone kills your entire defence in a heavy defensive city, it would take around a week to rebuild, but this is definitely among the most rare events of rebuilds (normally, you either support the city that kill the attacker, or your city is cpature). I single city will never be able to defend against persistent attacks on its own anyway. I don't know why you would want to build barracks past level 5 in a bireme city. And, as said, even if you lose your 40-50 light ships in addition to some transports, your land troops in that city would still take longer to rebuild. True, if you don't lose any units, you would have to wait for the rebuild, but with level 5 harbor, it would still only take 1,5 day to rebuild. I would consider a higher level of harbor in a pure light ship city as the nuke wouldn't be very useful until it is fully built, but I have generally decided against it as light ship nukes are so population fragile, a 200 light ship nuke often being quite a lot less effective than one with 260.
True, good point about not building the special building on the right side. I was actually lying thinking about the guide, and this popped up in my head as one of the things I hadn't mentioned. Seemed I never bothered to put it down, though, but since you have now mentioned it, I will make sure to add it in right away (although I'm soon running short on characters).