@ai6yr

No.

Microsoft does not know how exchange works.

"But when the FedRAMP team asked Microsoft to produce the diagrams showing how such encryption would happen for each service in GCC High, the company balked, saying the request was too challenging. So the reviewers suggested starting with just Exchange Online, the popular email platform.
...
It was the kind of detail that other major cloud providers such as Amazon and Google routinely provided, members of the FedRAMP team told ProPublica. Yet Microsoft took months to respond. When it did, the former reviewer said, it submitted a white paper that discussed GCC High’s encryption strategy but left out the details of where on the journey data actually becomes encrypted and decrypted — so FedRAMP couldn’t assess that it was being done properly."

https://www.propublica.org/article/microsoft-cloud-fedramp-cybersecurity-government

@ai6yr I’ve definitely had trouble fixing servers when, for instance, sshd keeps getting oomkilled. But you’d think they could bring a server up and watch it get unhealthy. Unless they’re just failing instantly but that doesn’t seem to be the case based on the status report. I just end up feeling for their ops team. #HugOps. This stuff is hard. I hope there’s a good postmortem published. AI or not, there’s a team of really stressed-out people somewhere staring at their screens.

@ai6yr Wrong question. The correct question I think should be: Do we ever want a time where 90% of our personnel should have to know how our system operates?

@ai6yr I thought this would’ve been resolved within an hour or two, but now that it’s dragged on this long, I was thinking about it and I’m guessing it’s hard to bring the system up quickly enough to not implode in on itself as backlogged demand hammers away at the first nodes to come up.

We might be in “this has never been attempted” territory. As in the system has never been cold started with the same order of magnitude of subscribers as they have now.

…

@ai6yr “We're analyzing diagnostic data from these servers with our networking and platform experts to determine what's triggering this behavior.”

I read this as “we’re scrambling to get emergency contracts in place with all the people we laid off who actually know how to diagnose the problems”.

@ai6yr it's also probably a matter of having enough servers for the usual load then having them under perform means they are in ddos territory

@ai6yr

@ai6yr I hate to come to their defence, but these are complex systems with millions of lines of code. Something like a subtle race condition can make it extremely hard to diagnose.