Microsoft 365 outage leads to chlorine gas poisoning of 15 hotel guests
What else should they rely on?
An actual automation system. Ideally, a self-hosted one.
and it would alert everyone how?
Idk, any self-hosted system, there are so many stuff one can do…even having displays or alarms…
MS wants businesses to sign up for their cloud services, promises the world, and says don’t worry. Businesses rely on this nd when it goes down it’s not MS’s fault even tho MS is the service that went down? Come on.
It’s not MS fault if you don’t also have a fallback system or any system that can work even offline/dipendently on your network. Regardless of microsoft being down or not, emails for such stuff is a VERY bad idea and should be changed with a better system that cannot be accidentally ignored.
Ok, you got me. What’s the better system and why don’t businesses use it from the get-go instead?
Cool, good talk. Give me an example of even one and what other than email it uses that is on by default.
Auto-shutoff on the pumps side, like it’s standard for even sludge pumps.
So why didn’t they have that? Why isn’t it code?
Good engineering practices for starters.
Which are? Please tell me and give examples. I’m not an engineer and don’t know what’s code
Very rigorous maritime engineering standards.
So why didn’t the maritime code enforcement agency come in and make them fix it?
We used to have these things called blinking lights and alarm sirens that required only a few cheap components and no internet at all.
“Used to”. Awesome, what do you have now? Why did you switch from “these things called blinking lights and alarm sirens” and what did you switch to?
There are a ton of on-call systems available that have hard contractual uptime reliability requirements.
Every time I’ve been on a team with on call we had a required app that pushed critical notifications to your phone, plus SNS, plus email, plus slack, plus an actual phone call system.
And I’m not even an SRE or anything like that.
Cool what was the name of those systems you personally setup that had all of those messaging targets on by default?
Chlorine level monitors that staff check before opening the pool? That doesn’t require internet and would only be a simple embedded device. You might be able to have an electricity-free option too, not a chemistry expert but I imagine there are tests for chlorine levels that don’t require electricity?
Yeah they exist. Most people who own home pools use them and then take their water to be tested if something is getting way off.
I helped out at a hotel for a couple months at one point and their setup was simple. Every morning we would check everything manually to ensure it was correct, it was also controlled by machine. Essentially if the machine detects the level of something gets to low, it automatically adds the amount needed. Essentially all of the water is cycled into a room near the pool, and that’s where all water is added, tested automatically from.
The manual tests we would do from 2 different areas of the pool to ensure they matched what the pool sensors detected in the behind the scenes area. If something was way off somehow it alarm lights would show on the panel, and no chemicals would be added and a contracted maintenance group would be dispatched to come address the major issues.
Also pools are large amounts of water, so they don’t just suddenly change drastically unless someone is intentionally pouring chemicals of some kind into them.
Id say 98% of chemical issues are dealt with by normal staff and the automated system. The pool company with the contract comes to verify everything working once a month, and maybe once or twice a year you’d call them out for something, most likely after winter when opening the pool just to make sure they clear everything, and once a year when something weird happens.
Public pools here are required to have audible alarms and flashing lights on redundant electric circuits in the dosing room and main hall, plus a shitload of ventilation.
Every walk in fridge is required to have CO2 alarms, just like smoke detectors.
Not everything needs to go through a computer. A loud ass alarm is sufficient.
See that makes too much sense and might cause too much safety
An alarm perhaps? That’s what I’d demand if it was part of a system I was involved with designing
Why the fuck would you open the pool without getting a mail saying it is safe?
The email is that there is a dangerous condition, not that it is safe.
That’s such a rookie mistake to make in safety design…
Yeah, standardly you’d have someone manually check chemicals daily, and the automated system would correct chemicals when not in use, but where I grew up it isn’t so much of a pool being “opened” as it was just that it’s open around 365 days a year and less people just use it in “winter.”
But if its 81F/27C outside on Christmas day, people will be swimming on Christmas…
Subject: Fire.
Dear Sir/Madam,
I am writing to inform you of a fire that has broken out on the premises of 123 Cavendon Road.
Fire! Fire! Help me!
123 Cavendon Road. Looking forward to hearing from you.
Yours truly,
Maurice Moss
I know he tried calling 0118 999 881 999 119 725 3 first, but had he not ended up with another country on the line it would have been a much quicker response.
So long as he organized and put all the objects on fire together, he’ll be fine. Also don’t use fire extinguishers from Britain apparently.
Yeaaah not MS fault here, no sane person would rely solely on email for something like this