Discussion
“Why are these systems even on a network? Just disconnect them!”
I see this after every OT attack, usually from smart IT people. As someone who responds to these incidents, I wish it were that easy. I wrote up how we got here, and why water utilities and refrigerators keep getting hit.
https://tisiphone.net/2026/11/08/lets-talk-about-why-all-this-ot-stuff-is-connected/
@hacks4pancakes 100% If we accept that there’s some value in reaching them remotely for monitoring or whatever, that still doesn’t mean they need to be on the _public_ internet. Make them accessible only on the local network, perhaps reachable by VPN. At least that way you’re not entrusting the devices themselves to have decent security (they won’t).
@hacks4pancakes
Thanks for this; I hadn't realised how big a part OEMs played in the rise of these problems.
@hacks4pancakes There was this post on Reddit a few days ago, which nicely explains how these systems end up open to the Internet:
@hacks4pancakes
A great summary of the journey. Understanding the whole problem is only way to develop appropriate solutions.
@hacks4pancakes I am reminded of a startup I worked for about 10 years ago to ensure cold chain. We cared about security. It made our devices more expensive. The customer did not care. It went out of business.
I hear you and I put mouse-over definitions on all the acronyms.
Yes guys, the answer is capitalism.
With a side of convenience and laziness capitalism.
But we're stuck with the problem.
@hacks4pancakes One of my early hair-raising moments was when I worked at Ford 20+ years ago. I found that they had 19/8 and used it for everything. Workstations. Servers. Robots on factory floors. RFC1918 hadn't been ratified when their network plan was originally developed. Further, many of their systems had RTOS/Embedded kernels that literally required editing source and recompiling to change the IP address. Having even an hour of downtime for deployments of updated firmware (assuming all went well) would have amounted to millions and millions of dollars in lost productivity. This was not going to happen during "Way Forward" (https://en.wikipedia.org/wiki/The_Way_Forward) when they were targeting the layoff of around 20,000 people.
Additionally, we rebuilt their time and paysheet system in Linux VMs that had an emulation layer so HP MPE mainframe software would run on them and display using an in browser TN3270 emulator. Why not a full rewrite? It would have cost more to provide required training for UAW workers than playing horseshoes and hand grenades with an abstraction layer that left the input and output the same, with supported hardware from the 21st century in the middle.
I'm sure you have more horrifying anecdotes that are more recent, but (a) my NDA has expired and (b) if it was that bad then, I can't imagine what the current landscape is like.
@hacks4pancakes There is also more than a little bit of racism involved too.
@hacks4pancakes Good article. I would simply blame capitalism since many companies/utilities do not do the most basic protections that are free. 1. Change all default passwords and create good passwords. 2. When someone leaves the company delete their access and if there is a shared password change that password. 3. If possible delete default accounts. So many objects on the internet still have an account name and password of “admin/admin”
@hacks4pancakes thank you so much for the detailed explanation!
It mirrors my expectation of anything embedded systems. But it also helps understand what factors brought us to where we're now. And yes, it does help answer the question my brain is shrieking of why anyone would connect fridges to the open internet ...
@hacks4pancakes I'd be interested to hear your thoughts about legislation to raise the base level of product security, like EU's CRA.
@maswan oh, legislation can always push this problem forward! But it has to come with funding, resources, and education!
@hacks4pancakes @maswan Have you seen legislation similar to CRA in other jurisdictions?
I have seen several countries more or less copying the GDPR and integrated it into their own legislation, and I was wondering if the same is happening with CRA?
@hacks4pancakes
For commercial products, the optimist in me sees a way of Reasonable Security Product A not being totally undercut by the Security Is The Customer's Problem Product B.
For the rest, as someone who runs publicly funded research infrastructure, I certainly know the pain of "You must now also do A,...,H - within the same budget".
@hacks4pancakes I'm not sure we can just blame capitalism. Governments make the same short-sighted decisions for much the same reasons...
@hacks4pancakes Oof, I hope “OT” dies a fast death. It expands to something long-winded that seems to just imply “ops”. But then the expansion is also misleading because what you’re really trying to say is telemetry. I think enough people generally understand “telemetry”, so why not stick with that?
@hacks4pancakes I am guessing OT is not occupational therapy, in this context :)
@hacks4pancakes cool, thanks. It's a great article, this makes it more accessible.
@hacks4pancakes Thank you for this. As someone with years of experience in IT security and only the most minimal view of OT security, this is an enormously helpful article.
@hacks4pancakes It was a really good read, and I learned a lot. Thank you :)
@hacks4pancakes @mitexleo An excellent description of how over-optimisation (to maximise extractive value) leads to extreme fragility.
The cost of replacing mountains of stock will not be accounted for as a consequence of the poor decision-making.
@hacks4pancakes The answer to “why is XYZ a clusterfuck?” is always “because doing it properly would have cost money”.
@hacks4pancakes Honestly, I'm more enraged about the recent MS Paint issue with Paint being exploitable zero-click over network rather than OT connected to the Internet. Yes, the latter has terrible impact, but as you said... there are reasons (despite most of them not being purely technical but "just" monetary/staffing). Unlike with Paint.
@hacks4pancakes
"So clever engineers and businesspeople looked at the IT landscape and had an idea. Using Windows computers as human-machine interfaces, and commercial Ethernet to encapsulate serial traffic, would be much cheaper to buy, replace, and train technicians to support. "
And that's were the horror started. 😬
@hacks4pancakes this article reminds me of those HR emails that say a lot, too much jargon in fact, but they don't ever explain anything.
@hacks4pancakes
I'm a public sector water utility civil engineer and supervisor. I work closely with and rely on my colleagues who maintain our SCADA system (and they rely on me).
This article was useful. Thank you.
@hacks4pancakes I often find myself wondering if the a competently administered homelab wpuls be more secure than most corporates, because the only barriers they have to being more secure is technical.
There's no need to grapple with the organisational messes to do what needs to be done to lock things down. And it seems like organisations, people and politics are the bigger challenge to doing technology well than the technology is.