Discussion
“Why are these systems even on a network? Just disconnect them!”
I see this after every OT attack, usually from smart IT people. As someone who responds to these incidents, I wish it were that easy. I wrote up how we got here, and why water utilities and refrigerators keep getting hit.
https://tisiphone.net/2026/11/08/lets-talk-about-why-all-this-ot-stuff-is-connected/
The problem already is in the devices firmware themselves. Usually they have no permission model beyond everything or nothing. You want to read temperatures? Fine. But across exactly the same bus you get access to setting the temperature, or even uploading new firmware entirely with usually the thinnest of default passwords and encryption that would not have passed muster in the Napoleonic wars.
@hacks4pancakes "sensible business decisions" are the reason we've got PFAS in the water supply.
Putting on my grey hat for the moment: increasing the costs of remediation above the cost of implementing better architectures should (ultimately) result in better outcomes. I just hope we survive the interim ;)
@hacks4pancakes Spot on with the assessment that most assume the DMZ+jump box model should be standard practice (if not full air gap).
and it totally makes sense that it was a business decision behind abandoning the concept that industrial controls systems have no business being on the internet.
I appreciate the explanation, and i suspected it was probably something like you described.
@hacks4pancakes I understand why it's difficult to disconnect it. But why not VPN it to reduce the attack vector?
@hacks4pancakes Thanks for this Lesley. Very educational.
@hacks4pancakes I often find myself wondering if the a competently administered homelab wpuls be more secure than most corporates, because the only barriers they have to being more secure is technical.
There's no need to grapple with the organisational messes to do what needs to be done to lock things down. And it seems like organisations, people and politics are the bigger challenge to doing technology well than the technology is.
@hacks4pancakes 100% If we accept that there’s some value in reaching them remotely for monitoring or whatever, that still doesn’t mean they need to be on the _public_ internet. Make them accessible only on the local network, perhaps reachable by VPN. At least that way you’re not entrusting the devices themselves to have decent security (they won’t).
@hacks4pancakes
Thanks for this; I hadn't realised how big a part OEMs played in the rise of these problems.
@hacks4pancakes There was this post on Reddit a few days ago, which nicely explains how these systems end up open to the Internet:
@hacks4pancakes
A great summary of the journey. Understanding the whole problem is only way to develop appropriate solutions.
@hacks4pancakes I am reminded of a startup I worked for about 10 years ago to ensure cold chain. We cared about security. It made our devices more expensive. The customer did not care. It went out of business.
I hear you and I put mouse-over definitions on all the acronyms.
Yes guys, the answer is capitalism.
With a side of convenience and laziness capitalism.
But we're stuck with the problem.
@hacks4pancakes But Capitalism told me the cheapest way was always the morally correct answer!
How am I supposed to operate in a world where consequences other than "saved/made more money" have to be considered?
@hacks4pancakes One of my early hair-raising moments was when I worked at Ford 20+ years ago. I found that they had 19/8 and used it for everything. Workstations. Servers. Robots on factory floors. RFC1918 hadn't been ratified when their network plan was originally developed. Further, many of their systems had RTOS/Embedded kernels that literally required editing source and recompiling to change the IP address. Having even an hour of downtime for deployments of updated firmware (assuming all went well) would have amounted to millions and millions of dollars in lost productivity. This was not going to happen during "Way Forward" (https://en.wikipedia.org/wiki/The_Way_Forward) when they were targeting the layoff of around 20,000 people.
Additionally, we rebuilt their time and paysheet system in Linux VMs that had an emulation layer so HP MPE mainframe software would run on them and display using an in browser TN3270 emulator. Why not a full rewrite? It would have cost more to provide required training for UAW workers than playing horseshoes and hand grenades with an abstraction layer that left the input and output the same, with supported hardware from the 21st century in the middle.
I'm sure you have more horrifying anecdotes that are more recent, but (a) my NDA has expired and (b) if it was that bad then, I can't imagine what the current landscape is like.
@hacks4pancakes There is also more than a little bit of racism involved too.
@hacks4pancakes Good article. I would simply blame capitalism since many companies/utilities do not do the most basic protections that are free. 1. Change all default passwords and create good passwords. 2. When someone leaves the company delete their access and if there is a shared password change that password. 3. If possible delete default accounts. So many objects on the internet still have an account name and password of “admin/admin”
@hacks4pancakes thank you so much for the detailed explanation!
It mirrors my expectation of anything embedded systems. But it also helps understand what factors brought us to where we're now. And yes, it does help answer the question my brain is shrieking of why anyone would connect fridges to the open internet ...
@hacks4pancakes I'd be interested to hear your thoughts about legislation to raise the base level of product security, like EU's CRA.
@maswan oh, legislation can always push this problem forward! But it has to come with funding, resources, and education!
@hacks4pancakes @maswan Have you seen legislation similar to CRA in other jurisdictions?
I have seen several countries more or less copying the GDPR and integrated it into their own legislation, and I was wondering if the same is happening with CRA?
@hacks4pancakes
For commercial products, the optimist in me sees a way of Reasonable Security Product A not being totally undercut by the Security Is The Customer's Problem Product B.
For the rest, as someone who runs publicly funded research infrastructure, I certainly know the pain of "You must now also do A,...,H - within the same budget".
@hacks4pancakes I'm not sure we can just blame capitalism. Governments make the same short-sighted decisions for much the same reasons...
@hacks4pancakes Oof, I hope “OT” dies a fast death. It expands to something long-winded that seems to just imply “ops”. But then the expansion is also misleading because what you’re really trying to say is telemetry. I think enough people generally understand “telemetry”, so why not stick with that?
@hacks4pancakes I am guessing OT is not occupational therapy, in this context :)
@hacks4pancakes cool, thanks. It's a great article, this makes it more accessible.
@hacks4pancakes Thank you for this. As someone with years of experience in IT security and only the most minimal view of OT security, this is an enormously helpful article.
@hacks4pancakes It was a really good read, and I learned a lot. Thank you :)
@hacks4pancakes
Thanks for the detailed explanation! It didn't occur to me that "industrial refrigerators" fell into the same device class as PLC-based systems, but it makes sense with this context.