Boeing’s executives had motivated reasoning to skimp on actual engineering and give the results the exec wanted over the objections of actual engineers.
Sound familiar?
Frankly, no. LLMs are so sychophantic, there’s little evidence they would have flagged a warning that what he executives wanted was dangerous.
And no, LLMs are not just some tool. Never in human history has a hammer given it’s users psychosis.
The max issue on the Boeing planes wasn’t a software issue in that sense. The software actually worked as intended. Boeing just hid the fact that the software existed and made it purposefully hard to override, giving pilots quite the surprise when the plane didn’t operate the way they expected, especially when the angle of attack sensor wasn’t working.
If someone demands a piece of software that completely erases all available data when a certain button is pressed, no one is told about it, someone presses it by accident and deletes a whole bunch of data, that’s not a software development error. That’s a management failure. The software did exactly what it was supposed to do.
And LLMs wouldn’t make anything better in that regard. An LLM is built to do as the user wishes, with much, much less pushback than a human. So whereas a human getting the demand to write that software might say, “what in the actual fuck are you smoking and can I have some of it”, an LLM will typically say, “that’s a very bold decision!” In fact, in the case of MCAS, the issue of safety was raised by the engineers many times, and was flat out ignored by management. An LLM is much less likely to make a fuss and much more likely to just do it.
Removed by mod
Boeing’s executives had motivated reasoning to skimp on actual engineering and give the results the exec wanted over the objections of actual engineers.
Sound familiar?
Frankly, no. LLMs are so sychophantic, there’s little evidence they would have flagged a warning that what he executives wanted was dangerous.
And no, LLMs are not just some tool. Never in human history has a hammer given it’s users psychosis.
This is a completely nonsense argument.
Removed by mod
The max issue on the Boeing planes wasn’t a software issue in that sense. The software actually worked as intended. Boeing just hid the fact that the software existed and made it purposefully hard to override, giving pilots quite the surprise when the plane didn’t operate the way they expected, especially when the angle of attack sensor wasn’t working.
Removed by mod
If someone demands a piece of software that completely erases all available data when a certain button is pressed, no one is told about it, someone presses it by accident and deletes a whole bunch of data, that’s not a software development error. That’s a management failure. The software did exactly what it was supposed to do.
And LLMs wouldn’t make anything better in that regard. An LLM is built to do as the user wishes, with much, much less pushback than a human. So whereas a human getting the demand to write that software might say, “what in the actual fuck are you smoking and can I have some of it”, an LLM will typically say, “that’s a very bold decision!” In fact, in the case of MCAS, the issue of safety was raised by the engineers many times, and was flat out ignored by management. An LLM is much less likely to make a fuss and much more likely to just do it.
Removed by mod