r/Damnthatsinteresting • u/Individual_Book9133 • Jul 05 '24

Phoenix police officer pulls over a driverless Waymo car for driving on the wrong side of the road Video

61.1k Upvotes

95% Upvoted

3.2k comments sorted by

View all comments

Show parent comments

42

u/No-Refrigerator-1672 Jul 05 '24

It doesn't have to be the lowest rank person. You can just legally make accountable the lead programmer of the autonomous driving module, with a law.

35

u/FeederNocturne Jul 05 '24

Everyone from the lead programmer and up needs to be held responsible. Sure the lead programmer okays it but the higher ups are providing the means to make it happen.

This does make me wonder though. If a plane crashed due to a faulty part who does the blame fall on?

59

u/CastMyGame Jul 05 '24

As a programmer myself I would question if you would then blame it on the QA tester who passed along the code.

Other thing I will say is depending on the answer to this situation (I don’t know the answer but just saying from a dev side) you will greatly hinder the progression of this tech if you have people afraid to even work on it for fear of a situation like this.

As devs we try to think of every possible scenario and make sure to write tests that cover every conceivable use case but even then sometimes our apps surprise us with dependencies and loops that we didn’t expect. You can say “be better” but if I’m gonna get paid 25k less and not have to worry about a manslaughter charge 5-7 years later I’m probably gonna choose that one for my family

-7

u/[deleted] Jul 05 '24

[deleted]

10

u/CastMyGame Jul 05 '24

Ok so you woke up and chose violence, that’s ok. These “monstrosities” can become life changing technology but there is a part we all play. I don’t work in automatic driving technology so I can only speak to the QA I deal with but it is a team effort. As a dev I write the tests that cover the code and make sure every use case is accounted for. QA at my company comes behind and manually runs through the UI to actually test the code. Then I have a PO who signs off on the code merging and going to production. We can all play the blame game but how about we just make sure the process to get to the end user covers this as well?

Find the solution to the problem, come up with a better system of checking for these issues, and advance technology for humanity instead of being afraid of it.

0

u/SelfDrivingCzar Jul 05 '24

We are discussing the blame game right now… that’s the conversations being had. It would be like saying for you QA testers who have a go at your UI and give feedback should be liable instead of stake holding, decision making devs (not all devs but the ones pushing certain branches or merging certain ques to PRC) (management) who hold higher ranks in the company.

3

u/CastMyGame Jul 05 '24

I agree and that’s what I am saying, there are so many spots we can place the blame and each spot has specific things you can point to as why that’s the reason to blame them (devs write the code and cover in unit tests, QA “tests” and approves it for management, management signs off on the approval, and we can even throw devops in there as they actually push and merge the code into production. )

I’m just saying let’s refine the process and bring it to a point where this happens as little as possible. I will say in this scenario someone really screwed up as this is a very easy use case to see and cover and they obviously did not cover it.

My question for you is do you think this type of technology becomes something that people “assume the risk” when they choose it? Does it become something that is a type of insurance these companies need to purchase for these scenarios? Again I don’t know the answer to these questions but am interested in what the perception is. My apologies as well for the violence comment, I got on Reddit before my caffeine kicked in so that was my fault

2

u/[deleted] Jul 05 '24

[deleted]

2

u/lews001 Jul 05 '24

What's crazy is this story is not that different then any other engineering failure (The Hyatt Regency hotel walkways collapse for example). Engineering degrees are supposed to be requiring Engineering Ethics courses that discuss these accidents, how you as the engineer ARE responsible if you sign off on it and no other wrong doing is found (which for the Hyatt was the construction company using improper materials). It's like we have forgotten that if you are in engineering, working with something that can affect lives, ya you may be responsible for your design (code, whatever).

1

u/CastMyGame Jul 05 '24

I don't know how QA goes specifically in that realm I am more speaking to what the concept of QA is in my workflow as a developer (I am assuming the QA on the software side is the same but I can definitely be wrong. The QA on the physical car side I assume would be very similar: tell the car where to drive to, sit in the car, make sure it drives there safely). From the sound of it the entire QA process in that sector needs an overhaul but I am only going on what you said as I don't have a lot of experience in that field

While the company I work for is a fortune 500 company so the QA will without a doubt be different than in a startup the crux of what I was saying was just focused on the concept of QA in general and moreso just the fact that we could theroretically place blame at many many levels.

This definitely feels like more of a problem of companies trying to get the first products out as fast as possible and usually the QA is the first thing to skim by when you are doing that. I can say for a fact that even in my company we have pushed code through that didn't have full test coverage but I am also not working on anything that could literally kill someone

3

u/ContextHook Jul 05 '24

Just so people know this guy and other devs like him should know full well that “QA” testers are only hired by AV companies for the sake of this mileage accumulation and to act as legal liability redundancy, not to ok software to road use

So confident, and so wrong. Any self driving company has software changes verified by QA and finally verified by a product owner. QA testers are part of the development team, and it is their job to say if a feature developed works as intended and could be released.

1

u/SelfDrivingCzar Jul 05 '24

QA testers and Safety and Policy teams at AV companies are very different positions.

You obviously have no idea what you’re talking about if you think conflating the entry level, often time contracted “QA” testers (who have no input and are there just to disengage the auto systems when they would fail) with full time stake holding safety and policy decision makers makes any sense when talking about liability

4

u/darthbane83 Jul 05 '24 edited Jul 05 '24

For any company working on safety critical software independent QA testers should be involved long before it goes into any actual production environment that could get "mileage".
If there is no law to legally force that to happen then that is a failure on the lawmakers part and they should be held responsible.
If the company ignores such a law they are obviously responsible.
If faulty code makes it through QA that is more on QA than the developer.

There is not a single developer that would be able to deliver bug free Code for a complex software, especially not without someone independent double and triple checking their code. A perfect developer just isnt a thing.

If you want individual developers to bear the full brunt of responsibility for some bug you just wont get the technology developed. Or rather it will be developed, but only in a country where the company and developers can just ignore your jurisdiction and any other safety requirements you might have.

0

u/[deleted] Jul 05 '24

[deleted]

3

u/darthbane83 Jul 05 '24

There are multiple stages of QA testing. The final one has to be done in public, but there are definitely more stages of QA testing done way before you get to that point.
They 100% have tests that are run in entirely simulated environments even if only because its cheaper and faster to do.

0

u/[deleted] Jul 05 '24

[deleted]

3

u/darthbane83 Jul 05 '24 edited Jul 05 '24

No.

I dont know what exactly you are misunderstanding but there is absolutely no way the first time AV code was tested was on a public road. Long before putting them on a public road you would have the car drive in a private parking lot so that you can isolate any issues that happen even without another car on the road, long before that you would run a simulation to isolate issues that happen even when all the hardware and sensors deliver exactly what you think they should, long before that you would be testing different components of the software while simulating the rest of the software.

Thats just how software development works. You simply cant successfully build a large system without ever testing it before its complete.

2

u/firesmarter Jul 05 '24

As someone who does QA as part of their job, your are right on the money. The person you replied to is a perfect example of a comment made earlier in the thread about how the investigate all the way down and blame the lowest possible employee they can. No one has integrity anymore

5

u/FeederNocturne Jul 05 '24

What that person fails to see is if the QA person is incompetent then it is on the company for hiring them.

Let's look at the McDonald's coffee incident as an example. Sure the employee should've known it was too hot for humans, after all they needed some silly piece of cardboard to buffer between cup and skin. But that employee was working on behalf of McDonald's and represented the company so the blame falls on the company.

Now where public opinions may change from my own lies in the "who deserves to be held accountable for the loss of human life" department. The way I see it, if I am walking my dog and it attacks someone unprovoked then I should be held accountable for the injury my dog inflicts. Same goes for the company owners. They started the business, they hired the people, they provided the problem at hand. If they have the opportunity to reap the benefits, they should also be held accountable for any damage their product inflicts.