The Stanford 100 Year Study on AI discusses the potential of AI research and future advances. The report states that AI advancements in transportation, specifically driverless cars will become an integral part of the society in the near future.

In my opinion, it is too early to comment on this. There are some aspects which the study panel may have failed to take into consideration. Even though there is a lot of active and focused research going on, there are many issues to be addressed. I have tried to shed light on those issues.

Autonomous cars can be designed very intricately, taking into consideration a myriad of parameters and using many complex learning techniques, but, for any software designing team, it is very hard to completely factor in all possible circumstances. There may be situations like a person running across the street, change in road surfaces, drastic weather conditions, blind turns into traffic and other unexpected scenarios which cannot completely be accounted for. In such situations, how the AI will work cannot be accurately determined.

In the tragic accident that killed a Tesla driver in 2016 in Florida, whether the driver failed to control the vehicle, or he erroneously assumed that the AI in his semi-autonomous driving would avoid the crash remains unknown. Such incidents blow away the sense of confidence that the public had started developing in self-driving cars; trust is very hard to regain.

Ethical choices in case of unavoidable circumstances have always been a topic of debate. AI modeling unknowingly may reflect biases based on gender, age, cultural, social influences and not reflect an unbiased decision. In such cases who is to be held responsible? Yet proper legislation has not been formulated. So, until these questions are answered and definite boundaries are set, governments are reluctant to go ahead with such projects.

The results of driverless car testing are not very satisfying. Waymo has been the frontrunner in terms of testing its autonomous vehicles. According to the California DMV, Waymo has the best record, with 0.2 disengagements per 1000-mile rate. But this data can be a bit misleading. This is because Waymo does not count every manual intervention as a disengagement. Waymo simulates whether the AI system may have done something incorrect had the intervention not been done and only then counts it as a disengagement. Thus, such frequent intervention regardless of whether considered a disengagement itself is an indicator that we are far away from dependable, robust AI systems.

Like a double-edged sword, with powerful systems like AI, there is always a downside risk associated with it. Self-driving cars do not follow human driving intuitions. They work based on certain complex algorithms. Total reliance on AI to 'see' and interpret objects and signs correctly while driving is not completely safe. AI may get fooled and can incorrectly detect objects. Quoting from research by University of Wyoming based on DNN fooling in 2015: " It is easy to produce images that are completely unrecognizable to humans, but that state-of-the-art DNNs believe to be recognizable objects with 99.99% confidence". If a knowledgeable hacker finds the blind spots in an image detection algorithm and manipulates for harmful means, it could lead to catastrophic outcomes. Cost and scalability remain as major issues.

All this does not mean that self-driving cars are not practical. It only underscores that many steps, many advances, for ensuring safety and security, are yet to be invented. Certain critical questions are yet to be answered before driverless cars become an intrinsic part of the society.

