Every system will always have flaws and so is a system which is coherently developed with very good AI. The Study Panel report summarises the reflections about AI in the past and how it will grow and function in the future. The most recent story of Facebook shutting down its “negotiator bots” due to the bots developing its own language for negotiation is an example of how AI could well work within the given framework but still misbehave according to the ground rules laid out by us. Similar problems have been seen in the recent past and based on these instances, there are a few aspects of Study panel report that are challenged subsequently.

The first category that this report challenges is public safety and privacy. The Study Panel report emphasises on how AI can provide better security and safety to the public through white collar crime detecting softwares and automatic identification of crimes from camera and so on. Although, these are great advances, two recent incidents of misuse of AI have put a mark on this particular domain. First, an insurance company, tried to use Facebook data of users to see if they could fit into a particular scheme called “firstcarquote” and mentioned that if they did qualify they could return with very good savings. Facebook then blocked this particular company as they were in violation of Facebook’s Platform Policy of using data for eligibility criteria. This is a case of the privacy of a user being intruded by AI. Another example is of Microsoft’s chatbot Tay which turned racist on Twitter. The main reason for this failure was that the bot was not taught on how to understand inappropriate behaviour from other users. Though, the Study Panel report mentions that every AI system developed should not be biased and should work in the best interest of the public, it is probably not yet possible to develop AI to handle such diverse data when the scope of data is so wide. Therefore, a longer period may be required before AI is let in to handle an area as sensitive as human security and safety.

Healthcare services and AI is probably the most exciting combination that is due by 2030. If it comes off well, then the implications for our future generations could be great. The Study Panel mentions that for AI to be successful in this domain it has to be accepted by the doctors, nurses and EHR will have to be supported and most importantly, AI will have to gain the patient’s trust. One important point to note is that if AI is introduced in the clinical surgery field or even as an “AI Physician”, the dataset in this field is ever growing and diversifies by the second. Hence, to be able to train a robot to perform a task which appears to be redundant could become fatal because of an unknown variable. Healthcare workers are trained to not only pass the required knowledge criteria but they are also trained on how to deal with circumstances which are completely unforeseen and hence, an “AI Physician” or a “Clinical robot” could find itself out of depth in this domain. Therefore, AI in this domain will need a much longer period of training and supervision with very large amounts of data before actual practice is started.

The above mentioned two categories are the most prominent challenges that AI will have to deal with before it can be realised efficiently with people’s trust in any urban city.