Artificial Intelligence and Autonomous Machines: Influences, Consequences, and Dilemmas in Human Care

In the field of robotics and in the health sciences, transitions have been occurring in the control of robots operating with predetermined logic and rules. Robotics in health care are influencing human caring dynamics in many ways such as enhancing dependency and surrender to machine technologies. Situations such as these are charged with possibilities of legal liabilities triggered by influences and consequences of advancing robotic technology dependency. The purpose of this paper is to identify, describe, and explain legal issues and/or dilemmas centered on robotics in healthcare while providing engaging opportunities to limit consequent legalities thus forming beneficial human health care outcomes. Laying bare these liabilities will provide critically in-formative data that can foster proactive encounters which can or may deter health care liabilities while ensuring quality healthcare outcomes. An attempt is made to re-conceptualize how to view agency, causality, liability responsibility, culpability, and autonomy for the new age of autonomous robots. While it is still not clear how this would turn out, a clear framing of the problem is the first step in the project.


Introduction
Professor Zeynep Tufekci of the University of North Carolina gave an astounding keynote speech at XPONENTIAL 2018. To paraphrase, two pilots will no longer be needed in the future. Planes will be driven by just one "captain" and a dog. The dog will be there to bite the human captain in case he touches anything. Alan Turing, one of the earliest scholars of artificial intelligence (AI), suggested a functional test that a machine is "intelligent" whenever its behavior is indistinguishable from that of human beings [5].
In the field of robotics, a transition has been occurring in the control of robots operating with predetermined logic and rules. Robots have been given the capacity to learn and express some level of autonomy [6]. These robots have been given sensors that can translate as eyes, ears, and nose [7]. This transition has resulted in challenges in the law which deals directly with liability for injury or property damage [8]. Thus, as techniques in machine learning control more of a robot's behavior, being able to determine who is liable for damages that may occur is a necessary question for consideration of the courts.
In health, the emerging provision of care with robots is expected to exponentially grow together with the anticipated scarcities of healthcare providers, changes in generational demography, demand continuous quality improvement (precision), and aid to the improvement of the quality of life of physically unable [9]. Applications of robots to healthcare include assistance and support to preventive therapies, diagnosis, assistive technology, professional care, rehabilitation treatment, nursing care, pharmacy, mental health, medical information sys-tem, and medical interventions [9]- [14]. Although valuable, these advancements do not evade a potential dilemma. For example, if a nursebot commits an error that harms a patient, who will take the accountability [15]? According to the survey conducted by Ito et al., it is the nurse who operates the robot is accountable [16]. But another ethical question arises when robots are becoming more autonomous. How do we make sure that robot is competent enough to function appropriately and not run into impediments autonomously?
Non-human agents which include robotics and software agents, most especially those using a sophisticated form of AI are becoming increasingly autonomous in terms of the complexity of tasks that they are able to perform [17]. These non-human agents are increasingly unmitigated and have resulted in the diminishing ability of human agents to understand, predict, or control how they operate [18]. As robots equipped with sophisticated AI learn to solve problems using solutions that are unknown to the human operators, and with their increased independence in mobility and decision making, they are increasing the type of skills and abilities that could lead them to actions which are entirely unpredictable [19]. In this case, who can be held liable or accountable if damages to property or injury to people occur, especially if no human was aware of the actions of the robot or was knowledgeable of the workings of the algorithms that control the behavior of the robot? Therefore, managing the liability problem is central to addressing these challenges, while resolving these liability problems will require untangling a set of theoretical and philosophical issues that surround intention, causation, responsibility, agency, compensation, and culpability [20].

Purpose
The primary purpose of this paper is to identify and describe problems of determining liability issues in cases wherein robots may be culpable for injury to human beings in the context of healthcare. Through a theoretical discussion that is informed by existing legal and moral theories grounded in technological capabilities of current and potential artificial mechanical agents, a critical determination of influences, consequences and dilemmas are heretofore offered.

Influences, Consequences and Dilemmas on Robotics in Healthcare
The prevailing ambiguity surrounding the issue of liability has hampered furthering the development of AI, and establishing comprehensive standards of its utilization, thereby creating an environment in which health and community sectors are being reluctant to engage in harnessing AI systems in spite of its revolutionary influences. With the current liability systems specifically designed and created for human beings, liability only cover cases in which the cause of a robot's actions or inaction (omission) can be traced back to a specific human agent such as the operator, the manufacturer, the owner, or the user, and where that human agent could have foreseen and therefore avoided a robot's behavioral errors that was harmful [21]. It is imperative to understand that when a robot takes autonomous decisions, the traditional rules will not be enough to give rise to legal liabilities for damages caused by an autonomous robot, since it would not be possible to determine the party responsible for providing compensation, and to require that party to settle the damage the robot has caused [22]. It is still difficult to determine who is liable when a robot that is capable of making autonomous decision causes harm to human beings or makes damages to property because autonomous decisions by robots disturb the chain of causation.
However, in determining liability for an adverse event involving a robot that has been sold as a product, courts need to determine whether the robot was defective in some way [23]. Under tort law, products liability may be used to determine liability for property damage or injuries which involve products that are manufactured for sale. In common law jurisdictions, the basis for products liability claims can be on a theory of negligence, strict liability, or breach of warranty of fitness [24].
Nevertheless, under established law, the physical embodiment of a robot is considered as "property", and it has no rights of its own no matter how intelligent and autonomous it acts from the control of human beings [25]. If a commercially developed robot is involved in an adverse event, the law that is applied to a product that is placed in the stream of commerce is activated. Robots that are manufactured and are sold are considered products and are subjected to product liability laws. In the U.S., the law of products liability is found in the Article 2 of the Uniform Commercial Code (UCC) which deals with sales of products [26]. In the UCC, there are two important products liability sections namely, the implied, and express warranties of merchantability in the sales of products, in which 2-314 and 2-315 of UCC states that "Where the seller at the time of contracting has reason to know any particular purpose for which the goods are required and that the buyer is relying on the seller's skill or judgment to select or furnish suitable goods, there is unless excluded or modified under the next section an implied warranty that the goods shall be fit for such purpose." Therefore, states in the U.S. have enacted the comprehensive products liability statutes; however, proving that the product is defective is a requirementthis may be difficult to do with autonomous robots that may successfully operate despite not having any mechanical defects, but would still cause and result in property damage or injuries due to the machine learning capabilities of robots [27].

Robots as Products: Defects and Culpability
There are three types of product defects that would make suppliers and manufacturers of robots liable. These are design defects, manufacturing defects, and marketing defects [28]. Design defects are intrinsic in nature; they exist before a product is manufactured. While a product can serve its functions well, its use can be unreasonably dangerous due to a flaw in the design. Manufacturing de-fects occur during the production or construction of a product. Only a minority of a manufactured batch of the same type are flawed in this case. Marketing defects deal with inappropriate instructions and negligence in warning consumers of latent dangers in the product. On the final point, under product liability law, the developer of algorithms controlling a robot can provide a warning in a multitude of ways, an example would be an explicit and implicit documentation of the assumptions inherent in the algorithm, but this has become increasingly more difficult to implement as algorithms become more unpredictable and complex [29].
Product Liability Directive no. 85/374/EEC, is a directive of the Council of the European Union that states that responsibility may be identified upon the robots' manufacturers [30]. The Directive is based on strict liability (responsabilità oggettiva) of the defective product producers, in the event of damage to property or personal injury. There are grounds that can be used to argue that the Product Liability Directive may apply to robots causing damages to goods/individuals.
One instance in which strict liability may be implemented are cases wherein the producer did not properly inform the customer of dangers associated with the autonomous robot or perhaps the security systems of the robot were deficient.
Strict products liability is presumed on the existence of an unconscionably dangerous product whose predictable use has caused injury. Under fact patterns, consumers who have suffered injury caused by a defectively manufactured product may rely on strict liability cause of action.
However, under the doctrine of strict products liability, a manufacturer must assure that its products are suitable for their intended use when they are placed on the market for the consumption of the public. The law of torts will hold the manufacturers strictly liable for any adverse events that result from placing an unreasonably dangerous product into the stream of commerce, without accounting for the amount of care applied in preparing a product for sale and distribution without regard as to whether the consumer purchased a product from, or agreed into a contractual relationship with the manufacturer [31]. Therefore, for a plaintiff that is asserting a strict liability claim against a manufacturer of robots as a manufactured product, he/she must plead and prove that the defendant (manufacturer) sold a product that was defective and unreasonably dangerous at the time the product left the defendant's hand; the product reached the plaintiff without meaningful change; and the defect was imminent cause of the injuries of the plaintiff. Likewise, strict liability cases do not put liability on the manufacturer or other parts of the chain of distribution for changes made to the product after delivery to the consumer unless those changes were predictable [32]. Therefore, predictability makes strict liability laws unsuitable for injury caused by autonomous robots. Furthermore, in tort law, negligence is used to describe behavior that creates unreasonable risk of harm to property and people [33]. Based on the law, a negligent action is something a "reasonably prudent" person would not do. Thus, negligent act of a person is when his/her behavior departs from the behavior ordinarily expected of a person that is "reasonably prudent" under the situation. An issue for autonomous robots is whether the "reasonably prudent person" standard can be applied.
Moreover, a claim for negligence may stand even in the absence of a "defect" under strict liability principles [34]. The requirements for a prima facie negligence claim are: the defendant has a duty to conform to a certain conduct standard; the defendant violated that duty; such violation resulted in the injury in question; and the plaintiff incurred actual damage or loss. In a negligent design theory, a plaintiff is required to show that a manufacturer of robots had a duty to exercise reasonable care in the manufacture of the robot, the manufacturer failed to exercise reasonable care in the creation of the robot, and the conduct of the defendant proximately caused damages to the plaintiff.

Legal Aspects Liabilities in the Commission on Civil Law Rules on Robotics [35]
Liability of the Producer. The producer (as named in Directive 85/374/EEC of 25 July 1985 on the approximation of the laws, regulations, and administrative provisions of the member states concerning liability for defective products) can mean the producer of raw materials and the manufacturer of a finished product or of a part of a component. While the product importer is any person who puts their name, trademark, or other distinguishing features on the product; and/or any person who supplies a product whose producer or importer cannot be identified.
From a general viewpoint, a robot is composed of two things: software and hardware. In effect, the distinction between producer and programmer can be established: the producer is responsible for the whole electronic (sensors and actuators) and mechanical parts and the programmer is responsible for the intern process of the robot (image processing, learning capability, decision-making process, etc.).
• The producer can be responsible in two ways: based on the dangerous qualities of the products (defective product) and based on the product behavior (negligence). • The producer can be responsible on two ways: based on detrimental qualities of his products (defective product) and based on the product behavior (negligence).
Therefore, for the producer to be liable, the consumer needs to prove that he/she has received a defective product or the producer has committed negligence of his/her responsibility to the consumer.
Liability of the Programmer. There are two kinds of programmers: The programmer whose job is to prepare the robot for use by the customer and the user-programmer who is a user that programs the robot that is restricted by the manufacturer's established limitations. The programmer will be liable for the damages caused by robots when the damage is related to faults, errors, or software failures.
A failure is an event that occurs when there is a deviation from correct service because it does not adhere with the functional specification or because a particular specification did not adequately describe the system function. An error is defined as a part of a system's total state that may result in a failure. And, the cause of an error is called the fault. For example, if a user acquired a closed architecture robot, any damage that has been caused by the robot is the liability of the programmer and is seen as a product liability case. The producer bears to own all omission and acts by his/her employees which happen during the course of their employments. However, if the programmer is employed by the producer, the assumption of liability goes to the producer. If the programmer is external, the producer would also bear his/her acts too. Accordingly, the producer also has a procedural advantage if an external programmer has contributed to produce the final results, the producer can direct the action of compensation for damages to the external programmer.
The User-Programmer. When a robot permits a degree of personalization, there is an effect of relegation of liability and voluntary assumption of risk by the user. This relegation cannot be understood as a complete release of the producer from responsibility; the discussion about relegation starts at the assumption that the user is not an expert on robotics and he/she only uses the technology that has been created by others. The assumption of risk by the user is limited only to the risk that he/she is capable of knowing. Robots that are customizable are not "empty"; they possess a minimum knowledge on which to work, given by the software from the factory. Therefore, a distinction needs to be made between wrong customization-calibration and wrong basis programming. In the case of damage caused by a robot that has been customized, the user would be liable if the injury has originated in the wrong final programming only. Nevertheless, if the damage had happened in any way, regardless of whether the robot was properly or badly personalized due to a software problem of the factory, the user would not be responsible.
Liability of the Owner. A distinction is required between the concepts of owner and user regarding the liability of the owner. The owner is the person that has procured the product. The duties of an owner include the machine's maintenance, upgrading, and preservation. The user is the person who is able to use the product. Both can be the same person or not. The owner has the ultimate responsibility over the procured product, therefore the owner will be responsible for negligence. Negligence regulates the liability regimes of other subjects.
The legal regime of product liability is peremptory. This means that it cannot be changed according to the will of the parties, except if the doctrine pro consumatore is applied: any changes of the liability of the producer should be for aggravating it. As a result, damages caused by robots as a result of errors in design, assembly, or manufacturing are usually the producer's liability, but me-chanical failures can be due to the inability in keeping the robot in good condition and that is the responsibility of the owner.
However, in the case of user/owner-programmer, the legal regime has been explained previously for damages caused by wrong customization. With regards to wrong customization, it has been noted that the behavior resulting from education given to a robot should not be confused with behavior that depends strictly on its self-learning abilities when looking to identify the person to whom the robot's dangerous behavior is actually due. Because of this, concern has been expressed about the "bad ideas" that the originator has and those "bad ideas" has been expressed through the robot. The originator is not necessarily the programmer or the owner, but is the person who has given the order to the robot. Over time, as AI moves further away from predetermined instructions and learn, the more robots will display behaviors that were not just unpredicted by their creators but were entirely unpredictable. This point is important because predictability is a key ingredient for liability in negligence because there is a lack of legal rights provided for autonomous robots under current law, therefore, those who write the algorithms and develop analytical techniques that control a robot's behavior are parties that should be considered in the chain of liability that leads to damage or injuries.
Furthermore, it is within the software that the capability for machine learning and algorithms that controls a robot that provides the courts focus in decisions involving the determination of liability [36]. As such, robots that are manufactured and are for sale are considered products and subjected to product liability law.
However, some integral questions need to be asked. What about the status of algorithms and other analytical techniques which may have been provided to a manufacturer of robots by a third party? It is essential to know that if an algorithm is not considered a product, what legal theory would compensation be levied as pertinent to damages that result from decisions that were made by algorithms? It seems that currently established laws may be useful in determining liability for mechanical defects, but not for errors that result from the "thinking" of an autonomous robot and this is a major flaw in the current legal approach to autonomous robots. If the algorithms and analytical techniques that are used to control autonomous robots are not seen as products, perhaps service liability may be a cause of action evolving from products liability. This is perhaps applicable in situations where the line between a service and a product is unclear which may be the case with algorithms that are embedded in robots that enable them to perform a range of "service-like" activities [37]. The same as the actions of human professionals may result in malpractice, it can be argued that a similar course of action be developed for computers and more recently for AI and autonomous robots.

The Robotic Liability Matrix (RLM)
One must consider the different types of software that are involved in the con-trol and learning of autonomous robots. If the autonomous robot that is involved in damages is manufactured without mechanical flaws, to determine liability, courts need to look carefully at the robot's operating system, deep learning algorithms, neural nets, and other analytical techniques controlling the robot's behavior. It is clearly difficult to assign liability to any human being if the party that was the cause of the injury was an autonomous robot guided by machine learning algorithms.
A potential solution proposed by the Commission on Civil Law Rules on Robotics is the Robotic Liability Matrix [38]. The plan is to distribute the liability between the producer and the owner. The Committee's real intention is to apply this liability distribution system to all autonomous robots, with or without a learning capacity. The matrix has key factors to ensure reliable results. The succeeding elements have been identified as acceptable and sound to endow the matrix (See Table 1): The matrix works with identified and isolated accident situations. The matrix determines the level of implication of each agent in the accident. The accident is not an isolated fact, but is integrated by little and multiple facts whose result is the final accident. Each of these facts (called stages) must be analyzed separately. This process of analysis and individualization allows the discrimination between failures in software and hardware and human errors.

Possible Solutions to the Dilemma of Liabilities
Possible solutions to these dilemmas focus on varying views of robots by courts of law. Robots can be viewed much like pets in terms of liability where the person sued does not fully control the actions of the third party or animal that has resulted to an injury but, in some circumstances, is liable for the consequences [22]. Still, another approach for liability are laws that apply to domesticated animals [39]. These laws could be adopted for autonomous robots. For animals that are considered "wild," the courts apply strict liability for any damages that result from what is considered to be "dangerous propensity" of the animal species [22]. But among those who have made a proposal to apply the animal law for robots, robots which have sophisticated AI are more comparable to domesticated animals than wild animals. For domesticated animals, the application of strict liability can only happen if the owner knew or had reason to know, that a specific type of animal (or type of robot for this discussion) had a tendency to bite or attack. For robots, the likelihood to cause an injury can be ascertained through a review of the algorithms used to direct the behavior of the robot; and if the owner, producer, or programmer of the robot had reason to know of the robot's likelihood of committing dangerous acts, liability for the robot's actions would occur [39].
Another possible solution that has been proposed by the European Union is creating a new legal status only for robots [40] [41]. There is a tendency to humanize robots if robots have grown sophisticated enough to exhibit "human behavior." This is a result of AI and the type of work that they do. An important part of the term "artificial intelligence" is the term "artificial" which means that they are manufactured by human beings and can be controlled, limited, and can be equated to any other product [42]. However, the Committee on Legal Affairs of the European Union has taken into consideration that the more autonomous robots are, the less they can be considered as mere tools in the hands of producers, owners, and users [35]. The possibility of creating a specific legal status for the most sophisticated robots (named electronic persons) and to apply electronic personality to cases where sophisticated robots make autonomous decision or interact with third parties [40]. Having a legal status of "electronic person" is seen as similar to the legal personality of a corporation, robots could be held liable for their behaviors and enter into legal agreements [41]. The creation of a separate status for robots would introduce a clear and separate entity in which the actions and behavior of robots are controlled and managed separately from the owner [43]. This also helps in clarifying what happens when robots enter a legal relationship because it defines the relationship between the robot and the third party, and between the owner and the robot. It is necessary to emphasize the role of the robot owner in the creation of this electronic personality [44]. The robot does not suddenly gain obligations and rights that are similar to a humans', but rather the robot owner sets up a legal fiction, of which he is in control, much like a majority shareholder [45].
The development of a rigorous safety standards and establishing safety certification processes for algorithms is necessary in a future where more autonomous robots enter society [46]. Establishing standards is the most effective method to ensure high levels of product safety and provide certainty ex-ante to producers who align with them [47]. To create a suitable framework of processes and institutions that focus on damage and injuries resulting from autonomous robots, input is needed from robotics and AI experts to help establish a regulatory framework [48]. This is because of the complexity of machine learning techniques and the general lack of understanding of techniques in machine learning outside the AI and robotics community.
This would also entail that advisory committees to legislatures and governments should be established to aid in the determination of possible ways to regulate autonomous robots [49]. Any framework developed would need to be flexible enough to take into account both global considerations and local jurisdictional considerations possibly consisting of mutual recognition of safety standards and certification between countries, and the need to comply with international treaties or conventions in the future [50].
Introducing a robust regulatory framework with relevant input from industry, policymakers, and government would create greater encouragement for AI developers and manufacturers to reduce their exposure by building in supplemental safeguards to minimize potential risks to humanity [49]. At the same time, it would facilitate an evolutionary adoption of intelligent AI systems and self-learning robots [51]. This new paradigm would require a re-evaluation of long established legal principles.
Furthermore, potential safety issues with Humanoid Robots include, environmental situations between older adults and HCRs; identification of potential "leakage" of personal information from stored data in the cloud server; and issues of access authority for HCRs' stored data. Therefore, it is necessary to have accurate findings supporting the legislation about HCRs to provide safe and effective care for older adults, and to limit healthcare facilities to reasonable risk level [52].
Another proposal being considered is the creation of a mandatory insurance scheme to ensure that victims of accidents that involve robots and intelligent AI systems have access to adequate compensation [53]. This has similarity with the mandatory Comprehensive Insurance that car owners need to have before being able for vehicle registration. Establishing specific protections for potential victims of robots or intelligent AI-related incidents can give consumers a legal recourse if an adverse event occurs [49].
Creation of an insurance scheme entails defining what a robot is in an insurance policy [51]. The definition of robot may vary widely depending on the type of robot, its function, the insurance product at issue, and the coverage intended.
Policy language will be the primary reference points for disputing parties to turn to for consideration about coverage.

Should a New Liability Regime Be Created for Intelligent
Robots? Unlike conventional engineering and design, the actual functioning of autonomous robots is not entirely predictable in the same way as most engineered systems [60]. Some autonomous robots may be unpredictable in theory, and many will be unpredictable in practice. A critical approach to current legal approaches to liability is predictability. In conventional product liability, the producer is responsible for the product working as it was designed, and predict likely problems or dangers it may cause. Determining what is "predictable" often falls upon courts to decide, but the current legal standards used are whether the producer had any prior knowledge of the potential dangers, whether a reasonable person would have been able to predict it, or whether there is a standard of practice in the industry that would have been able to reveal it.
In the case of advanced AI, a system learns from environmental data and may act in ways that the designers have no possible way to predict, would be difficult to determine liability. When AI systems are permitted to continue altering their functions and learn after they are set, their behavior will become dependent on new input data, which the programmers and users are unable to control or predict. As a result, the behavior of learned functions will, at a certain degree, also be unpredictable.
Another factor that is challenging conventional approaches to liability is that autonomous robots may "act" on their own, yet are not liable in a legal sense.
With most engineered products, the predictability is confined by the actions of other agents such as the users, consumers, service technicians, etc. and how they use and maintain a given product. In those cases, there is a clear legal agent who may have utilized the product improperly and can be held liable for the results of the use in that instance.
There are two fundamental liability frameworks in the law, criminal and civil (primarily tort) liability. In both frameworks, it is difficult to hold the autonomous robot legally liable for its actions, as they are not legal persons. At some point in the evolution of autonomous robots, they might become moral and legal agents, and society will be facing the question of whether entitling them of some or all of the legal rights bestowed on corporations or persons. At that point, some or all of the current laws on liability might be applied to those autonomous robots that qualify as legal persons, though it may still be unclear what the exact boundaries of a particular entity might be, or how to punish it.

Conclusions
In the near future, the problem of hard liability for autonomous robots will lie in constructing a system of liability that promotes innovation that is beneficial while offering adequate and just compensation to those that are harmed. Justice requires that those who will be held liable should be capable of understanding the scope and extent of the risks and liability, they are accepting in deploying autonomous robots, and have some ways of managing that risk through system controls. Adequate compensation is defined as the sufficient means to compensate those harmed, monetary or in other ways-whether this entails holding big corporations accountable, or the provision of proper risk-pooling insurance. One possible result of this is that those producers who are likely to bear the liability burden would seek limitations on the ability of consumers and users to change, adapt, or customize their autonomous robots in order to retain greater control over the way they are used.
We could avert or ban autonomous robots because of their uncertainties and risks. We could be able to call it a precautionary approach. To the extent that it works, it would also hamper the development and deployment of many beneficial advances that autonomous robots have because of their problematic autonomy. We could however, permit the development and deployment of autonomous artificial agents, and accept the costs and risks at a social level, without developing a better framework for regulating autonomy. This may be called the permissive approach and this would allow many beneficial applications of autonomous robots, but also a host of harmful ones, including harms for which no one may be held liable and those who are harmed are not compensated. As an effect, there would likely be a general backlash against autonomous robots as the technology has been seen as harmful without offering restitution for those who have been harmed. Or, perhaps we could pursue one of the heavy handed schemes in liability such as strict liability that would regulate the industry to some extent, but also confine innovation to those areas where there are adequate profits to motivate companies with large capital to enter the market and accept the risks.
Alternatively, we could look for a better solution to the problem of liability than current models can afford. We could attempt to re-conceptualize how we see agency, causality, liability responsibility, culpability, and autonomy for the new age of autonomous robots. While it is still not clear how this would turn out, a clear framing of the problem is the first step in that project.