CS 540 Lecture Notes: Introduction

University of Wisconsin - Madison

CS 540 Lecture Notes

C. R. Dyer

Introduction (Chapter 1)

What is AI?

One answer: A field that focuses on developing techniques to enable computer systems to perform activities that are considered intelligent (in humans and other animals).
Another answer: "It is the science and engineering of making intelligent machines, especially intelligent computer programs. It is related to the similar task of using computers to understand human intelligence, but AI does not have to confine itself to methods that are biologically observable.
"Intelligence is the computational part of the ability to achieve goals in the world." -- J. McCarthy
See also more of John McCarthy's views on this.

Goals of AI

Replicate human intelligence
"AI is the study of complex information processing problems that often have their roots in some aspect of biological information processing. The goal of the subject is to identify solvable and interesting information processing problems, and solve them." -- David Marr
Solve knowledge-intensive tasks
"AI is the design, study and construction of computer programs that behave intelligently." -- Tom Dean
"... to achieve their full impact, computer systems must have more than processing power--they must have intelligence. They need to be able to assimilate and use large bodies of information and collaborate with and help people find new ways of working together effectively. The technology must become more responsive to human needs and styles of work, and must employ more natural means of communication." -- Barbara Grosz and Randall Davis
Intelligent connection of perception and action
AI not centered around representation of the world, but around action in the world. Behavior-based intelligence. (see Rod Brooks in the movie Fast, Cheap and Out of Control)
Enhance human-human, human-computer and computer-computer interaction/communication
Computer can sense and recognize its users, see and recognize its environment, respond visually and audibly to stimuli. New paradigms for interacting productively with computers using speech, vision, natural language, 3D virtual reality, 3D displays, more natural and powerful user interfaces, etc. (See, for example, projects in Microsoft's "Advanced Interactivity and Intelligence" group.)

Some Application Areas of AI

Game Playing
Deep Blue Chess program beat world champion Gary Kasparov
Speech Recognition
PEGASUS spoken language interface to American Airlines' EAASY SABRE reseration system, which allows users to obtain flight information and make reservations over the telephone. The 1990s has seen significant advances in speech recognition so that limited systems are now successful.
Computer Vision
Face recognition programs in use by banks, government, etc. The ALVINN system from CMU autonomously drove a van from Washington, D.C. to San Diego (all but 52 of 2,849 miles), averaging 63 mph day and night, and in all weather conditions. Handwriting recognition, electronics and manufacturing inspection, photointerpretation, baggage inspection, reverse engineering to automatically construct a 3D geometric model.
Expert Systems
Application-specific systems that rely on obtaining the knowledge of human experts in an area and programming that knowledge into a system.
- Diagnostic Systems
  Microsoft Office Assistant in Office 97 provides customized help by decision-theoretic reasoning about an individual user. MYCIN system for diagnosing bacterial infections of the blood and suggesting treatments. Intellipath pathology diagnosis system (AMA approved). Pathfinder medical diagnosis system, which suggests tests and makes diagnoses. Whirlpool customer assistance center.
- System Configuration
  DEC's XCON system for custom hardware configuration. Radiotherapy treatment planning.
- Financial Decision Making
  Credit card companies, mortgage companies, banks, and the U.S. government employ AI systems to detect fraud and expedite financial transactions. For example, AMEX credit check. Systems often use learning algorithms to construct profiles of customer usage patterns, and then use these profiles to detect unusual patterns and take appropriate action.
- Classification Systems
  Put information into one of a fixed set of categories using several sources of information. E.g., financial decision making systems. NASA developed a system for classifying very faint areas in astronomical images into either stars or galaxies with very high accuracy by learning from human experts' classifications.
Mathematical Theorem Proving
Use inference methods to prove new theorems.
Natural Language Understanding
AltaVista's translation of web pages. Translation of Catepillar Truck manuals into 20 languages. (Note: One early system translated the English sentence "The spirit is willing but the flesh is weak" into the Russian equivalent of "The vodka is good but the meat is rotten.")
Scheduling and Planning
Automatic scheduling for manufacturing. DARPA's DART system used in Desert Storm and Desert Shield operations to plan logistics of people and supplies. American Airlines rerouting contingency planner. European space agency planning and scheduling of spacecraft assembly, integration and verification.

Some AI "Grand Challenge" Problems

Translating telephone
Accident-avoiding car
Aids for the disabled
Smart clothes
Intelligent agents that monitor and manage information by filtering, digesting, abstracting
Tutors
Self-organizing systems, e.g., that learn to assemble something by observing a human do it.

A Framework for Building AI Systems

Perception
Intelligent biological systems are physically embodied in the world and experience the world through their sensors (senses). For an autonomous vehicle, input might be images from a camera and range information from a rangefinder. For a medical diagnosis system, perception is the set of symptoms and test results that have been obtained and input to the system manually. Includes areas of vision, speech processing, natural language processing, and signal processing (e.g., market data and acoustic data).

Reasoning
Inference, decision-making, classification from what is sensed and what the internal "model" is of the world. Might be a neural network, logical deduction system, Hidden Markov Model induction, heuristic searching a problem space, Bayes Network inference, genetic algorithms, etc. Includes areas of knowledge representation, problem solving, decision theory, planning, game theory, machine learning, uncertainty reasoning, etc.

Action
Biological systems interact within their environment by actuation, speech, etc. All behavior is centered around actions in the world. Examples include controlling the steering of a Mars rover or autonomous vehicle, or suggesting tests and making diagnoses for a medical diagnosis system. Includes areas of robot actuation, natural language generation, and speech synthesis.

Some Fundamental Issues for Most AI Problems

Representation
Facts about the world have to be represented in some way, e.g., mathematical logic is one language that is used in AI. Deals with the questions of what to represent and how to represent it. How to structure knowledge? What is explicit, and what must be inferred? How to encode "rules" for inferencing so as to find information that is only implicitly known? How to deal with incomplete, inconsistent, and probabilistic knowledge? Epistemology issues (what kinds of knowledge are required to solve problems).
Example: "The fly buzzed irritatingly on the window pane. Jill picked up the newspaper." Inference: Jill has malicious intent; she is not intending to read the newspaper, or use it to start a fire, or ...
Example: Given 17 sticks in 3 x 2 grid, remove 5 sticks to leave exactly 3 squares.
Search
Many tasks can be viewed as searching a very large problem space for a solution. For example, Checkers has about 10⁴⁰ states, and Chess has about 10¹²⁰ states in a typical games. Use of heuristics (meaning "serving to aid discovery") and constraints.
Inference
From some facts others can be inferred. Related to search. For example, knowing "All elephants have trunks" and "Clyde is an elephant," can we answer the question "Does Clyde hae a trunk?" What about "Peanuts has a trunk, is it an elephant?" Or "Peanuts lives in a tree and has a trunk, is it an elephant?" Deduction, abduction, non-monotonic reasoning, reasoning under uncertainty.
Learning
Inductive inference, neural networks, genetic algorithms, artificial life, evolutionary approaches.
Planning
Starting with general facts about the world, facts about the effects of basic actions, facts about a particular situation, and a statement of a goal, generate a strategy for achieving that goals in terms of a sequence of primitive steps or actions.

Design Methodology and Goals

Engineering Goal: Develop concepts, theory and practice of building intelligent machines. Emphasis on system building.
Science Goal: Develop concepts, mechanisms and vocabulary to understand biological intelligent behavior. Emphasis on understanding intelligent behavior.

Alternatively, methodologies can be defined by choosing (1) the goals of the computational model, and (2) the basis for evaluating performance of the system:

1 Think like humans "cognitive science" Ex. GPS	2 Think rationally => formalize inference process "laws of thought"
3 Act like humans Ex. ELIZA Turing Test	4 Act rationally "satisficing" methods

Box 1
"Cognitive science" approach - Focus not just on behavior and I/O, look at reasoning process. Computational model should reflect "how" results were obtained. GPS (General Problem Solver): Goal not just to produce humanlike behavior (like ELIZA), but to produce a sequence of steps of the reasoning process that was similar to the steps followed by a person in solving the same task.
Box 2
The goal is to formalize the reasoning process, producing a system that contains logical inference mechanisms that are provably correct, and guarantee finding an optimal solution. This brings up the question: How do we represent information that will allow us to do inferences like the following one? "Socrates is a man. All men are mortal. Therefore Socrates is mortal." -- Aristotle
Problem : Not all problems can be solved just by reasoning and inferences.
Box 3
Behaviorist approach. Not interested in how you get results, just the similarity to what human results are. Exemplified by the Turing Test (Alan Turing, 1950): 3 rooms contain a person, a computer, and an interrogator. The interrogator can communicate with the other 2 by teletype (to avoid requiring that the machine imitate the appearance or voice of the person). The interrogator tries to determine which is the person and which is the machine. The machine tries to fool the interrogator into believing that it is the human, and the person also tries to convince the interrogator that it is the human. If the machine succeeds in fooling the interrogator, we conclude that the machine is intelligent.
The Turing test is a one-sided test in that a machine that passes the test should certainly be considered intelligent, but a machine could still be considered intelligent without knowing enough about humans to imitate humans.
The Loebner Prize is one version of the Turing Test that is awarded annually to the "most human computer."
ELIZA: A program that simulated a psychotherapist interacting with a patient and successfully passed the Turing Test. Some extracts from the test:
```
   Patient:   Men are all alike.
   ELIZA:     In what way?
   Patient    They're always bugging us about something or other.
   ELIZA:     Can you think of a specific example?
   Patient:   Well, my boyfriend made me come here.
   ELIZA:     Your boyfriend made you come here?
   Patient:   He says I'm depressed much of the time.
   ELIZA:     I am sorry you are depressed.
   ...
   
```
Problems: ELIZA and other similar programs stressed simple syntactic analysis and generation of sentences. They used pattern matching with known sentences. They used pattern matching with known vocabulary and key words with templates of sentences to generate. For example,
if sentence = "* mother *" then respond with "Tell me about your family."
Note that even with simple syntactic style, ELIZA managed to fool people. Purely behavioral-based approach can be simulated without a deeper understanding or true "intelligence."
Box 4
For a given set of inputs, tries to generate an appropriate output that is not necessarily correct but gets the job done. Rational and sufficient ("satisficing" methods, not "optimal").

Most of AI work falls into Boxes 2 and 4. These don't rely on tests that correspond to human performance.

Symbols versus Signals

Most of AI built on an information processing model called a "Physical-Symbol System" (PSS) (Newell and Simon). Symbols usually correspond to objects in the environment. Symbols are physical patterns that can occur as components of an expression or symbol structure. A PSS is a collection of symbol structures plus processes that operate (i.e., create, modify, reproduce) expressions to produce other expressions. Hence, a PSS produces over time an evolving collection of symbol structures.

=> AI is the enterprise of constructing physical-symbol system that can reliably pass the Turing Test [or whatever your performance test is].

Physical-Symbol System Hypothesis (Newell and Simon, 1976): A physical-symbol system has the necessary and sufficient means for general intelligent action.

==> Intelligence is a functional property and is completely independent of any physical embodiment.

==> Develop structural/functional theory of intelligence, i.e., what are the mechanisms, physical or formal structures, which form the basis of intelligent behavior.

An alternative, less-symbolic paradigm: Neural Networks