Chapter 2 We will use the concept of a rational agent to define a system that can be called intelligent. We’ll define rational agents that behaves as well as possible in a given environment. 2.1 agents and environments • An agent is anything that can be viewed as perceiving its environment through sensors and acting upon that environment through actuators. • Percept – the agent’s perceptual inputs at any given instant. Percept sequence – complete history of everything the agent has ever perceived. An agent’s choice at any time can depend on the entire percept sequence observed to date. • Mathematically speaking, we say the agent’s behavior is defined by the agent function that maps any given percept sequence to an action. • Tabulating the agent function would result in a giant unusable table and take up too much space to be stored. The table might be an external characterization, and the agent function is an internal thing implemented by an agent program. • The notion of an agent is a tool for analyzing systems, not an absolute characterization that divides the world into agents and non-agents 2.2 good behavior: the concept of rationality • A rational agent does the right thing: that which will make the agent the most successful. Thus we need to quantify success. Performance measures • A performance measure embodies the criterion for success of an agent’s behavior • An agent generates a sequence of actions in response to percepts. The sequence causes state transitions. If the sequence is desirable, the agent performed well. Agents should not have subjective opinions about their success or they would report it all the time, thus we must have objective performance measurements imposed by the agent’s designer(s). • As a general rule, it is better to design performance measures according to what you want in the environment, rather than according to how one thinks the agent should behave. • Averages of behavior are confusing, as there could be dramatic highs or lows whose average is equivalent to that of a stable system that has no major fluctuations. Rationality • Rationality depends on four things o Performance measures used as criterion of success o Agent’s prior knowledge of the environment o Actions the agent can perform o Agent’s percept sequence to date • The definition of a rational agent is that for each possible percept sequence, a rational agent should select an action it expects to maximize its performance measure, given the evidence provided by the percept sequence and whatever built-in knowledge the agent has. • We need to define a performance measure in terms of the environment, inputs and outputs (percepts and actuators). Omniscience, learning and autonomy • Difference between rationality and omniscience: an omniscient agent knows the actual outcome of its actions…but this isn’t possible in reality. Rationality is not the same as perfection. Rationality maximizes expected performance, while perfection maximizes actual performance. • Our definition of rationality doesn’t require omniscience. Rational choices depend on percept sequence to date. We must ensure we don’t allow the agent to behave decidedly underintelligently. (example of crossing the street, if a piano was dropped on a rational agent trying to cross a street we couldn’t call it irrational…that’s an odd circumstance) • Rational agents must be able to gather information and learn as much as possible from what it perceives. • Successful agents spread the task of computing into three different periods: o Agent is being designed o Agent is deliberating on its next action o Agent is deciding how to modify its behavior • An agent lacks autonomy in the sense that it relies on the knowledge of its designer, but a rational agent should be able to compensate for this by learning. It doesn’t usually have complete autonomy from the start, when it has no experience to build on…it would have to act randomly if not for help from the designer. After sufficient experience it should grow to being effectively independent of its prior knowledge. 2.3 the nature of environments • A task environment is the “problem” to the rational agent, which is the “solution” Specifying the task environment • PEAS: o Performance: the desirable qualities, goals and tradeoffs, ways to maximize profit/utility o Environment: “things” the agent will interact with, including other agents, conditions, etc. the more restricted the environment, the easier the design problem o Actuators: methods of interacting with the environment, such as motors, gears, output screen, etc. o Sensors: ways to get information about the environment, inputs • What matters is not the distinction between “real” and “artificial” environments, but the complexity of the relationship among the behavior of the agent, the percept sequence generated by the environment, and the performance measure. • Some software agents exist in rich, unlimited domains (e.g. internet) Properties of task environments • Fully observable vs. partially observable: o Fully observable: an agent’s sensors give it access to the complete state of the environment at any point in time, effectively fully observable if the sensors detect all aspects relevant to the choice of action (depends upon the performance measure). Convenient because agent won’t need to keep track of the world. o Partially observable: cannot detect all relevant aspects, such as the state of mind of another agent • Deterministic vs. stochastic: o Deterministic: next state is completely determined by the previous state and the action of the agent o Stochastic: the next state is not completely dependent on the prior state plus the agent’s actions. A partially observable environment can seem stochastic. o it’s important to think of these views as being from the point of view of the agent. o Strategic: a special case of a deterministic environment where the next state is determined by the previous state and the agent’s actions and the actions of other agents • Episodic vs. sequential: o Episodic: the agent’s experience is divided into atomic episodes (turns), each episode consists of the agent getting percepts, then performing a single action…crucially, the next episode does not depend on the actions taken in prior episodes. Simpler environment, agent doesn’t have to think ahead. o Sequential: the current decision can affect all future decisions. (e.g. chess or driving in traffic) • Static vs. dynamic: o Static: easy to deal with, environment doesn’t change while the agent is deliberating…no need to worry about time o Dynamic: environment can change while agent is considering next action…agent must keep looking at the world. o Semidynamic: special case where the environment itself doesn’t change during deliberation, but time affects the performance score • Discrete vs. continuous: (state, time, percepts, actions) o State, time, percepts or actions can be seen as discrete or continuous. o State:  Discrete: chess game (finite discrete states)  Continuous: traffic on I-5 o Time:  Discrete: turns/episodes  Continuous: real-time o Percepts:  Discrete: digital camera input  Continuous: interpretation of frames of digital camera input over time o Actions:  Discrete: one move in chess  Continuous: steering a car • Single agent vs. multiagent: o Single agent: only one agent in the environment o Multiagent: chess game. Communication is often rational behavior in multiagent environments. In a partially observable competitive environment, stochastic behavior is rational because it avoids the pitfalls of predictability. o Distinctions: if a second agent’s maximized performance depends on the first agent’s, then it’s a multi-agent environment  Competitive: chess  Cooperative: (most) driving situations • the hardest case is partially observable, stochastic, sequential, dynamic, continuous, and multiagent. • You don’t care about specific performance, generally. You care about behavior in many environments based on an environment class. The online code repository has an environment generator for that reason. 2.4 the structure of agents • The job of AI is to design the agent program that implements the agent function, mapping percepts to actions. It will run on some sort of device with sensors and actuators, which we call the architecture. The architecture makes the percepts available, runs the program, and feeds the action choices to the actuators. Agent programs • Takes the current percept, returns an action to the actuators • The agent program takes the percepts, the agent function takes the entire percept history. • Tables are not feasible because of their intractable growth. • The key challenge for AI design is to produce rational behavior from a small amount of code rather than a large number of table entries. • There are four types of agent programs: o Simple reflex agents o Model-based reflex agents o Goal-based agents o Utility based agents Simple reflex agents • Selects actions based on the current percepts, ignores the percept history • The connection between percept and action is called a condition-action rule and is usually something like an if-then. The first rule encountered in a set is returned. • A flexible approach to designing these is to build a general-purpose interpreter for condition-action rules, then create rule sets for specific task environment • These are simple, but they have limited intelligence. They only appear rational in cases where the decision can be based entirely on the current precept, which requires that the environment be fully observable. • Infinite loops are sometimes unavoidable in partially observable environments. This can be escaped by randomizing the actions. In single-agent environments, randomization is usually not rational. Model-based reflex agents • Keep track of the part of the world that the agent cannot see, keep track of internal state that depends on the percept history. • Two kinds of knowledge must be encoded in order to do this: o Information about how the world evolves independent of the agent o How the agent’s actions affect the world • Knowledge of how the world works is called a model of the world, thus the term “model-based agent” • The current percept is combined with the old internal state to update the state, then the expected outcome of actions is combined to decide upon an action Goal-based agents • Knowing the environment’s state isn’t always enough, a correct decision must be based upon goals that state which outcome is desirable. This info gets combined with the state, world model and expected action results to decide on a utilitarian action. • Search and planning are subfields of AI devoted to finding action sequences that achieve the agent’s goals. • Boils down to “what will happen if I…?” and “what will make me happy?” Utility-based agents • Goals alone aren’t always enough. Goals are just “happy” and “unhappy”. If one world state is preferred to another, then that state has higher utility for the agent. • A utility function maps a state or sequence of states to a real number describing the degree of happiness. This helps in two kinds of cases where goals would be insufficient: o Conflicting goals where one can result in higher value o Several goals of different probabilities of success, the probability can be factored in to the comparison of utility • Any rational agent must behave as if it possesses a utility function whose expected value it tries to maximize. Learning agents • Turing proposed methods to teach machines. • Learning has an advantage: allows the agent to operate in initially unknown environments and to become more competent than its initial knowledge would otherwise allow • There are four conceptual components to a learning agent: o Learning element: responsible for making improvements o Performance element: responsible for selecting actions. Performance standard should be fixed, since an agent could otherwise modify the standard to fit its behavior. A performance standard can distinguish part of the incoming percept as a reward or penalty. o Critic passes feedback to the learning element, which is used to determine how the performance element should be modified to do better going forward. Necessary because percepts are meaningless when taken alone. o Problem generator: suggests actions that will lead to new and informative experiences. Might find areas in need and suggest experiments there. • Question to ask during design: what kind of performance element will my agent need once it has learned to do things in the environment? Single unifying theme: learning in intelligent agents can be summarized as a process of modification of each component of the agent to bring the components into closer agreement with the available feedback information, thereby improving the overall performance of the agent.