Third edition of Artificial Intelligence: foundations of computational agents, Cambridge University Press, 2023 is now available (including the full text).
2.4 Embedded and Simulated Agents
There are a number of ways an agent's controller can be used:
- An embedded agent is one that is run in the real world, where the actions are carried out in a real domain and where the sensing comes from a domain.
- A simulated agent is one that is run with a simulated body and environment; that is, where a program takes in the commands and returns appropriate percepts. This is often used to debug a controller before it is deployed.
- A agent system model is where there are models of the controller (which may or may not be the actual code), the body, and the environment that can answer questions about how the agent will behave. Such a model can be used to prove properties of agents before they are built, or it can be used to answer hypothetical questions about an agent that may be difficult or dangerous to answer with the real agent.
Each of these is appropriate for different purposes.
- Embedded mode is how the agent must run to be useful.
- A simulated agent is useful to test and debug the controller when
many design options must be
explored and building the body is expensive or when the environment is dangerous or
inaccessible. It also allows us to test the agent under unusual combinations
of conditions that may be difficult to arrange in the actual world.
How good the simulation is depends on how good the model of the environment is. Models always have to abstract some aspect of the world. Appropriate abstraction is important for simulations to be able to tell us whether the agent will work in a real environment.
- A model of the agent, a model of the set of possible environments, and a specification of correct behavior allow us to prove theorems about how the agent will work in such environments. For example, we may want to prove that a robot running a particular controller will always get within a certain distance of the target, that it will never get stuck in mazes, or that it will never crash. Of course, whether what is proved turns out to be true depends on how accurate the models are.
- Given a model of the agent and the environment, some aspects of the agent can be left unspecified and can be adjusted to produce the desired or optimal behavior. This is the general idea behind optimization and planning.
- In reinforcement learning, the agent improves its performance while interacting with the real world.
