Title: EAI / 12_making_decisions

Q: What is a decision-theoretic agent?
A: - agent that combines utility theory with probability theory.
- makes rational decisions based on beliefs and desires in context of uncertainty and conflicting goals
- has a continous measure of outcome quality (in contrast to goal-based agents which have only goal/non-goal states)

Q: How is the probability of an outcome \(s'\) defined in decision theory?
A: Given evidence observations \(\mathbf{e}\), the probability of outcome \(s'\) is written as
\[P(Result(a)=s' | a,\mathbf{e})\]where \(a\) stands for the event that action \(a\) is executed.

Q: How are decision-theoretic agent's preferences expressed?
A: The agents preferences are expressed by a utility function \(U(s)\).

Q: Define the expected utility.
A: The expected utility of an action \(a\) given evidence \(\mathbf{e}\), denoted \(EU(a|\mathbf{e})\), is the average utility value of the outcomes, weighted by the probability that the outcome occurs:
\[EU(a|\mathbf{e})=\sum_{s'}P(Result(a)=s'|a,\mathbf{e})U(s')\]

Q: What is the principle of maximum expected utility (MEU)?
A: A rational agent should choose the action that maximises the agent's utility.
\[\text{action}=\underset{a}{\text{argmax}} \ EU(a|\mathbf{e})\]

Q: Name the three types of preferences agents can have.
A: \(A \succ B\): the agent prevers A over B
\(A \sim B\): the agent is indifferent between A and B
\(A \succsim B\): the agent prefers A over B or is indiffernet between them

Q: What is a lottery?
A: A lottery \(L\) is a set of possible outcomes \(S_1,...,S_n\) that occur with probabilities \(p_1,...,p_n\):
\[L=[p_1,S_1;,p_2,S_2;,...,p_n,S_n]\]Each \(S_i\) is either an atomic state or another lottery.

Q: What is the primary issue of utility theory?
A: How preferences between complex lotteries relate to preferences between the underlying states in those lotteries.

Q: Name the axioms of utility theory. (6)
A: - Orderability: \((A \succ B) \lor (B \succ A) \lor (A \sim B)\)
- Transitivity: \((A \succ B) \land (B \succ C) \Rightarrow (A \succ C)\)
- Continuity: \(A \succ B \succ C \Rightarrow \exists p\ [p, A; 1-p,C] \sim B\)
- Substitutability:
\(A \sim B \Rightarrow [p, A; 1-p, C] \sim [p, B; 1-p, C]\) and
\(A \succ B \Rightarrow [p, A; 1-p, C] \succ [p, B; 1-p, C]\)
- Monotonicity: \(A \succ B \Rightarrow (p>q \Leftrightarrow [p, A; 1-p, B] \succ [q, A; 1-q, B]\)
- Decomposability: \([p, A; 1-p, [q, B; 1-q, C]] \sim [p, A; (1-p)q, B; (1-p)(1-q), C]\)

Q: Describe the existence of utility function.
A: Given an agents preferences that satisfy the axioms of utility theory, there exists a real-valued function \(U\), such that
\[U(A)>U(B) \quad \Leftrightarrow \quad A \succ B\]\[U(A)=U(B) \quad \Leftrightarrow \quad A \sim B\]

Q: Define the expected utility of a lottery.
A: The expected utility of a lottery is the sum of the probability of each outcome times the utility of that outcome.
\[U([p_1, S_1;...;p_n,S_n)=\sum_i p_i U(S_i)\]

Q: What does a linear transformation of the utility function do to an agent's behaviour?
A: The agent's behaviour does not change.

Q: In which environments is the behaviour of an agent unchanged by applying any monotonic transformation?
A: In deterministic environments, where there are states and not lotteries, the behaviour of an agent is unchanged by an application of any monotonic transformation.

Q: What is preference elicitation?
A: Preference elicitation is how to work out an agents utility function by presenting choices to an agent and use observed preferences to pin down the underlying uitlity function.

Q: How can a utility scale be established?
A: - Fix the utilites of any two particular outcomes
- Typically, we fix the utility of a "best possible prize" \(S_b\), at \(U(S_B)=u_\top\) and a "worst possible catastrophe" \(S_w\) at \(U(S_w)=u_\bot\)
- Normalized utilities use a scale with \(u_\bot = 1\) and \(u_\top=1\)

Q: What scale can the utility of money be compared to?
A: The utility of many is almost exactly proportional to the logarithm of the amount.

Explain the Ellsberg Paradox.

Q: Explain a decision network.
A: Decision networks contain information about
- an agent's current state
- it's possible actions
- the state that will result from the agent's action
- the utility of that state
The network contains
- chance nodes (oval): represent random variables
- decision nodes (rectangles): represent points where a decision maker has a choice of actions
- utility nodes (diamonds): agent's utility function

Q: Explain the algorithm for evaluating decision networks.
A: 1. Set the evidence variables for the current state
2. For each possible value of the decision node:
  1. Set the decision node to that value
  2. Calculate the posterior probabilities for the parent nodes of the utility node
  3. Calculate the resulting utility for the action
3. Return the action with the highest utility

Q: How is the value of a given piece of information defined?
A: The value of a given piece of information is defined to be the difference in expected value between the best actions before and after an information is obtained.

Q: Define the properties of the value of perfect information. (3)
A: - The expected value of information is nonnegative: \(VPI_e(E_J) \geq 0\), for all \(\mathbf{e}\) and all \(E_j\)
- VPI is nonadditive: \(VPI_e(E_j,E_k) \neq VPI_e(E_j) + VPI_e(E_k)\)
- VPI is order independent: \(VPI_e(E_j, E_k) = VPI_e(E_k, E_j)\)