discount

Note

Discount factors are only implemented in the learning mechanisms Guided associative learning, Expected SARSA, Q-learning, and Actor-critic.

Specifies the discount factor, written as \(\gamma\) in the equations for memory updates, that tells how important future rewards are to the current state. The discount factor is a value between 0 and 1. A reinforcement value \(u\) that occurs \(N\) steps in the future from the current state, is multiplied by \(\gamma^N\) to describe its importance to the current state. For example, consider \(\gamma=0.9\) and a reinforcement value \(u=10\) that is 3 steps ahead of the current state. The importance of this reward to the subject from where it stands is equal to \(10 \cdot 0.9^3 = 7.29\).

The value of the parameter discount is used in the updating equations described in the mechanisms.

Syntax

discount = v

where v is a scalar expression.

Description

discount = v sets the discount factor to v.

Examples

discount = 0.5

sets the discount factor to 0.5.

@variables x = 0.5
discount = x + 0.1

sets the discount factor to 0.6.