## Discrete Time Markov And Semimarkov Reward Processes And Generalised Annuities

In Janssen and Manca (2006) the continuous time Markov reward processes (CTMRWP) were described. In Janssen and Manca (2004c) it is described how the DTMRWP can be seen as the natural stochastic generalization of the concept of discrete time annuity. In this section, after a short introduction of DTMRWP and their natural relation with annuities, we will show how it is possible to give a further generalisation to the annuity concept by means of SMRWP.

2.1 Annuities And Markov Reward Processes

The annuity concept is very simple and can be easier understood by means of Figure 2.1 for the immediate case,

Figure 2.1: constant rate annuity-immediate.

and by Figure 2.2 for the due case.

Figure 2.2: constant rate annuity-due.

Clearly the periodic payments can be variable. The simple problem to face is to compute the value at time 0 (present value) or at time n (capitalisation value) of the annuity. The present value formulas are presented at the beginning of Chapter 4. There exist similar results for capitalisation values (see Kellison (1991). Instead of assuming as usual that S is deterministic, we will now consider S as r.v. with a set of possible values:

Furthermore if we assume that the value at time k will depend only on the value at time k - 1, we can model it with a Markov process and, as a sum or amount of money is associated with each state, we can use the framework given by the Markov reward model.

We will now give relations that describe the simplest case of discounted DTMRWP, trying to show in the immediate case the recursive nature of the process. Then the most two general relations in the immediate and due cases are given.

V,(2) = (1 + r)-y, + k2Xp(1Vk = V(1) + k21 pkw , (2.3)

and in general:

+k(s, s + n)Z P&-1 (s)Z Pk (s + n) ( Yg (s, s + n) + Yig (s, s + n)).

mm mm

V(n) = V(n-1) + v(n - 1)2pkn-2)XPk¥j(n - 1) +Kn)XZPY(n). (2.6)

In the first case we have only fixed permanence rewards, in the other two cases we have variable permanence, transition rewards and interest rates. Figure 2.3 can be associated to relation (2.4). It will be possible also to describe the due case by means of a similar figure.

The figure clearly shows that MRWP can be considered a natural generalisation of the annuity concept.

Any case of MRWP can be seen as a generalisation of the annuity concept. We think that the connection between Markov reward processes and annuities is natural and that an annuity can be seen as the Markov reward process with only one state and only permanent rewards; for more details see Janssen and Manca (2004c).

In this light, in the finance environment, we can define Markov reward processes as stochastic annuities.

This first step also allows us to generalize also the payments of the annuities. In fact as we saw before, the rewards can be of permanence and of transition

 m