An Optimal Control Approach to Customer Lifetime Value

Darkhijav Bayanjargal¹, Batsukh Davaasuren²

¹School of Applied Science and Engineering, National University of Mongolia, Ulaanbaatar, Mongolia.
²Business School, National University of Mongolia, Ulaanbaatar, Mongolia.

DOI: 10.4236/ib.2020.124011                

Abstract

It is very important for a company to determine spending for acquisition and retention of customers which affect the Customer lifetime value of the company. In this paper, we formulate the Customer lifetime value model as an optimal control problem. The obtained problem is to find an acquisition policy which maximizes the present value of all future profits generated from a customer. Here the state variable is the average margin for each customer about how much each customer contributes to the company and the control variable is the cost of acquisition. We use the maximum principle for optimality to solve the problem, estimate some parameters of the problem using statistical data for Mobile service sector of Mongolia and calculate the optimal Customer lifetime values of the sector for some values of parameters.

Keywords

Customer Lifetime Value, Acquisition Cost, Optimal Control

1. Introduction

Over the years, a number of applications of optimal control theory have been made to the field of marketing. Many of these applications deal with the problem of finding or characterizing an optimal advertising price over time (Singh & Jain, 2013). In this work, we deal with an optimal acquisition model which is a special case of the well-known Nerlove-Arrow model because the acquisition cost includes advertising and other marketing expenses. The Nerlove & Arrow (1962) model treats advertising as an investment in building up some sort of advertising capital usually called goodwill. Goodwill may be created by adding new customers or by altering the taste and preferences of customers (Sethi & Thompson, 2006).

In the late 1990’s many companies, especially the dot-coms, went on a binge to acquire customers in the belief that customer acquisition and rapid growth of the firm were critical to success. This belief was so strong that several companies focused on acquiring customers regardless of the acquisition cost. However, commonsense suggests that to acquire a customer, a company should not spend more than the lifetime value of that customer (Gupta, Lehmann, & Stuart, 2004). While some companies followed this basic economic principle, others apparently did not (Nerlove & Arrow, 1962). Therefore, it is very important for a company to determine spending for acquisition and retention of customers. In order to find an optimal acquisition cost, we formulated the Customer lifetime value model as an optimal control problem. The obtained problem is to find an acquisition policy which maximizes the present value of net profit streams which depend on the average margin.

Customer lifetime value (CLV) is generally defined as the present value of all future profits generated from a customer over his or her life of relationship with a firm. CLV is similar to the discounted cash flow approach used in finance (Nerlove & Arrow, 1962). CLV for a customer is (Gupta & Lehmann, 2003)

$\text{CLV}={\displaystyle {\sum }_{t=0}^T\frac{\left(p_t-c_t\right)r_t}{{\left(1+i\right)}^t}}-AC$ (1)

where $p_t$ is the price paid by a customer and $c_t$ is the direct cost of servicing the customer at time t; i is the discount rate or cost of capital for the firm; $r_t$ is the probability or customer repeat buying at time t; AC is the acquisition cost and T is time horizon for estimating CLV.

For simplicity, it can be written as

$\text{CLV}={\displaystyle {\sum }_{t=0}^T\frac{m_t}{{\left(1+i\right)}^t}}-AC$ (2)

where $m_t$ is the average margin or contribution for each customer in a given time period . The formulation assumes that a customer stays with a company for T periods with certainly. In general, a customer has a probability to switch or leave the company at any time period (Gupta, Hanssens et al., 2006).

The paper is organized as follows. In Section 2, we consider an optimal control problem and its optimality conditions. In Section 3, we formulate the Customer lifetime model as an optimal control problem and apply the maximum principle to the problem. In Section 4, we estimated some parameters of the problem using publicity available data for the Mobile sector of Mongolia and we summarize our results.

2. Research Methodology

Optimal control problem and its optimality condition

In this paper, optimal control theory was used to find optimal acquisition policy. Let us consider the following optimal control problem which is usually used in most management science and economics problems:

${\max }_{u\in \mathcal{U}}\left\{J={\displaystyle {\int }_0^T{\text{e}}^{-\rho t}F\left(x,u,t\right)\text{d}t}\right\}$ (3)

subject to

$\{\begin{array}{l}{x}^{·}=f\left(x,u,t\right)\\ x\left(0\right)=x_0,\\ u\in \mathcal{U}\left(t\right),t\in \left[0,T\right]\end{array}$ (4)

where $x\left(t\right)$ is a state variable whose initial value $x_0$ is given, $u\left(t\right)$ is control variable, J is objective function where $F:E^n\times E^m\times E^1\to E^1$ and $f:E^n\times E^m\times E^1\to E^1$ are assumed to be continuously differentiable and $\rho$ is a constant continuous discount rate $\rho \ge 0$.

In order to find an optimal solution of the problem, we have to change the constrained optimization problem into an unconstrained problem by using the function called current-value Hamiltonian function denoted as H,

$H\left(x,u,\lambda ,t\right)=F\left(x,u,t\right)+\lambda \left(t\right)f\left(x,u,t\right)$ (5)

where $\lambda \left(t\right)$ is known as the costate or adjoint variable which is a shadow price of the relationship between the state and control variables defined by $f(\cdot )$ (Intriligator, 2002).

The necessary conditions of the maximum principle for $u^{*}$ to be an optimal control is

$\{\begin{array}{l}{\stackrel{\dot{}}{x}}^{*}=f\left(x^{*},u^{*},t\right),x^{*}\left(0\right)=x_0,\\ \stackrel{\dot{}}{\lambda }=\rho \lambda -H_x\left[x^{*},u^{*},\lambda ,t\right],\lambda \left(T\right)=0,\\ H\left[x^{*},u^{*},\lambda \left(t\right),t\right]\ge H\left[x^{*},u,\lambda \left(t\right),t\right]\end{array}$ (6)

for all $u\in \mathcal{U}\left(t\right),t\in \left[0,T\right]$.

For the maximum principle, we must simultaneously solve two sets of differential equations with $u^{*}$ obtained from the current-value Hamiltonian maximizing conditions in (6). Since the state equation for $x^{*}$ is given with the initial value $x_0$ and the adjoint equation for $\lambda$ is specified with a condition on the terminal value $\lambda \left(T\right)$, the system of equations is a two point boundary value problem. In general, it is difficult to solve the problem. Hence we can use some numerical approaches to construct approximations to these equations.

To state the sufficient conditions for optimality, we define a current-value function $H^0:E^n\times E^m\times E^1$ as follows:

$H^0\left(x,\lambda ,t\right)=\underset{u\in \mathcal{U}\left(t\right)}{\max }H\left(x,u,\lambda ,t\right).$

Theorem. (Nerlove & Arrow, 1962)(Sufficiency conditions). Let $u^{*}$ ,and corresponding $x^{*}\left(t\right)$ and $\lambda \left(t\right)$ satisfy the necessary conditions of the maximum principle (6).If $H^0\left(x,\lambda \left(t\right),t\right)$ is concave in x for each t,then $u^{*}$ is an optimal control.

3. Optimal Control Formulation for CLV

Now we formulate the Customer lifetime value model as an optimal control problem. The obtained problem is to find an acquisition policy which maximizes the present value of all future profits generated from a customer. Here the state variable is the margin $m\left(t\right)$ which measures how much each customer contributes to the company at a time t. We assume that the average margin per customer grows at a rate proportionally to the current acquisition expenditure and depreciates over time. Let $\delta$ be the rate at which the customers tend to leave the company and the cost of acquisition is measured by the control variable (t). Therefore, the state equation is

$\stackrel{\dot{}}{m}\left(t\right)=\alpha u\left(t\right)-\delta m\left(t\right)$ (7)

with the initial condition $m\left(0\right)=m_0>0$ where $\alpha$ is the fraction of acquisition expenditure that contributes to the margin, $0\le \alpha \le 1$.

There are also control conditions $0\le u\left(t\right)\le \bar{U}$. Where $\bar{U}$ is the upper bound on the acquisition cost.

To define the objective function, we assume that the company wants to maximize its Customer lifetime value which is the present value of net profit streams discounted at fixed rate $\rho$, i.e.

${\max }_{u\ge 0}\left\{J={\displaystyle {\int }_0^T{\text{e}}^{-\rho t}\left[m(t)-u\left(t\right)\right]\text{d}t}\right\}$ (8)

subject to (7).

The problem can be rewritten as

$\{\begin{array}{l}{\max }_{u\ge 0}\left\{J={\displaystyle {\int }_0^T{\text{e}}^{-\rho t}\left[m(t)-u\left(t\right)\right]\text{d}t}\right\}\\ \stackrel{\dot{}}{m}\left(t\right)=\alpha u(t)-\delta m\left(t\right),m\left(0\right)=m_0,\\ 0\le u\left(t\right)\le \bar{U}\end{array}$ (9)

where $m_0$ is given initial value, $u=u\left(t\right)>0$ is the acquisition effort at time t and $\bar{U}$ is the upper bound on the acquisition expenditure.

This problem is an optimal control problem with control constraint for which the state variable is the average margin, the control is the cost of acquisition and the objective functional is the Customer lifetime value.

The Maximum Principle

For solving the problem, we first form the current-value Hamiltonian

$H\left(m,u,\lambda ,t\right)=m-u+\lambda \left[\alpha u-\delta m\right]$ (10)

with the current-value adjoint variable $\lambda$ satisfying the differential equation

$\stackrel{\dot{}}{\lambda }=\rho \lambda -\frac{\partial H}{\partial m}=\left(\rho +\delta \right)\lambda -1$ (11)

with the transversality conditions

$\lambda \left(T\right)=0$ (12)

Because this equation does not involve x and u, we can easily find the solution as follows:

$\lambda \left(t\right)=\frac{1}{\gamma }\left[1-{\text{e}}^{\gamma \left(t-T\right)}\right]$ (13)

where $\gamma$ is defined as the sum of the discount rate and the rate of depreciation for simplicity, that is, $\gamma =\rho +\delta$.

Since the current Hamiltonian is linear in u, that is,

$H\left(m,u,\lambda ,t\right)=\left(1-\lambda \delta \right)m\left(t\right)+\left(\lambda a-1\right)u\left(t\right)$ (14)

the form of optimal control is of bang-bang type:

$u^{*}=\{\begin{array}{l}0,\text{if}\lambda \left(t\right)<1/\alpha ,\\ \text{undefined},\text{if}\lambda \left(t\right)=1/\alpha ,\\ \bar{U},\text{if}\lambda \left(t\right)>1/\alpha ,\end{array}$ (15)

or $u^{*}=bang\left[0,\bar{U};\lambda \left(t\right)\alpha -1\right]$.

From (13), we can see that $\lambda \left(t\right)$ is monotonically decreasing function depending on $1/\gamma$, therefore the control (15) can switches once between two values 0 and $\bar{U}$ when $\lambda \left(t\right)=1/\alpha$. Let the switching time be $t^{*}\le T$ and since we can set $u^{*}\left(t^{*}\right)=0$ which is the singular control. Then the optimal acquisition policy is

$u^{*}\left(t\right)=\{\begin{array}{l}0,\text{for}0\le t\le t^{*},\\ \bar{U},\text{for}{t}^{*}<t\le T.\end{array}$ (16)

Since the control switches at $t^{*}$, $\lambda \left(t^{*}\right)$ must be $1/\alpha$. So the switching time is the solution of $\frac{1}{\gamma }\left[1-{\text{e}}^{\gamma \left(t^{*}-T\right)}\right]=1/\alpha$, i.e.

$t^{*}=T+\frac{1}{\gamma }\ln \left|1-\frac{\gamma }{\alpha }\right|$. (17)

Using $u^{*}$ given in (16), we can solve the state equation and obtain the optimal state trajectory

$m^{*}\left(t\right)=\{\begin{array}{l}{m}_0{\text{e}}^{-\delta t},0\le t\le t^{*},\\ \frac{\alpha \bar{U}}{\delta }\left(1-{\text{e}}^{\delta \left(t^{*}-t\right)}\right)+m_0{\text{e}}^{-\delta t},t^{*}<t\le T.\end{array}$ (18)

The graphs of optimal acquisition policy, $u^{*}\left(t\right)$, the optimal state, $m^{*}\left(t\right)$, and adjoint trajectories, $\lambda \left(t\right)$, are shown in Figure 1 for the different values of the parameters when $m_0=30$, $\bar{U}=15$, $\alpha =0.6$ and $T=5$ or $T=10$. Using formula (17), we can find the switching time as 2.1 or 7.0 when $T=5$ or $T=10$ respectively.

From the optimal state trajectory, we can see that if there is no effort for acquisition, the average margin would decline exponentially. However, by spending some money for the investment to acquire customers such as advertisement, special discount coupons or a new brand at the switching time, the company would increase the average margin per customer. Using the optimal values and trajectories, we can easily estimate the optimal Customer lifetime values of a

Figure 1. Optimal policy and state and adjoint trajectories. Note: The first 2 graphs shows the paths of adjoint trajectories, $\lambda \left(t\right)$, next 2 graphs shows the paths of optimal acquisition policy, $u^{*}\left(t\right)$, and last 2 graphs shows the paths of the optimal state, $m^{*}\left(t\right)$ when $\rho =0.08,\delta =0.04,T=5$ and $\rho =0.08,\delta =0.04,T=10$ respectively.

company which also gives us an opportunity to estimate the optimal upper bound on the acquisition cost by comparing the lifetime values.

4. Data and Parameter Estimation

In this section, we estimate some parameters of the model using data of Mongolian Mobile service sector and calculate the optimal Customer lifetime value of the sector. The statistical data such as the number of subscribers, the total income and investment of Mobile service from 2005 to 2017 are obtained by the annual reports of Communications Regulatory Commission of Mongolia. According to the annual reports, the number of subscribers is increasing year by years which is shown in Figure 2. Nowadays, the most of people use a mobile as a main technology source to obtain and spread information by internet today. There are 3886.2 thousand subscribers who actively using mobile phones in their everyday life by the end of 2017 in Mongolia when the number of Mongolian population was 3177.9 thousand according to the annual report.

We also show the total revenue and investment of Mobile service sector in Figure 3. From the figure, we can see that there is an increasing trend to the total income of the sector where it reached to 540 billion MNT in 2017. The cellular service sector annually accounts for 21 percent of its total revenue for the investment. Since technology is constantly being updated, investments in this sector are more important than other sectors.

Figure 2. Number of active subscribers/thous. person. Source: http://www.crc.gov.mn/.

Figure 3. Total income and investment of mobile service/bill. MNT. Source: http://www.crc.gov.mn/.

We used the annual data to estimate the average margin per customer and customer acquisition cost. In our work, we assume that the total profit generated by mobile operators is 20 percent of its total income, since there is no information about the total cost of mobile companies. Hence, we can find the average margin per customer for a Mobile service company by dividing the total profit to the number of active subscribers. Figure 4 shows the time trend for the average margin of a Mobile service operator. According to the estimation, in 2011, the average margin per customer was about 31.5 thousand MNT and there was an increase in the average margin from 2012 to 2014 where it reached 35.1 thousand MNT. Nowadays, people can use any free application such as Facebook messenger, Skype and WeChat to connect. Therefore the average margin is decreased during the last four years and reached to 27.8 thousand MNT in 2017, while the number of subscribers is increased continuously as shown in Figure 2.

Figure 4. Annual average margin per customer/thous. MNT.

In order to determine the unknown parameters of the model and upper limit on the acquisition cost, we use the annual investment of Mobile service sector. The graph of the investment per customer which is estimated by dividing the total investment of Mobile service sector to the number of active subscribers is shown in Figure 5. From the following graph, we can see that the investment per customer reached about 48.8 thousand MNT in 2016 where the average margin per customer declined to 32.1 thousand MNT in 2017.

The least square method is used to determine the unknown parameters $\alpha$ and $\delta$ of the state Equation (7). The estimation problem involves minimizing the total error,

$F\left(\alpha ,\delta \right)={\displaystyle {\sum }_{i=1}^{n-1}{\left(\Delta m_i-\alpha u_i+\delta m_i\right)}^2}$ (19)

with respect to the parameters $\alpha$ and $\delta$. Here $m_i=m\left(t_i\right)$ is the average margin and $u_i=u\left(t_i\right)$ is the investment per customer at time $t_i$, and $\Delta m_i=m_{i+1}-m_i$.

The minimum solution of the problem is

$\alpha =\frac{{\displaystyle {\sum }_{i=1}^{n-1}\Delta m_i{u}_i}{\displaystyle {\sum }_{i=1}^{n-1}{m}_i^2}-{\displaystyle {\sum }_{i=1}^{n-1}{m}_i{u}_i}{\displaystyle {\sum }_{i=1}^{n-1}\Delta m_i{m}_i}}{{\displaystyle {\sum }_{i=1}^{n-1}{u}_i^2}{\displaystyle {\sum }_{i=1}^{n-1}{m}_i^2}-{\left({\displaystyle {\sum }_{i=1}^{n-1}{m}_i{u}_i}\right)}^2}$

and

$\delta =\frac{{\displaystyle {\sum }_{i=1}^{n-1}\Delta m_i{u}_i}{\displaystyle {\sum }_{i=1}^{n-1}{m}_i{u}_i}-{\displaystyle {\sum }_{i=1}^{n-1}{u}_i^2}{\displaystyle {\sum }_{i=1}^{n-1}\Delta m_i{m}_i}}{{\displaystyle {\sum }_{i=1}^{n-1}{u}_i^2}{\displaystyle {\sum }_{i=1}^{n-1}{m}_i^2}-{\left({\displaystyle {\sum }_{i=1}^{n-1}{m}_i{u}_i}\right)}^2}$

where n is the number of year.

In our case, $n=12$ and the estimated data on the average margin and investment per customer we can estimate the parameters as $\alpha =-0.003$ and $\delta =0.001$.

In order to estimate optimal Customer lifetime value of the Mobile service sector in the next 5, 10 or 15 years, we use the interest rate as a discount rate

Figure 5. Investment per customer/thous. MNT.

Table 1. Optimal Customer lifetime value.

*Indicates the estimated values.

which is 11% according to the Central bank of Mongolia. As an initial data, the average margin per customer is 27.8 thousand MNT ( $m_0=31.6$ ) and the average investment per customer is 32.1 thousand MNT ( $\bar{U}=32.1$ ) in 2017. The table (Table 1) provides the estimation of the optimal Customer lifetime value for Mobile service sector using our model for the estimated and chosen values of parameters.

From the table, we can see that the optimal value is decreased as the rate of acquisition effort increased. We also can conclude that is possible to manage its acquisition cost in order to maximize company’s profit using the model.

5. Conclusion

In this work, we formulated the Customer lifetime value model as an optimal control problem. The obtained problem is to find an acquisition policy which maximizes the present value of all future profits generated from a customer. We use the maximum principle to solve the problem and estimated the parameters of the problem using the least square method based on data of Mobile service sector of Mongolia and estimated the optimal Customer lifetime values of the sector for some parameters.

Conflicts of Interest

The authors declare no conflicts of interest regarding the publication of this paper.

References

[1]	Gupta, S., & Lehmann, D. R. (2003). Customers as Assets. Journal of Interactive Marketing, 17, 9-24. https://doi.org/10.1002/dir.10045
[2]	Gupta, S., Hanssens, D. et al. (2006). Modeling Customer Lifetime Value. Journal of Service Research, 9, 139-155. https://doi.org/10.1177/1094670506293810
[3]	Gupta, S., Lehmann, D. R., & Stuart, J. A. (2004). Valuing Customers. Journal of Marketing Research, 41, 7-18. https://doi.org/10.1509/jmkr.41.1.7.25084
[4]	Intriligator, M. D. (2002). Mathematical Optimization and Economic Theory. Philadelphia: SIAM. https://doi.org/10.1137/1.9780898719215
[5]	Nerlove, M., & Arrow, K. J. (1962). Optimal Advertising Police under Dynamic Conditions. Economica, 29, 129-142. https://doi.org/10.2307/2551549
[6]	Sethi, S. P., & Thompson, G. L. (2006). Optimal Control Theory: Applications to Management Science and Economics (2nd ed.). Berlin: Springer.
[7]	Singh, S. S., & Jain, D. C. (2013). Measuring Customer Lifetime Value: Models and Analysis. Faculty and Research Working Paper, Fontainebleau: INSEAD, The Business School of the World.