The Principal Component Transform of Parametrized Functions

Ilia Zabrodskii, Arcady Ponosov

Department of Science and Technology, Norwegian University of Life Sciences, Ås, Norway.

DOI: 10.4236/am.2017.84037                

Abstract

Many advanced mathematical models of biochemical, biophysical and other processes in systems biology can be described by parametrized systems of nonlinear differential equations. Due to complexity of the models, a problem of their simplification has become of great importance. In particular, rather challengeable methods of estimation of parameters in these models may require such simplifications. The paper offers a practical way of constructing approximations of nonlinearly parametrized functions by linearly parametrized ones. As the idea of such approximations goes back to Principal Component Analysis, we call the corresponding transformation Principal Component Transform. We show that this transform possesses the best individual fit property, in the sense that the corresponding approximations preserve most information (in some sense) about the original function. It is also demonstrated how one can estimate the error between the given function and its approximations. In addition, we apply the theory of tensor products of compact operators in Hilbert spaces to justify our method for the case of the products of parametrized functions. Finally, we provide several examples, which are of relevance for systems biology.

Keywords

Principal Component Analysis, Discretization of Functions, Metamodeling, Latent Parameters

1. Introduction

This study is closely related to applications in the so-called “metamodeling” of differential equations, where a “proper” model of an e.g. complex biological process is replaced by its approximation which contains “most information” about the model, but which is simpler. In particular, the true parameters of the model are replaced by “the latent parameters”, which makes the model linear with respect to the latter and hence enables the usage of the (if necessary, partial) least-squares regression. This explains why this idea proved to be efficient in parameter estimation (see e.g. [1] ). This also justifies the high numerical efficiency of metamodeling, which has been widely used in statistics [2] , chemometrics [3] , biochemstry [1] , genetics [4] [5] [6] , infrared spectroscopy [7] to simplify theoretical and computational analysis of the “true” models.

Let $x=x\left(u,\omega \right)$ be a function, where $u\in U\subset {ℝ}^N$ and $\omega \in \Omega$ , $\Omega \subset {ℝ}^M$ being a space of parameters and $k\in \mathbb{N}$ be a given number. The kth Principal Component Transform (PCT) is a specially constructed parametrized function $\text{PCT}\left(x\mathrm{,}k\right)\equiv x_k$ of the form $x_k={\displaystyle \underset{i=1}{\overset{k}{\sum }}{p}_i\left(u\right)t_i\left(\omega \right)}$ . The image $x_k$ is constructed to yield the minimum distance (in some sense) between $x$ and all possible approximations of $x$ of the form ${\displaystyle \underset{i=1}{\overset{k}{\sum }}}{z}_i\left(u\right)y_i\left(\omega \right)$ . The distance is chosen to ensure an efficient way to estimate the deviation of $x_k$ from $x$ .

Geometrically, the parametrized function $x$ may be regarded as a curve $\omega \mapsto x\left(\cdot \mathrm{,}\omega \right)$ in a separable Hilbert space. Then $x_k=\text{PCT}\left(x\mathrm{,}k\right)$ can be inter- preted as a projection of this curve onto an $k$ -dimensional subspace, which is chosen in such a way that the image $x_k$ gives a best possible individual fit to $x$ among all $k$ -dimensional subspaces. As we will see in Subsection 3.1, this necessarily leads to nonlinearity of the mapping PCT.

As we will see in Subsection 3.3, discretizing the function $x\left(u\mathrm{,}\omega \right)$ and its PCT yields matrices and the projections onto their first $k$ principal compo- nents, respectively. This explains our terminology: PCT can be regarded as a functional analog of the principal component analysis (PCA) of matrices. This terminology was suggested by Prof. E. Voit in a private talk with the second author during his seminar lecture in Oslo in 2014.

All the papers cited above concentrate on efficiency of the metamodeling approach and disregard mathematical properties of PCT and their justification, which is, for instance, quite important for understanding the limitations of the method and describing the exact conditions under which the method is applicable. In particular, the convergence properties of the sequence of metamodels to the original model has not been studied in the available literature. In our paper we try to fill this gap suggesting a rigorous mathematical approach to PCT and analysis of its basic properties. More precisely, we demonstrate how the theory of compact operators in separable Hilbert spaces can be used to provide such an analysis.

The paper is organized as follows. In Section 2 we introduce the distance in the space of parametrized functions, formulate the theorem on the best indivi- dual fit in terms of PCT of functions (Subsection 2.1) and provide some examples relevant for systems biology (Subsection 2.2). In Section 3 we study mathematical properties of PCT: nonlinearity (Subsection 3.1), continuity (Subsection 3.2) and show relations of PCT and PCA via discretization of functions (Subsections 3.3 and 3.4). In Section 4 we study PCT of products of parametrized functions which are interpreted as elements of the tensor product of two or several Hilbert spaces (Subsection 4.1). We aslo show that PCT pre- serves the tensor products and therefore the product of parametrized functions (Subsection 4.2) and give some examples (Subsection 4.3). In Appendix 5 we offer short proofs of some auxiliary results used in the paper: Allahverdiev’s theorem (Subsection 5.1) and some propositions related to tensor products of linear compact operators in Hilbert spaces (Subsection 5.2).

2. The Best Individual Fit Theorem

In this section we define the distance in the space of parametrized functions and describe how best individual fits $\text{PCT}\left(x\mathrm{,}k\right)\left(k\in \mathbb{N}\right)$ to a given function $x$ can be obtained using the theory of compact operators in Hilbert spaces. We also prove nonlinearity and continuity of PCT and give some specific examples.

2.1. The Distance in the Space of Parametrized Functions

Let $U$ be a compact subset of ${ℝ}^N$ and $\Omega$ be a compact subset of ${ℝ}^M\mathrm{.}$ We consider the separable Hilbert spaces $L^2\left(U\right)$ and $L^2\left(\Omega \right)$ with the standard scalar products $\left(\cdot \mathrm{,}\cdot \right)$ and the norms $\Vert \cdot \Vert$ .

Suppose we are given a measurable, square integrable function $x:U\times \Omega \to ℝ$ , i.e.

${\displaystyle \underset{U}{\int }}{\displaystyle \underset{\Omega }{\int }}{\left|x\left(u,\omega \right)\right|}^2\text{d}u\text{d}\omega <\infty$ (1)

The aim is to find a best possible approximation of $x$ in the class ${\mathcal{L}}_k$ of all functions of the form $x_k\left(u,\omega \right)={\displaystyle \underset{i=1}{\overset{k}{\sum }}}{z}_i\left(u\right)y_i\left(\omega \right)$ , where $z_i\in L^2\left(U\right)$ and $y_i\in L^2\left(\Omega \right)$ .

To explain better the nature of topology we use in this case let us have a look at finite dimensional Hilbert, i.e. Euclidean, spaces. Let $X=\left[x_{ij}\right]$ be an $m\times n$ -matrix, for instance, a discretized function $x\left(u\mathrm{,}\omega \right)$ where $x_{ij}=x\left(u_i,{\omega }_j\right)$ . In this case, the best approximation $X_k$ to $X$ in the class of $m\times n$ -matrices of rank not greater than $k$ is given by the first $k$ terms in the singular value decomposition of $X$ :

$X_k={\displaystyle \underset{i=1}{\overset{k}{\sum }}{t}_i{p}_i^{*}},$ (2)

where $t_i=X{p}_i$ and $p_i$ are the normalized eigenvectors of the matrix $X^{\mathrm{<mo>\; *\; </mo>}}X$ and $A^{\mathrm{<mo>\; *\; </mo>}}$ is the conjugate (transpose) of a matrix $A$ . In other words,

$\min \Vert X-Y\Vert =\Vert X-X_k\Vert ,\text{where}\text{rank}Y\le k$ (3)

The matrix norm is defined as $\Vert Z\Vert =\underset{\Vert \alpha \Vert \le 1}{\sup }\Vert Z\alpha \Vert$ , where $\Vert \alpha \Vert$ is the Euclidean norm in ${ℝ}^n$ .

Now we will look at arbitrary real separable Hilbert spaces which are denoted by $H$ and $K$ and which are equipped with the scalar products ${\left(\cdot \mathrm{,}\cdot \right)}_H$ and ${\left(\cdot \mathrm{,}\cdot \right)}_K$ and the corresponding norms ${\Vert \cdot \Vert }_H$ and ${\Vert \cdot \Vert }_K$ , respectively. Assume that

$X:H\to K$ is a linear compact operator. Its norm is again defined as $\Vert X\Vert =\underset{{\Vert \alpha \Vert }_H\le 1}{\sup }{\Vert X\alpha \Vert }_K$ .

Put

${\mathcal{L}}_k\left(H\mathrm{,}K\right)=\left\{Y\text{isalinearboundedoperatorfrom}H\text{to}K\text{suchthat}\text{dim}\left(\text{Im}Y\right)\le k\right\}$ (4)

We want to find an operator $X_k\in {\mathcal{L}}_k\left(H\mathrm{,}K\right)$ for which $\Vert X-X_k\Vert \to \min$ . The construction of $X_k$ is very close to the singular value decomposition of matrices.

Assume that $X^{\mathrm{<mo>\; *\; </mo>}}\mathrm{<mo>\; :\; </mo>}H\to K$ is the adjoint of $X$ . Then the linear compact operators $X^{\mathrm{<mo>\; *\; </mo>}}X\mathrm{<mo>\; :\; </mo>}H\to H\mathrm{,}$ $X{X}^{\mathrm{<mo>\; *\; </mo>}}\mathrm{<mo>\; :\; </mo>}K\to K$ are self-adjoint and positive-definite.

Let ${\sigma }_1^2\ge {\sigma }_2^2\ge \cdots \ge {\sigma }_i^2\ge \cdots \to 0,{\sigma }_i>0,\left(i=1,2,\cdots \right)$ be all positive eigen- values of the operator $X^{\mathrm{<mo>\; *\; </mo>}}X$ , the associated normalized eigenvectors being $p_1,p_2,p_3,\cdots \in H$ , respectively:

$X^{*}X{p}_i={\sigma }_i^2{p}_i,{\Vert p_i\Vert }_H=1,i\in \mathbb{N}$ (5)

It is well-known that $p_i$ can always be chosen to be orthogonal: $p_i\perp p_j,i\ne j$ and for any $\alpha \in H$ there is a unique set $c_i\in ℝ$ , $i\in \mathbb{N}$ and a unique $p_0\in \text{Null}\left(X^{\mathrm{<mo>\; *\; </mo>}}X\right)$ for which $\alpha =p_0+{\displaystyle \underset{i=1}{\overset{\infty }{\sum }}}{c}_i{p}_i$ and, moreover, ${\Vert \alpha \Vert }_H^2={\Vert p_0\Vert }_H^2+{\displaystyle \underset{i=1}{\overset{\infty }{\sum }}}{c}_i^2.$ Now, the operator $X$ can be represented as

$X\alpha ={\displaystyle \underset{i=1}{\overset{\infty }{\sum }}}{\left(\alpha ,p_i\right)}_H{t}_i,$ (6)

where $t_i=X{p}_i$ and the convergence is understood in the sense of the norm in the space $K$ . The truncated versions $X_k\in {\mathcal{L}}_k\left(H\mathrm{,}K\right)$ of this representation is defined by

$X_k\alpha ={\displaystyle \underset{i=1}{\overset{k}{\sum }}}{\left(\alpha ,p_i\right)}_H{t}_i$ (7)

The following result, a short proof of which is offered in Appenix 5.1, is known as Allahverdiev’s theorem, see e.g. [8, Chapter II, p. 28]:

Theorem 1. For any linear compact operator $X\mathrm{<mo>\; :\; </mo>}H\to K$

$\underset{Y\in {\mathcal{L}}_k\left(H,K\right)}{\min }\Vert X-Y\Vert =\Vert X-X_k\Vert ={\sigma }_{k+1}$ (8)

The functions in numerical calculations are usually replaced by their discreti- zations, which in the case of parametrized functions gives matrices. That is why, the distance in the space of the parametrized functions $x\left(u\mathrm{,}\omega \right)$ should be consistent with the distance in the space of matrices, so that we can get all the advantages of the finite dimensional singular value decomposition as well as Allahverdiev’s theorem. To define the distance in the space of matrices we have to interpret matrices as linear operators between two Euclidean spaces. Analo- gously, we have to interpret parametrized functions as operators between suitable Hilbert spaces, and define the distance accordingly.

Let us therefore go back to the spaces $L^2\left(U\right)$ , $L^2\left(\Omega \right)$ , where $U$ , as before, is a compact subset of ${ℝ}^N$ and $\Omega$ is a compact subset of ${ℝ}^M\mathrm{.}$ We denote the norm in both spaces as ${\Vert \cdot \Vert }_{L^2}\mathrm{.}$ Consider the integral operator

$\left(X\alpha \right)\left(\omega \right)={\displaystyle \underset{U}{\int }x\left(u,\omega \right)\alpha \left(u\right)\text{d}u}$ (9)

Under the assumptions of the square integrability of the kernel $x\left(u\mathrm{,}\omega \right)$ the operator $X$ becomes compact and linear from the space $L^2\left(U\right)$ to the space $L^2\left(\Omega \right)$ (see e.g. [9] , Chapter 7, p. 202]).

The distance between two square integrable parametrized functions $x$ and $x^{\prime }$ can be now defined in the following way:

$\text{dist}\left(x,x^{\prime }\right)=\Vert X-X^{\prime }\Vert ,$ (10)

where $X$ is defined in (9) and $\left(X^{\prime }\alpha \right)\left(\omega \right)={\displaystyle \underset{U}{\int }{x}^{\prime }\left(u,\omega \right)\alpha \left(u\right)\text{d}u}\mathrm{.}$ The norm of the linear operators acting from $L^2\left(U\right)$ to $L^2\left(\Omega \right)$ is defined in the standard way.

Remark 1. Evidently,

$\Vert X\Vert \le C{\displaystyle \underset{U}{\int }}{\displaystyle \underset{\Omega }{\int }}{\left|x\left(u\mathrm{,}\omega \right)\right|}^2\text{d}u\text{d}\omega$ (11)

for some constant $C$ . Therefore, $L^2$ -convergence of the sequence $\left\{x^{\left(n\right)}\right\}$ implies the convergence in the sense of the distance dist.

Let $X^{\mathrm{<mo>\; *\; </mo>}}\mathrm{<mo>\; :\; </mo>}{L}^2\left(\Omega \right)\to L^2\left(U\right)$ be the adjoint of $X$ , so that

$\left(X^{*}\beta \right)\left(u\right)={\displaystyle \underset{\Omega }{\int }x\left(u,\omega \right)\beta \left(\omega \right)\text{d}\omega }$ (12)

Now, the self-adjoint and positive-definite integral operators

$X^{\mathrm{<mo>\; *\; </mo>}}X\mathrm{<mo>\; :\; </mo>}{L}^2\left(U\right)\to L^2\left(U\right)\text{and}X{X}^{\mathrm{<mo>\; *\; </mo>}}\mathrm{<mo>\; :\; </mo>}{L}^2\left(\Omega \right)\to L^2\left(\Omega \right)$ (13)

can be written as follows:

$\left(X^{*}X\alpha \right)\left(u\right)={\displaystyle \underset{U}{\int }\gamma \left(u,v\right)\alpha \left(v\right)\text{d}v},\text{where}\gamma \left(u,v\right)={\displaystyle \underset{\Omega }{\int }x\left(u,\omega \right)x\left(v,\omega \right)\text{d}\omega }$ (14)

and

$\left(X{X}^{*}\beta \right)\left(\omega \right)={\displaystyle \underset{\Omega }{\int }\delta \left(\omega ,\xi \right)\beta \left(\xi \right)\text{d}\xi },\text{where}\delta \left(\omega ,\xi \right)={\displaystyle \underset{U}{\int }x\left(u,\omega \right)x\left(u,\xi \right)\text{d}u}\mathrm{,}$ (15)

respectively. Let, as before,

${\sigma }_1^2\ge {\sigma }_2^2\ge \cdots \ge {\sigma }_i^2\ge \cdots \to 0\left(i=1,2,\cdots \right)$ (16)

be all positive eigenvalues of the integral operator (14) associated with its normalized and mutually orthogonal eigenfunctions $p_i\in L^2\left(U\right)$ , i.e.

$\left(\Gamma p_i\right)\left(u\right)={\displaystyle \underset{U}{\int }\gamma \left(u,v\right)p_i\left(u\right)\text{d}u}={\sigma }_i^2{p}_i\left(u\right),{\displaystyle \underset{U}{\int }{p}_i\left(u\right)p_j\left(u\right)\text{d}u}=\{\begin{array}{ll}0\hfill & \left(i\ne j\right)\hfill \\ 1\hfill & \left(i=j\right)\hfill \end{array}$ (17)

From Theorem 1 we immediately obtain the Best Individual Fit Theorem.

Theorem 2. For a given function $x\mathrm{<mo>\; :\; </mo>}U\times \Omega \to ℝ$ satisfying (1) the best approximation of $x$ in the class ${\mathcal{L}}_k$ of all functions of the form ${\displaystyle \underset{i=1}{\overset{k}{\sum }}{z}_i\left(u\right)y_i\left(\omega \right)}$ , where $z_i\in L^2\left(U\right)$ and $y_i\in L^2\left(\Omega \right)$ , is given by

$x_k\left(u,\omega \right)={\displaystyle \underset{i=1}{\overset{k}{\sum }}{p}_i\left(u\right)t_i\left(\omega \right)},$ (18)

where $p_i$ are the normalized, mutually orthogonal eigenfunctions of the operator (14) and $t_i\left(\omega \right)=\left(X{p}_i\right)\left(\omega \right)={\displaystyle \underset{U}{\int }x\left(u,\omega \right)p_i\left(u\right)\text{d}u}$ . Moreover, $\text{dist}\left(x,x_k\right)={\sigma }_{k+1}$ for all natural $k$ .

In other words,

$\text{dist}\left(x\mathrm{,}y\right)\ge \text{dist}\left(x\mathrm{,}{x}_k\right)={\sigma }_{k+1}\text{forall}y\in {\mathcal{L}}_k$ (19)

Remark 2. The functions $t_i$ have the following properties (which we do not use in this paper):

• $t_i\perp t_j$ for all $i\ne j$ ;

• $t_i={\sigma }_i$ for all $i$ ;

• $X{X}^{*}{t}_i={\sigma }_i^2{t}_i$ for all $i$ .

Definition 1.

• The kth Principal Component Transform (PCT) of the function $x\in L^2\left(U\times \Omega \right)$ is defined as

$\text{PCT}\left(x,k\right)\left(u,\omega \right)=x_k\left(u,\omega \right)={\displaystyle \underset{i=1}{\overset{k}{\sum }}{p}_i\left(u\right)t_i\left(\omega \right)}$ (20)

• The Full Principal Component Transform of the function $x\in L^2\left(U\times \Omega \right)$ is given by

$\text{PCT}\left(x,\infty \right)\left(u,\omega \right)={\displaystyle \underset{i=1}{\overset{\infty }{\sum }}{p}_i\left(u\right)t_i\left(\omega \right)}$ (21)

We will also write $\text{PCT}\left(x\mathrm{,}\infty \right)\equiv \text{PCT}\left(x\right)\mathrm{.}$

We remark that none of these transforms is uniquely defined: even if all ${\sigma }_i$ are all different, we have always a choice between two normalized eigenfunctions $p_i$ . However, the distance between $x$ and any $x_k$ is independent of the projection we use. On the other hand, this means that the properties of PCT should be formulated with a care.

2.2. Examples of PCT

In this subsection we consider three examples which are of importance in systems biology.

Example 1. Let

$x\left(u,\omega \right)=u^{\omega }$ (22)

Assume that $u\in \left[a,b\right],a,b\in ℝ,a>0,\omega \in \left[0,1\right].$ Then, using Formulas (14) and (15), we obtain the following representations of the kernels $\gamma$ and $\delta$

$\gamma \left(u,v\right)={\displaystyle \underset{0}{\overset{1}{\int }}{u}^{\omega }{v}^{\omega }\text{d}\omega }={\displaystyle \underset{0}{\overset{1}{\int }}}{\left(uv\right)}^{\omega }\text{d}\omega =\frac{uv-1}{\ln \left(uv\right)},$ (23)

$\delta \left(\omega ,\xi \right)={\displaystyle \underset{a}{\overset{b}{\int }}{u}^{\omega }{u}^{\xi }\text{d}u}={\displaystyle \underset{a}{\overset{b}{\int }}{u}^{\omega +\xi }\text{d}u}=\frac{b^{\omega +\xi +1}-a^{\omega +\xi +1}}{\omega +\xi +1}$ (24)

Therefore the normalized eigenfunctions $p_i\left(u\right)$ can be obtained from the equation

${\displaystyle \underset{0}{\overset{1}{\int }}\left(\frac{uv-1}{\ln \left(uv\right)}\right)p_i\left(u\right)\text{d}u}={\sigma }_i^2{p}_i\left(u\right)$ (25)

The functions $t_i\left(\omega \right)={\displaystyle \underset{a}{\overset{b}{\int }}{u}^{\omega }{p}_i\left(u\right)\text{d}u}$ can be alternatively found from the equations

${\displaystyle \underset{0}{\overset{1}{\int }}}\left(\frac{b^{\omega +\xi +1}-a^{\omega +\xi +1}}{\omega +\xi +1}\right)t_i\left(\omega \right)\text{d}\omega ={\sigma }_i^2{t}_i\left(\omega \right)$ (26)

The parametrized power function $x^{\omega }$ is of crucial importance in the bioche- mical system theory, where $u$ represents the concentration of a metabolite, while $\omega$ stands for the kinetic order. In the case of several metabolites, one gets products of such power functions, which, in turn, are included into the right- hand side of the so-called “synergetic system”, see (e.g. [10] , Chapter 2, p. 51) and the references therein. The products of parametrized power functions are considered in Section 4.

Example 2. Consider the function

$x\left(u,\omega \right)=e^{-\omega \left|u\right|}$ (27)

Assume that $u\in \left[-c,c\right],c\in ℝ,c>0,\omega \in \left[a,b\right],a,b\in ℝ,a>0.$ Then, using Formulas (14) and (15), we obtain the following representations of the kernels $\gamma$ and $\delta$

$\gamma \left(u,v\right)={\displaystyle \underset{a}{\overset{b}{\int }}{e}^{-\omega \left|u\right|}{e}^{-\omega \left|v\right|}\text{d}\omega }={\displaystyle \underset{a}{\overset{b}{\int }}{e}^{-\omega \left(\left|u\right|+\left|v\right|\right)}\text{d}\omega }=\frac{1}{\left|u\right|+\left|v\right|}\left(e^{-a\left(\left|u\right|+\left|v\right|\right)}-e^{-b\left(\left|u\right|+\left|v\right|\right)}\right),$ (28)

$\delta \left(\omega ,\xi \right)={\displaystyle \underset{-c}{\overset{c}{\int }}{e}^{-\omega \left|u\right|}{e}^{-l\left|u\right|}\text{d}u}={\displaystyle \underset{-c}{\overset{c}{\int }}{e}^{-\left|u\right|\left(\omega +l\right)}\text{d}u}$ (29)

We denote for simplicity

$F\left(s,\omega ,\xi \right)={\displaystyle \underset{0}{\overset{s}{\int }}{e}^{-\left|u\right|\left(\omega +\xi \right)}\text{d}u}=\{\begin{array}{l}\frac{1}{\omega +l}{e}^{s\left(\omega +l\right)}\text{for}s<0\\ \frac{1}{\omega +l}{e}^{-s\left(\omega +\xi \right)}\text{for}s>0\end{array}$ (30)

and get

$\delta \left(\omega ,\xi \right)=F\left(c,\omega ,\xi \right)-F\left(-c,\omega ,\xi \right)$ (31)

Therefore the normalized eigenfunctions $p_i\left(u\right)$ can be obtained from the equation

${\displaystyle \underset{b}{\overset{a}{\int }}\left(\frac{1}{\left|u\right|+\left|v\right|}\right)\left(e^{-a\left(\left|u\right|+\left|v\right|\right)}-e^{-b\left(\left|u\right|+\left|v\right|\right)}\right)p_i\left(u\right)\text{d}u}={\sigma }_i^2{p}_i\left(u\right)$ (32)

The functions $t_i\left(\omega \right)={\displaystyle \underset{a}{\overset{b}{\int }}{e}^{-\omega \left|u\right|}{p}_i\left(u\right)\text{d}u}$ can be also obtained from the equations

${\displaystyle \underset{-c}{\overset{c}{\int }}}\left(F\left(c\mathrm{,}\omega \mathrm{,}\xi \right)-F\left(-c\mathrm{,}\omega \mathrm{,}\xi \right)\right)t_i\left(\omega \right)\text{d}\omega ={\sigma }_i^2{t}_i\left(\omega \right)$ (33)

The function $e^{-\omega \left|u\right|}$ is often used in the neural field models, where it serves as the simplest example of the so-called “connectivity functions” describing the interactions between neurons, see e.g. [11] and the references therein.

Example 3. Consider the Hill function

$x\left(u,\omega \right)=\frac{u^q}{u^q+{\theta }^q}$ (34)

Assume that $u\in \left[a\mathrm{,}b\right]\mathrm{,}a\mathrm{,}b\in ℝ\mathrm{,}a\mathrm{<mo>\; >\; </mo>0}$ , $q\in \left[q_0,q_m\right],q_0,q_m\in ℝ,q_0>0$ , $\theta \in \left[{\theta }_0,{\theta }_m\right],{\theta }_0,{\theta }_m\in ℝ,{\theta }_0>0.$ Putting $\omega =\left(q,\theta \right)$ and $\xi =\left(q^{\prime },{\theta }^{\prime }\right)$ we obtain

$\gamma \left(u,v\right)={\displaystyle \underset{q_0}{\overset{q_m}{\int }}}{\displaystyle \underset{{\theta }_0}{\overset{{\theta }_m}{\int }}}\frac{u^q}{u^q+{\theta }^q}\frac{v^q}{v^q+{\theta }^q}\text{d}q\text{d}\theta$ (35)

and

$\delta \left(\omega \mathrm{,}\xi \right)={\displaystyle \underset{a}{\overset{b}{\int }}}\frac{u^q}{u^q+{\theta }^q}\frac{u^{q^{\prime }}}{u^{q^{\prime }}+{{\theta }^{\prime }}^{q^{\prime }}}\text{d}u$ (36)

The Hill function plays central role in the theory of gene regulatory networks, where it stands for the gene activation function, $x$ being the gene concentra- tion and $\theta$ being the activation threshold, see e.g. [12] and the references therein.

3. Some Properties of PCT

The Principal Component Transform $\text{PCT}\left(x\mathrm{,}k\right)$ is not uniquely defined. That is why, we will use a special notation when comparing PCT of different func- tions, namely, we will write $\text{PCT}\left(x\mathrm{,}k\right)\stackrel{\dot{}}{=}\text{PCT}\left(y\mathrm{,}k\right)$ if there exist coinciding versions of PCT of $x$ and $y$ .

3.1. PCT Is Homogeneous, But Not Additive

Theorem 3.

1. $\text{PCT}\left(cx\mathrm{,}k\right)\stackrel{\dot{}}{=}c\text{PCT}\left(x\mathrm{,}k\right)$ for any $c\in ℝ$ and $k\in \mathbb{N}\mathrm{.}$

2. In general, $\text{PCT}\left(x^{\left(1\right)}+x^{\left(2\right)}\mathrm{,}k\right)$ is different from $\text{PCT}\left(x^{\left(1\right)}\mathrm{,}k\right)+\text{PCT}\left(x^{\left(2\right)}\mathrm{,}k\right)\mathrm{.}$

Proof.

1. The case $c=0$ is trivial. We assume therefore that $c\ne 0$ . Let $\left(X\alpha \right)\left(\omega \right)={\displaystyle \underset{U}{\int }x\left(u,\omega \right)\alpha \left(u\right)\text{d}u}$ and $\text{PCT}\left(x\right)\left(u,\omega \right)={\displaystyle \underset{i=1}{\overset{\infty }{\sum }}{p}_i\left(u\right)t_i\left(\omega \right)}$ , see (21). By definition, $p_i$ are normalized, mutually orthogonal eigenfunctions of the ope- rator $X^{\mathrm{<mo>\; *\; </mo>}}X$ and $t_i=X{p}_i$ . Let $X_c\left(\alpha \right)\equiv X\left(c\alpha \right)$ . Then

$X_c^{*}{X}_c{p}_i=X^{*}\left(c{p}_i\right)X\left(c{p}_i\right)=c^2{X}^{*}X{p}_i=c^2{\sigma }_i^2{p}_i,$ (37)

so that $p_i$ are the same for $X_c$ and $X$ . On the other hand, $X_c\left(p_i\right)=X\left(c{p}_i\right)=cX\left(p_i\right)=c{t}_i$ and

$\text{PCT}\left(cx,k\right)\left(u,\omega \right)={\displaystyle \underset{i=1}{\overset{k}{\sum }}{p}_i\left(u\right)c{t}_i\left(\omega \right)}\stackrel{\dot{}}{=}c\text{PCT}\left(x\mathrm{,}k\right)\left(u\mathrm{,}\omega \right)$ (38)

2. Before constructing an example illustrating nonlinearity of PCT we remark that this statement, in its more precise formulation, says that there are no

versions of $\text{PCT}\left(x^{\left(1\right)}+x^{\left(2\right)}\mathrm{,}k\right)$ , $\text{PCT}\left(x^{\left(1\right)}\mathrm{,}k\right)$ , $\text{PCT}\left(x^{\left(2\right)}\mathrm{,}k\right)$ , for which $\text{PCT}\left(x^{\left(1\right)}+x^{\left(2\right)}\mathrm{,}k\right)=\text{PCT}\left(x^{\left(1\right)}\mathrm{,}k\right)+\text{PCT}\left(x^{\left(2\right)}\mathrm{,}k\right)\mathrm{.}$

Let $U=\Omega =\left[0,1\right]$ and the functions $r_{\tau }:\left[0,1\right]\to ℝ\left(\tau =1,2\right)$ satisfy

${\displaystyle \underset{0}{\overset{1}{\int }}{r}_{\tau }^2\left(u\right)\text{d}u=1}\text{}\text{and}{\displaystyle \underset{0}{\overset{1}{\int }}{r}_1\left(u\right)r_2\left(u\right)\text{d}u=0}$ (39)

We put

$\begin{array}{l}\left(X^{\left(1\right)}\alpha \right)\left(\omega \right)=2r_1\left(\omega \right){\displaystyle \underset{0}{\overset{1}{\int }}{r}_1\left(u\right)\alpha \left(u\right)du}+r_2\left(\omega \right){\displaystyle \underset{0}{\overset{1}{\int }}{r}_2\left(u\right)\alpha \left(u\right)\text{d}u},\hfill \\ \left(X^{\left(2\right)}\alpha \right)(\omega )=r_1\left(\omega \right){\displaystyle \underset{0}{\overset{1}{\int }}\left(2r_1\left(u\right)+r_2\left(u\right)\right)\alpha \left(u\right)\text{d}u}+r_2\left(\omega \right){\displaystyle \underset{0}{\overset{1}{\int }}\left(r_1\left(u\right)+r_2\left(u\right)\right)\alpha \left(u\right)\text{d}u}.\hfill \end{array}$ (40)

To calculate PCT we observe that both operators have a 2-dimensional image in $L^2\left(\Omega \right)$ . Using the representation $\alpha \left(u\right)=c_1{r}_1\left(u\right)+c_2{r}_2\left(u\right)+\hat{\alpha }\left(u\right)$ where $\hat{\alpha }\perp r_{\tau }\left(\tau =1,2\right)$ we reduce the operators $X^{\left(1\right)}$ and $X^{\left(2\right)}$ to the matrices

$A=\left[\begin{array}{cc}2& 0\\ 0& 1\end{array}\right]\text{and}B=\left[\begin{array}{cc}2& 1\\ 1& 2\end{array}\right],\text{respectively},$

so that

$X^{\left(1\right)}\alpha =\left(r_1{r}_2\right)A{\left(c_1{c}_2\right)}^{*}\text{and}{X}^{\left(2\right)}\alpha =\left(r_1{r}_2\right)B{\left(c_1{c}_2\right)}^{*},$ (41)

where $\left(a\mathrm{,}b\right)$ and ${\left(a\mathrm{,}b\right)}^{\mathrm{<mo>\; *\; </mo>}}$ are row and column vectors, respectively.

Matrices $A$ and $B$ are symmetric. Then $A^{*}A=A^2$ and $B^{*}B=B^2$ . The first eigenpairs of $A^2$ and $B^2$ are $\mathrm{4,}{\left(10\right)}^{\mathrm{<mo>\; *\; </mo>}}$ and $\mathrm{9,}{\left(11\right)}^{\mathrm{<mo>\; *\; </mo>}}$ , respectively. There- fore the best rank 1 approximations of $A$ and $B$ are

$A_1=\left[\begin{array}{cc}2& 0\\ 0& 0\end{array}\right]\text{and}{B}_1=\left[\begin{array}{cc}1.5& 1.5\\ 1.5& 1.5\end{array}\right],\text{respectively},$

so that $\text{PCT}\left(X^{\left(1\right)}\mathrm{,1}\right)\left(u\mathrm{,}\omega \right)=2r_1\left(u\right)r_1\left(\omega \right)$ and $\text{PCT}\left(X^{\left(2\right)},1\right)\left(u,\omega \right)=1.5\left(r_1\left(u\right)+r_2\left(u\right)\right)\left(r_1\left(\omega \right)+r_2\left(\omega \right)\right),$ which both are operators with an 1-dimensional image. However, their sum

$3.5r_1\left(u\right)r_1\left(\omega \right)+1.5r_1\left(u\right)r_2\left(\omega \right)+1.5r_2\left(u\right)r_1\left(\omega \right)+1.5r_2\left(u\right)r_2\left(\omega \right)$ (42)

has a 2-dimensional image, as its representation in the basis $\left\{r_1\mathrm{,}{r}_2\right\}$ is given by the non-singular matrix $A=\left[\begin{array}{cc}3.5& 1.5\\ 1.5& 1.5\end{array}\right]$ . Therefore $\text{PCT}\left(X^{\left(1\right)}\mathrm{,1}\right)+\text{PCT}\left(X^{\left(2\right)}\mathrm{,1}\right)$ cannot coincide with any version of $\text{PCT}\left(X\mathrm{,1}\right)$ .

3.2. PCT Is Continuous

Let us consider a sequence of parametrized, square integrable functions $x^{\left(n\right)}\mathrm{<mo>\; :\; </mo>}U\times \Omega \to ℝ$ .

Theorem 4. Let $k\in \mathbb{N}$ and $\text{dist}\left(x^{\left(n\right)}\mathrm{,}x\right)\to 0\left(n\to \infty \right)$ for some parame- trized, square integrable functions $x^{\left(n\right)}\mathrm{,}x\mathrm{<mo>\; :\; </mo>}U\times \Omega \to ℝ$ . Then for any version $x_k=PCT\left(x,k\right)$ there are versions $x_k^{\left(n\right)}=\text{PCT}\left(x^{\left(n\right)}\mathrm{,}k\right)$ such that

$\text{dist}\left(x_k^{\left(n\right)}\mathrm{,}{x}_k\right)\to \mathrm{0,}n\to \infty$ (43)

Proof. Let $H=L^2\left(U\right)$ , $K=L^2\left(\Omega \right)$ . We define the compact linear integral operators $X^{\left(n\right)}\mathrm{,}X\mathrm{<mo>\; :\; </mo>}H\to K$ using the kernels $x^{\left(n\right)}$ , respectively. By the definition of the dist we immediately get that $\Vert X^{\left(n\right)}-X\Vert \to 0,n\to \infty .$

Let $p_i,i=1,\cdots ,k$ be the normalized, mutually orthogonal eigenfunctions of the operator $X^{\mathrm{<mo>\; *\; </mo>}}X$ corresponding to its first $k$ eigenvalues ${\sigma }_1^2\ge {\sigma }_2^2\ge \cdots \ge {\sigma }_k^2$ . Since $X^{\left(n\right)}$ converges to the operator $X$ in norm, we can always choose a sequence of the eigenfunctions $p_i^{\left(n\right)}$ such that

${\Vert p_i^{\left(n\right)}-p_i\Vert }_H\to 0,n\to \infty ,i=1,\cdots ,k$ (44)

In this case

$t_i^{\left(n\right)}=X^{\left(n\right)}{p}_i^{\left(n\right)}\to t_i=X{p}_i,n\to \infty ,i=1,\cdots ,k$ (45)

Therefore $\Vert X_k^{\left(n\right)}-X_k\Vert \to \mathrm{0,}n\to \infty \mathrm{,}$ which implies

$\text{dist}\left(x_k^{\left(n\right)}\mathrm{,}{x}_k\right)\to \mathrm{0,}n\to \infty$ (46)

The above theorem can be reformulated in terms of robustness of PCT.

Corollary 1. Let $k\in \mathbb{N}$ and $x:U\times \Omega \to ℝ$ be a parametrized, square inte- grable function and $k\in \mathbb{N}$ . Then given an $\epsilon >0$ there is a $\epsilon >0$ such that for every parametrized, square integrable function $x^{\prime }:U\times \Omega \to ℝ$ the follow- ing holds true:

$\text{dist}\left(x^{\prime }\mathrm{,}x\right)\mathrm{<mo>\; <\; </mo>}\delta \Rightarrow \text{dist}\left(\text{PCT}\left(x^{\prime }\mathrm{,}k\right)-\text{PCT}\left(x\mathrm{,}k\right)\right)<\epsilon$ (47)

for some suitable versions of PCT.

3.3. Discretization of Functions

In the papers [5] [6] , which are aimed at applying the metamodeling approach to gene regulatory networks, the approximations of the parametrized sigmoidal functions are performed numerically by using discretization and SVD of the resulting matrices. The continuity of PCT, proved in the previous subsection, can now be used to justify this analysis and, in particular, the results on the number of the principal components $k$ ensuring the prescribed precision.

In this subsection we suppose that all functions are continuous, which is sufficient for most applications. The general case is, however, unproblematic as well if we slightly adjust the approximation procedure.

Let $x$ be a continuous function on a compact set $D\subset {ℝ}^{N+M},D=U\times \Omega ,$ where $s=\left(u,\omega \right).$

For all $n\in \mathbb{N}\mathrm{,}$ $D$ is divided into $n$ measurable subsets $D_i^{\left(n\right)}$ :

$D={\displaystyle \underset{i=1}{\overset{n}{\cup }}{D}_i}$ (48)

We define the sequence of the functions $x_n\left(s\right)$ as follows:

$x^{\left(n\right)}\left(s\right)=x\left(s_i^{\left(n\right)}\right),s\in D_i^{\left(n\right)},$ (49)

where $s_i^{\left(n\right)}$ is an arbitrary point in $D_i^{\left(n\right)}\mathrm{.}$

Lemma 1. Let $x$ be a continuous function on $D$ . Then

$\text{dist}\left(x^{\left(n\right)}\mathrm{,}x\right)\to \mathrm{0,}n\to \infty$ (50)

provided that $\underset{1\le i\le n}{\max }\text{diam}{D}_i^{\left(n\right)}\to 0$ as $n\to \infty$ .

Proof. The function $x$ is continuous on the compact set $D$ , therefore $x\left(s\right)$ is uniformly continuous on $D$ . Then for all $\epsilon >0$ there is $\delta >0$ such that

$\left|s-s^{\prime }\right|<\delta \Rightarrow \left|x\left(s\right)-x\left(s^{\prime }\right)\right|<\epsilon$ (51)

On the other hand, there is a number $N$ for which $\underset{1\le i\le n}{\max }\text{diam}{D}_i^{\left(n\right)}<\epsilon$ as long as $n>N$ . Let $s$ be an arbitrary point from $D$ . Then for any $n$ there is $D_i^{\left(n\right)}$ such that $s\in D_i^{\left(n\right)}$ . Taking now an arbitrary $n>N$ we obtain

$\left|x^{\left(n\right)}\left(s\right)-x\left(s\right)\right|=\left|x\left(s_i^{\left(n\right)}\right)-x\left(s\right)\right|<\epsilon ,$ (52)

so that $\text{dist}\left(x^{\left(n\right)}\mathrm{,}x\right)\le C\epsilon$ , where $C^2$ is the Lebesgue measure of the set $D$ .

Hence $\text{dist}\left(x^{\left(n\right)}\mathrm{,}x\right)\to \mathrm{0,}n\to \infty \mathrm{.}$

Corollary 2. Let $k\in \mathbb{N}$ and $x\mathrm{<mo>\; :\; </mo>}U\times \Omega \to ℝ$ be a parametrized, continuous function, $\left\{x^{\left(n\right)}\right\}$ be a sequence of discrete approximations satisfied the assump- tions of Lemma 1. Then for any version $x_k=\text{PCT}\left(x\mathrm{,}k\right)$ there are versions $x_k^{\left(n\right)}=\text{PCT}\left(x^{\left(n\right)}\mathrm{,}k\right)$ such that $\text{dist}\left(x_k^{\left(n\right)}\mathrm{,}{x}_k\right)\to \mathrm{0,}n\to \infty \mathrm{.}$

Finally, we observe that if $D_i^{\left(n\right)}$ are defined as $U_j^{\left(n\right)}\times {\Omega }_l^{\left(n\right)}$ , where for any $n$ $\left\{U_j^{\left(n\right)}\right\}$ and $\left\{{\Omega }_l^{\left(n\right)}\right\}$ are measurable partitions of $U$ and $\Omega$ , respectively, and

$i=\left(j,l\right)$ , then PCT of the discrete functions $x^{\left(n\right)}$ coincide with the $k$ - truncated SVD of the matrix $\left[x^{\left(n\right)}\left(s_{\left(j\mathrm{,}l\right)}\right)\right]$ . In the next subsection we provide an example of such approximation stemming from the biochemical systems theory.

3.4. Examples of Discrete Approximations

In this subsection we study the parametrized power function $x\left(u,\omega \right)=u^{\omega }$ defined on the interval $\left[u_1,u_n\right],u_1,u_n\in ℝ,u_1>0$ with the parameter values $\omega \in \left[{\omega }_1\mathrm{,}{\omega }_m\right]\mathrm{.}$ To approximate this function we construct a matrix $\tilde{X}$ as follows: we divide $\left[u_1\mathrm{,}{u}_n\right]$ into $n-1$ parts: $u_1\mathrm{<mo>\; <\; </mo>}{u}_2\mathrm{<mo>\; <\; </mo>}\cdots \mathrm{<mo>\; <\; </mo>}{u}_n\mathrm{.}$ Similarly, we divide the interval $\left[{\omega }_0\mathrm{,}{\omega }_m\right]$ into $m-1$ parts. Every entry of the matrix $\tilde{X}$ will be given by the values $u_i^{{\omega }_j}\left(1\le i\le n,1\le j\le m\right)$ :

$\tilde{X}=\left[\begin{array}{cccc}{u}_1^{{\omega }_1}& u_2^{{\omega }_1}& \mathrm{...}& u_n^{{\omega }_1}\\ u_1^{{\omega }_2}& u_2^{{\omega }_2}& \mathrm{...}& u_n^{{\omega }_2}\\ \mathrm{...}& \mathrm{...}& \mathrm{...}& \mathrm{...}\\ u_1^{{\omega }_m}& u_2^{{\omega }_m}& \mathrm{...}& u_n^{{\omega }_m}\end{array}\right]$ (53)

The corresponding discretization of $\text{PCT}\left(x\mathrm{,}k\right)$ will be then given by the matrix

${\displaystyle \underset{i=1}{\overset{k}{\sum }}{\tilde{t}}_i{\tilde{p}}_i^{*}},{\tilde{t}}_i\in ℝ,{\tilde{p}}_i\in {ℝ}^n$ (54)

The vectors ${\tilde{p}}_i$ and ${\tilde{t}}_i$ can be obtained from the singular value decompo- sition of the matrix $\tilde{X}$

$\tilde{X}=U_{m\times m}{S}_{m\times n}{P}_{n\times n}^{*},$ (55)

where the rows of the scores matrix $T=US$ consists of the numbers ${\tilde{t}}_i$ and the columns of the loadings matrix $P$ are the vectors ${\tilde{p}}_i$ . As an example, let us consider the case $k=4$ , $\left[u_0,u_n\right]=\left[0.5,1.5\right]$ , $\left[{\omega }_0,{\omega }_m\right]=\left[-1,2\right]$ , $n=m=50$ . Then

$\tilde{X}=\left[\begin{array}{cccc}{u}_1^{{\omega }_1}& u_1^{{\omega }_1}& \mathrm{...}& u_{50}^{{\omega }_1}\\ u_1^{{\omega }_2}& u_2^{{\omega }_2}& \mathrm{...}& u_{50}^{{\omega }_2}\\ \mathrm{...}& \mathrm{...}& \mathrm{...}& \mathrm{...}\\ u_1^{{\omega }_{50}}& u_2^{{\omega }_{50}}& \mathrm{...}& u_{50}^{{\omega }_{50}}\end{array}\right],T=\left[\begin{array}{cccc}{t}_{11}& t_{12}& t_{13}& t_{14}\\ t_{21}& t_{22}& t_{23}& t_{24}\\ \mathrm{...}& \mathrm{...}& \mathrm{...}& \mathrm{...}\\ t_{m1}& t_{m2}& t_{m3}& t_{m4}\end{array}\right],P=\left[\begin{array}{cccc}{p}_{11}& p_{12}& \mathrm{...}& p_{1n}\\ p_{21}& p_{22}& \mathrm{...}& p_{2n}\\ p_{31}& p_{32}& \mathrm{...}& p_{3n}\\ p_{41}& p_{42}& \mathrm{...}& p_{4n}\end{array}\right],$ (56)

so that the Expression (54) becomes

$t_1{p}_1^{\mathrm{<mo>\; *\; </mo>}}+t_2{p}_2^{\mathrm{<mo>\; *\; </mo>}}+t_3{p}_3^{\mathrm{<mo>\; *\; </mo>}}+t_4{p}_4^{\mathrm{<mo>\; *\; </mo>}}$ (57)

Assume now that $\omega =0.5$ . This value corresponds to row $s$ in the matrix $T$ . We find a number $s$ as follows:

$s\approx m\frac{\omega -{\omega }_0}{{\omega }_m-{\omega }_0}=50\frac{0.5-\left(-1\right)}{0.5-\left(-1\right)}=25$ (58)

This yields

$\begin{array}{cccc}{t}_1=t_{s1}=-7.0579& t_2=t_{s2}=-0.0089& t_3=t_{s3}=0.2400& t_4=t_{s4}=0.0016\end{array}$

and hence

$u^{0.5}\approx -7.0579p_1^{*}\left(u\right)-0.0089p_2^{*}\left(u\right)+0.2400p_3^{*}\left(u\right)+0.0016p_4^{*}\left(u\right)$ (59)

where $p_i^{*}\left(x\right)\in {ℝ}^{50},i=1,2,3,4$ are the columns in the loadings matrix $P$ , see Figure 1.

The Figure 1 depicts the power function $u^{\omega }$ vs. its PCT with 4 components; $u\in \left[\mathrm{0.5,1.5}\right]\mathrm{,}\omega \in \left[-\mathrm{1,2}\right]$ ; the error is estimated as $\frac{{\sigma }_5}{{\sigma }_1}=0.0001$ and the Hill function $\frac{u^{1/q}}{u^{1/q}+{2.2}^{1/q}}$ vs. its PCT with 12 components; $u\in \left[\mathrm{1,3.5}\right]\mathrm{,}q\in \left[\mathrm{0.05,10}\right]\mathrm{,}\theta \in \left[\mathrm{0.01,5}\right]$ ; the error is estimated as $\frac{{\sigma }_{13}}{{\sigma }_1}=0.0013$ . The Figure 2 depicts the cumulative normal distribution function $\frac{1}{2}\left(1+\text{erf}\left(\frac{u-\mu }{\theta \sqrt{2}}\right)\right)$ vs. its PCT with 27 components and $u\in \left[-2,2\right],\mu \in \left[0.01,0.99\right],\theta \in \left[0.1,0.7\right]$ ; the error is estimated as $\frac{{\sigma }_{28}}{{\sigma }_1}=0.0019$ and the normal distribution function $x\left(u\right)=\frac{1}{\sqrt{2{\theta }^2\text{π}}}{\text{e}}^{-\frac{{\left(u-\mu \right)}^2}{2{\theta }^2}}$ vs. its PCT with 25 PCs; $u\in \left[-\mathrm{2.5,1.5}\right]\mathrm{,}\mu \in \left[-\mathrm{1.5,0.5}\right]\mathrm{,}\theta \in \left[\mathrm{0.1,1}\right]$ ; the error is estimated as $\frac{{\sigma }_{26}}{{\sigma }_1}=0.0029$ .

(a) (b)

Figure 1. (a) The power function and its PCT; (b) The Hill function and its PCT.

(a) (b)

Figure 2. (a) The cumulative normal distribution function and its PCT; (b) The normal distribution function and its PCT.

4. PCT of Products of Functions

To calculate PCT of products of parametrized functions we need to apply the theory of tensor products of Hilbert spaces and compacts operators. Appendix 5.2 includes all the necessary details we need in this section.

Below we use the following notation (where $\tau =1,2$ ):

• $U_{\tau }\subset {ℝ}^N$ , ${\Omega }_{\tau }\subset {ℝ}^M$ are compact sets;

• $U=U_1\times U_2$ , $\Omega ={\Omega }_1\times {\Omega }_2$ ;

• $H_{\tau }=L^2\left(U_{\tau }\right)$ , $K_{\tau }=L^2\left({\Omega }_{\tau }\right)$ , $H=L^2\left(U\right)$ , $K=L^2\left(\Omega \right)$ ;

• $x^{\left(\tau \right)}\left(u_{\tau }\mathrm{,}{\omega }_{\tau }\right)$ , $u_{\tau }\in U_{\tau }$ , ${\omega }_{\tau }\in {\Omega }_{\tau }$ are square integrable functions and $x\left(u,\omega \right)=x\left(u_1,{\omega }_1\right)x\left(u_2,{\omega }_2\right);$

• $\left(X^{\left(\tau \right)}{h}_{\tau }\right)\left({\omega }_{\tau }\right)={\displaystyle \underset{U_{\tau }}{\int }}{x}^{\left(\tau \right)}\left(u_{\tau },{\omega }_{\tau }\right)h_{\tau }\left(u_{\tau }\right)\text{d}{u}_{\tau }$ so that $X^{\left(\tau \right)}\mathrm{<mo>\; :\; </mo>}{H}_{\tau }\to K_{\tau }$ ;

• $\left(Xh\right)\left(\omega \right)={\displaystyle \underset{U}{\int }x\left(u,\omega \right)h\left(u\right)\text{d}u}$ so that $X\mathrm{<mo>\; :\; </mo>}H\to K$ .

4.1. Products of Parametrized Functions

Theorem 5. In the above notation:

• $H=H_1\otimes H_2$ , $K=K_1\otimes K_2$

• $X=X^{\left(1\right)}\otimes X^{\left(2\right)}$

Proof. We use the definition of the tensor product from Appendix 5.2.

Let $H_{\tau }=L^2\left(U_{\tau }\right)$ have an orthonormal basis $\left\{e_1^{\left(\tau \right)},e_2^{\left(\tau \right)},\cdots ,e_i^{\left(\tau \right)},\cdots \right\},$ so that any $h_{\tau }\in H_{\tau }$ can be represented as

$h_{\tau }={\displaystyle \underset{i=1}{\overset{\infty }{\sum }}{c}_i^{\left(\tau \right)}{e}_i^{\left(\tau \right)}}\left(\tau =1,2\right),$ (60)

where ${\displaystyle \underset{i=1}{\overset{\infty }{\sum }}}{\left|c_i^{\left(\tau \right)}\right|}^2<\infty .$

We prove now that the set $E\equiv \left\{e_i^{\left(1\right)}{e}_j^{\left(2\right)},i,j\in \mathbb{N}\right\}$ is an orthonormal basis in the space $H=L^2\left(U\right)$ . Its orthonormality follows directly from its definition. It remains therefore to check that the set of all linear combinations of the elements from $E$ is dense in $H$ . Indeed, the set of continuous functions, and hence the set $P$ of polynomials $P\left(u\right)$ , on $U$ is dense in $H$ . On the other hand, the set $\hat{P}$ of polynomials of the form $P^{\left(1\right)}\left(u^{\left(1\right)}\right)P^{\left(2\right)}\left(u^{\left(2\right)}\right)$ spans the set $P$ and, finally, the set $E$ spans the set $\hat{P}$ . Thus, $E$ spans $H$ and we have proved that any $h\in H$ can be represented as the $L^2$ -convergent series

$h={\displaystyle \underset{i,j=1}{\overset{\infty }{\sum }}}{c}_{ij}{e}_i^{\left(1\right)}{e}_j^{\left(2\right)}$ (61)

for some set $\begin{array}{c}{c}_{ij}\end{array}$ satisfying

${\displaystyle \underset{i,j=1}{\overset{\infty }{\sum }}}{c}_{ij}^2<\infty$ (62)

Defining

$\left(e_i^{\left(1\right)}\otimes e_j^{\left(2\right)}\right)\left(u\right)\equiv e_i^{\left(1\right)}\left(u_1\right)e_j^{\left(2\right)}\left(u_2\right)$ (63)

and comparing the Representation (61) with the Formula (94) proves the equality $H=H_1\otimes H_2$ . The equality $K=K_1\otimes K_2$ can be checked similarly.

Let us now prove the last formula of the theorem. First of all, we remark that the Definition (63) implies

$g_1\left({\omega }_1\right)g_2\left({\omega }_2\right)=\left(g_1\otimes g_2\right)\left(\omega \right)$ (64)

for any $g_{\tau }\in H_{\tau },\tau =1,2$ .

By the assumptions on the kernels, the operators in this equality are linear and bounded. Therefore, it is sufficient to check the equality for $h=h_1\otimes h_2$ (see Appendix 5.2).

$\begin{array}{c}\left(Xh\right)\left(\omega \right)={\displaystyle \underset{U}{\int }}x\left(u,\omega \right)h\left(u\right)\text{d}u\\ ={\displaystyle \underset{U_1\times U_2}{\int }}{x}^{\left(1\right)}\left(u_1,{\omega }_1\right)x^{\left(2\right)}\left(u_2,{\omega }_2\right)h_1\left(u_1\right)h_2\left(u_2\right)\text{d}{u}_1\text{d}{u}_2\\ ={\displaystyle \underset{U_1}{\int }}{x}^{\left(1\right)}\left(u_1,{\omega }_1\right)h_1\left(u_1\right)\text{d}{u}_1{\displaystyle \underset{U_2}{\int }}{x}^{\left(2\right)}\left(u_2,{\omega }_2\right)h_2\left(u_2\right)\text{d}{u}_2\\ =\left(X^{\left(1\right)}{h}_1\right)\left({\omega }_1\right)\left(X^{\left(2\right)}{h}_2\right)\left({\omega }_2\right)=\left(\left(X^{\left(1\right)}{h}_1\right)\otimes \left(X^{\left(2\right)}{h}_2\right)\right)\left(\omega \right)\end{array}$ (65)

due to (64). Hence $Xh=X\left(h_1\otimes h_2\right)=\left(X^{\left(1\right)}{h}_1\right)\otimes \left(X^{\left(2\right)}{h}_2\right)$ . Comparing this for- mula with the Definition (100) completes the proof of the theorem.

4.2. PCT Preserves Tensor Products

The main theoretical result of this subsection is the following theorem:

Theorem 6.

$\text{PCT}\left(X^{\left(1\right)}\otimes X^{\left(2\right)}\right)\stackrel{\dot{}}{=}\text{PCT}\left(X^{\left(1\right)}\right)\otimes \text{PCT}\left(X^{\left(2\right)}\right)$ (66)

Proof. For $\tau =1,2$ we have by definition

$\text{PCT}\left(X^{\left(\tau \right)}\right)\alpha ={\displaystyle \underset{i=1}{\overset{\infty }{\sum }}}\left(\alpha ,p_i^{\left(\tau \right)}\right)t_i^{\left(\tau \right)},$ (67)

where $p_i^{\left(\tau \right)}$ are normalized, mutually orthogonal eigenvectors of the operator ${\left(X^{\left(\tau \right)}\right)}^{\mathrm{<mo>\; *\; </mo>}}{X}^{\left(\tau \right)}$ corresponding to the eigenvalues ${\left({\sigma }^{\left(\tau \right)}\right)}^2$ and $t_i^{\left(\tau \right)}=\left(X^{\left(\tau \right)}\right)p_i^{\left(\tau \right)}$ .

Put $X=X^{\left(1\right)}\otimes X^{\left(2\right)}$ and $p_{ij}=p_i^{\left(1\right)}\otimes p_j^{\left(2\right)}$ . Using the properties of the tensor product listed in Appendix 5.2 we obtain

$\begin{array}{c}\left(X^{*}X\right)p_{ij}={\left(X^{\left(1\right)}\otimes X^{\left(2\right)}\right)}^{*}\left(X^{\left(1\right)}\otimes X^{\left(2\right)}\right)\left(p_i^{\left(1\right)}\otimes p_j^{\left(2\right)}\right)\\ =\left({\left(X^{\left(1\right)}\right)}^{*}{X}^{\left(1\right)}\right)\otimes \left({\left(X^{(2)}\right)}^{*}{X}^{\left(2\right)}\right)\left(p_i^{\left(1\right)}\otimes p_j^{\left(2\right)}\right)\\ =\left(\left({\left(X^{\left(1\right)}\right)}^{*}{X}^{\left(1\right)}\right)p_i^{\left(1\right)}\right)\otimes \left(\left({\left(X^{\left(2\right)}\right)}^{*}{X}^{\left(2\right)}\right)p_i^{\left(2\right)}\right)\\ =\left({\left({\sigma }_i^{\left(1\right)}\right)}^2{p}_i^{\left(1\right)}\right)\otimes \left({\left({\sigma }_j^{\left(2\right)}\right)}^2{p}_i^{\left(2\right)}\right)=\left({\sigma }_i^{\left(1\right)}{\sigma }_j^{\left(2\right)}\right)p_{ij},\end{array}$ (68)

where

$\begin{array}{c}\left(p_{ij},p_{lm}\right)=\left(\left(p_i^{\left(1\right)}\otimes p_j^{\left(2\right)}\right),\left(p_l^{\left(1\right)}\otimes p_m^{\left(2\right)}\right)\right)=\left(p_i^{\left(1\right)},p_l^{\left(1\right)}\right)\left(p_j^{\left(2\right)},p_m^{\left(2\right)}\right)\\ =1\text{if}i=l,j=m\text{and}=0\text{otherwise}\end{array}$ (69)

This proves that $p_{ij}$ are normalized, mutually orthogonal eigenvectors of the operator $X^{*}X$ corresponding to the eigenvalues ${\sigma }_i^{\left(1\right)}{\sigma }_j^{\left(2\right)}$ .

On the other hand,

$\begin{array}{l}X{p}_{ij}=\left(X^{\left(1\right)}\otimes X^{\left(2\right)}\right)\left(p_i^{\left(1\right)}\otimes p_j^{\left(2\right)}\right)\\ \left(X^{\left(1\right)}{p}_i^{\left(1\right)}\right)\otimes \left(X^{\left(2\right)}{p}_i^{\left(2\right)}\right)=t_i^{\left(1\right)}\otimes t_j^{\left(2\right)}\equiv t_{ij}\end{array}$ (70)

Therefore,

$\begin{array}{c}\left(\text{PCT}\left(X^{\left(1\right)}\otimes X^{\left(2\right)}\right)\right)\left({\alpha }_1\otimes {\alpha }_2\right)\stackrel{\dot{}}{=}{\displaystyle \underset{i=1}{\overset{\infty }{\sum }}}{\displaystyle \underset{j=1}{\overset{\infty }{\sum }}}\left(p_{ij},{\alpha }_1\otimes {\alpha }_2\right)t_{ij}\\ =\left({\displaystyle \underset{i=1}{\overset{\infty }{\sum }}}\left(p_i^{\left(1\right)},{\alpha }_1\right)t_i^{\left(1\right)}\right)\otimes \left({\displaystyle \underset{j=1}{\overset{\infty }{\sum }}}\left(p_i^{\left(2\right)},{\alpha }_1\right)t_i^{\left(2\right)}\right)\\ \stackrel{\dot{}}{=}\text{PCT}\left(X^{\left(1\right)}\right)\otimes \text{PCT}\left(X^{\left(2\right)}\right),\end{array}$ (71)

which proves the theorem. $\square$

Remark 3. Theorem 6 is only valid for the full PCT. The truncated versions of PCT are not necessarily valid, as the order of the singular values ${\sigma }_{ij}={\sigma }_i^{\left(1\right)}{\sigma }_j^{\left(2\right)}$ depends on the magnitude of the eigenvales ${\sigma }_i^{\left(1\right)}$ and ${\sigma }_j^{\left(2\right)}$ .

4.3. Examples of Products of Parametrized Functions

In this subsection we describe the kernels of the integral operators related to products of parametrized functions from Subsection 0. These examples are of importance in systems biology.

Example 1. Consider the following function

$x\left(u_1,u_2,{\omega }_1,{\omega }_2\right)=u_1^{{\omega }_1}{u}_2^{{\omega }_2},u_1,u_2\in U,{\omega }_1,{\omega }_2\in \Omega$ (72)

Assume that $U=\left[a,b\right],a,b\in ℝ,a>0,\Omega =\left[0,1\right].$ Then, using Formulas (14) and (15), we obtain the following representations of the kernels $\gamma$ and $\delta$

$\begin{array}{c}\gamma \left(u_1,u_2,v_1,v_2\right)={\displaystyle \underset{\Omega }{\iint }{u}_1^{{\omega }_1}{u}_2^{{\omega }_2}{v}_1^{{\omega }_1}{v}_2^{{\omega }_2}\text{d}{\omega }_1\text{d}{\omega }_2}\\ ={\displaystyle \underset{0}{\overset{1}{\int }}}{\displaystyle \underset{0}{\overset{1}{\int }}}{\left(u_1{v}_1\right)}^{{\omega }_1}{\left(u_2{v}_2\right)}^{{\omega }_2}\text{d}{\omega }_1\text{d}{\omega }_2\\ ={\displaystyle \underset{0}{\overset{1}{\int }}}{\left(u_2{v}_2\right)}^{{\omega }_2}\text{d}{\omega }_2{\displaystyle \underset{0}{\overset{1}{\int }}}{\left(u_1{v}_1\right)}^{{\omega }_1}\text{d}{\omega }_1\\ =\frac{u_1{v}_1-1}{\ln \left(u_1{v}_1\right)}\cdot \frac{u_2{v}_2-1}{\ln \left(u_2{v}_2\right)},\end{array}$

$\begin{array}{c}\delta \left({\omega }_1,{\omega }_2,{\xi }_1,{\xi }_2\right)={\displaystyle \underset{U}{\iint }{u}_1^{{\omega }_1}{u}_2^{{\omega }_2}{u}_1^{{\xi }_1}{u}_2^{{\xi }_2}\text{d}{u}_1\text{d}{u}_2}\\ ={\displaystyle \underset{U}{\iint }{u}_1^{{\omega }_1+{\xi }_1}{u}_2^{{\omega }_2+{\xi }_2}\text{d}{u}_1\text{d}{u}_2}\\ ={\displaystyle \underset{a}{\overset{b}{\int }}{u}_1^{{\omega }_1+{\xi }_1}\text{d}{u}_1}{\displaystyle \underset{a}{\overset{b}{\int }}{u}_2^{{\omega }_2+{\xi }_2}\text{d}{u}_2}\\ =\frac{b^{{\omega }_1+{\xi }_1+1}-a^{{\omega }_1+{\xi }_1+1}}{{\omega }_1+{\xi }_1+1}\cdot \frac{b^{{\omega }_1+{\xi }_1+1}-a^{{\omega }_{2}c+{\xi }_2+1}}{{\omega }_2+{\xi }_2+1}\end{array}$

Example 2. Consider the function

$x\left(u_1,u_2,{\omega }_1,{\omega }_2\right)=e^{-{\omega }_1\left|u_1\right|}\cdot e^{-{\omega }_2\left|u_2\right|},u_1,u_2\in U,{\omega }_1,{\omega }_2\in \Omega$ (73)

Assume that $U=\left[-c,c\right],c\in ℝ,c>0,\Omega =\left[a,b\right],a,b\in ℝ,a>0.$ Then, using Formulas (14) and (15), we obtain the following representations of the kernels $\gamma$ and $\delta$

$\begin{array}{c}\gamma \left(u_1,u_2,v_1,v_2\right)={\displaystyle \underset{\Omega }{\iint }{e}^{-{\omega }_1\left|u_1\right|}{e}^{-{\omega }_2\left|u_2\right|}{e}^{-{\omega }_1\left|v_1\right|}{e}^{-{\omega }_2\left|v_2\right|}\text{d}{\omega }_1\text{d}{\omega }_2}\\ ={\displaystyle \underset{\Omega }{\iint }{e}^{-{\omega }_1\left(\left|u_1\right|+|\left|v_1\right|\right)}{e}^{-{\omega }_2\left(\left|u_2\right|+\left|v_2\right|\right)}\text{d}{\omega }_1\text{d}{\omega }_2}\\ ={\displaystyle \underset{a}{\overset{b}{\int }}{e}^{-{\omega }_1\left(\left|u_1\right|+\left|v_1\right|\right)}\text{d}{\omega }_1{\displaystyle \underset{a}{\overset{b}{\int }}{e}^{-{\omega }_2\left(\left|u_2\right|+\left|v_2\right|\right)}\text{d}{\omega }_2}}\\ =\frac{1}{-\left|u\right|-\left|v\right|}\left(e^{-b\left(\left|u\right|+\left|v\right|\right)}-e^{-a\left(\left|u\right|+\left|v\right|\right)}\right),\end{array}$

$\begin{array}{c}\delta \left({\omega }_1,{\omega }_2,{\xi }_1,{\xi }_2\right)={\displaystyle \underset{U}{\iint }{e}^{-{\omega }_1\left|u_1\right|}{e}^{-{\omega }_2\left|u_2\right|}{e}^{-{\xi }_1\left|u_1\right|}{e}^{-{\xi }_2\left|u_2\right|}\text{d}{u}_1\text{d}{u}_2}\\ ={\displaystyle \underset{U}{\iint }{e}^{-\left|u_1\right|\left({\omega }_1+{\xi }_1\right)}{e}^{-\left|u_2\right|\left({\omega }_2+{\xi }_2\right)}\text{d}{u}_1\text{d}{u}_2}\\ ={\displaystyle \underset{-c}{\overset{c}{\int }}{e}^{-\left|u_1\right|\left({\omega }_1+{\xi }_1\right)}\text{d}{u}_1}{\displaystyle \underset{-c}{\overset{c}{\int }}{e}^{-\left|u_2\right|\left({\omega }_2+{\xi }_2\right)}\text{d}{u}_2}\\ =\frac{1}{\left({\omega }_1+{\xi }_1\right)\left({\omega }_2+{\xi }_2\right)}\left(e^{-u_1\left({\omega }_1+{\xi }_1\right)}-e^{u_1\left({\omega }_1+{\xi }_1\right)}\right)\cdot \left(e^{-u_2\left({\omega }_2+{\xi }_2\right)}-e^{u_2\left({\omega }_2+{\xi }_2\right)}\right)\end{array}$

Example 3. For the Hill function we obtain

$x\left(u_1,u_2,{\omega }_1,{\omega }_2\right)=\frac{u_1^{q_1}}{u_1^{q_1}+{\theta }_1^{q_1}}\frac{u_2^{q_2}}{u_2^{q_2}+{\theta }_2^{q_2}}$ (74)

Assume that

$u_i\in U,U=\left[a,b\right],a,b\in ℝ,a>0,$

${\omega }_i=\left(q_i,{\theta }_i\right),q_i\in \left[q_0,q_m\right],q_0,q_m\in ℝ,q_0>0,$

${\theta }_i\in \left[{\theta }_0,{\theta }_m\right],{\theta }_0,{\theta }_m\in ℝ,{\theta }_0>0,i=1,2.$

Putting $\Omega =\left[q_0,q_m\right]\times \left[{\theta }_0,{\theta }_m\right]$ and ${\xi }_i=\left({q^{\prime }}_i,{{\theta }^{\prime }}_i\right),i=1,2.$ Then, using Formu- las (14) and (15), we obtain the following representations of the kernels $\gamma$ and $\delta$

$\gamma \left(u_1,u_2,v_1,v_2\right)={\displaystyle \underset{\Omega }{\int }}{\displaystyle \underset{\Omega }{\int }}\frac{u_1^{q_1}}{u_1^{q_1}+{\theta }_1^{q_1}}\frac{u_2^{q_2}}{u_2^{q_2}+{\theta }_2^{q_2}}\frac{v_1^{q_1}}{v_1^{q_1}+{\theta }_1^{q_1}}\frac{v_2^{q_2}}{v_2^{q_2}+{\theta }_2^{q_2}}\text{d}{\omega }_1\text{d}{\omega }_2,$ (75)

$\delta \left({\omega }_1,{\omega }_2,{\xi }_1,{\xi }_2\right)={\displaystyle \underset{U}{\iint }}\frac{u_1^{q_1}}{u_1^{q_1}+{\theta }_1^{q_1}}\frac{u_2^{q_2}}{u_2^{q_2}+{\theta }_2^{q_2}}\frac{u_1^{{q^{\prime }}_1}}{u_1^{{q^{\prime }}_1}+{{\theta }^{\prime }}_1^{{q^{\prime }}_1}}\frac{u_2^{{q^{\prime }}_2}}{u_2^{{q^{\prime }}_2}+{{\theta }^{\prime }}_2^{{q^{\prime }}_2}}\text{d}{u}_1\text{d}{u}_2$ (76)

Remark 4. The eigenfunctions of the integral operators with the kernels that are products of parametrized functions are, according to Subsection 5.2, also products of the respective eigenfunctions of the factors.

5. Conclusions

The main results of the paper can be summarized as follows. We defined the distance in the space of parameterized functions. We defined the $k$ -th Principal Component Transform (PCT) and the Full Principal Component Transform of functions $x\in L^2\left(U\times \Omega \right)$ . The kth PCT is the best approximation of the given function, i.e. it minimizes $\text{dist}\left(\cdot \mathrm{,}\cdot \right)$ . We proved that if the sequence of functions $x^{\left(n\right)}\left(s\right)$ converge to the continuous function $x\left(s\right)$ , then the sequence of the PCT of $x^{\left(n\right)}\left(s\right)$ will converge to the PCT of $x\left(s\right)$ . Some properties of PCT were considered. These results can also serve as theoretical background for the design of some metamodels. Using the theory of the tensor product of Hilbert spaces and compact operators we calculated the PCT of products of functions. We provided several examples of the discrete approximations and products of the parametrized functions.

We will emphasize that our study is related to systems biology. In future works we aim to investigate the problem of “sloppiness” in nonlinear models [1] and create an effective parameter estimation method for the “S-systems” ( [10] , Chapter 2, p. 51).

Acknowledgements

The work of the second author has been partially supported by the Norwegian Research Council, grant 239070.

Appendix

1. Allahverdiev’s theorem

Let $K$ and $K$ be two real separable Hilbert spaces, equipped with the scalar products ${\left(\cdot \mathrm{,}\cdot \right)}_H$ and ${\left(\cdot \mathrm{,}\cdot \right)}_K$ and the corresponding norms ${\Vert \cdot \Vert }_H$ and ${\Vert \cdot \Vert }_K$ , respectively. Assume that $X\mathrm{<mo>\; :\; </mo>}H\to K$ is a linear compact operator. Its norm is

defined as $\Vert X\Vert =\underset{{\Vert \alpha \Vert }_H\le 1}{\sup }{\Vert X\alpha \Vert }_K$ .

Put

${\mathcal{L}}_k\left(H\mathrm{,}K\right)=\left\{Y\text{isalinearboundedoperatorfrom}H\text{to}K\text{suchthat}\text{dim}\left(\text{Im}Y\right)\right\}\le k\mathrm{.}$

We want to find an operator $X_k\in {\mathcal{L}}_k\left(H\mathrm{,}K\right)$ for which $\Vert X-X_k\Vert \to \text{min}$ . This construction is very close to the finite dimensional singular value decomposition.

Assume that $X^{\mathrm{<mo>\; *\; </mo>}}\mathrm{<mo>\; :\; </mo>}H\to K$ is the adjoint of $X$ . Then the linear compact operators $X^{\mathrm{<mo>\; *\; </mo>}}X\mathrm{<mo>\; :\; </mo>}H\to H\mathrm{,}$ $X{X}^{\mathrm{<mo>\; *\; </mo>}}\mathrm{<mo>\; :\; </mo>}K\to K$ are self-adjoint and positive-definite. Let ${\sigma }_1^2\ge {\sigma }_2^2\ge {\sigma }_3^2\ge \cdots \to 0$ , ${\sigma }_i>0$ be all positive eigenvalues of the operator $X^{\mathrm{<mo>\; *\; </mo>}}X$ , the associated normalized eigenvectors being $p_1\mathrm{,}{p}_2\mathrm{,}{p}_3\mathrm{,}\cdots \in H$ , respectively:

$X^{*}X{p}_i={\sigma }_i^2{p}_i,\Vert p_i\Vert =1,i\in \mathbb{N}\mathrm{.}$ (77)

It is well-known that $p_i$ can always be chosen to be orthogonal: $p_i\perp p_j\mathrm{,}i\ne j\mathrm{.}$ By the Hilbert-Schmidt theorem, for any $\alpha \in H$ there is a

unique set $c_i\in ℝ$ , $i\in \mathbb{N}$ and a unique $p_0\in \text{Null}\left(X^{\mathrm{<mo>\; *\; </mo>}}X\right)$ for which $\alpha =p_0+{\displaystyle \underset{i=1}{\overset{\infty }{\sum }}}{c}_i{p}_i$ and, moreover, ${\Vert \alpha \Vert }_H^2={\Vert p_0\Vert }_H^2+{\displaystyle \underset{i=1}{\overset{\infty }{\sum }}{c}_i^2}\mathrm{.}$ Thus, the operator $X$ can be represented as

$X\alpha ={\displaystyle \underset{i=1}{\overset{\infty }{\sum }}}{\left(\alpha ,p_i\right)}_H{t}_i={\displaystyle \underset{i=1}{\overset{\infty }{\sum }}{c}_i{t}_i},$ (78)

where $t_i=X{p}_i$ , and the convergence is understood in the sense of the norm in the space $K$ . We define the linear bounded operators $X_k\in {\mathcal{L}}_k\left(H\mathrm{,}K\right)$ by

$X_k\alpha ={\displaystyle \underset{i=1}{\overset{k}{\sum }}}{\left(\alpha ,p_i\right)}_H{t}_i={\displaystyle \underset{i=1}{\overset{k}{\sum }}{c}_i{t}_i}$ (79)

The following result is known as Allahverdiev’s theorem, see e.g. [8]:

Proposition 7. For any linear compact operator $X\mathrm{<mo>\; :\; </mo>}H\to K$

$\underset{Y\in {\mathcal{L}}_k\left(H,K\right)}{\min }\Vert X-Y\Vert =\Vert X-X_k\Vert ={\sigma }_{k+1}$ (80)

Proof. First of all, we prove that $\Vert X-X_k\Vert ={\sigma }_{k+1}$ . By definition,

${\Vert X-X_k\Vert }^2=\underset{{\Vert \alpha \Vert }_{H\le 1}}{\sup }{\Vert \left(X-X_k\right)\alpha \Vert }_K^2$ (81)

From (79) and (78) we get

$\left(X-X_k\right)\alpha =X\alpha -X_k\alpha ={\displaystyle \underset{i=1}{\overset{k}{\sum }}{c}_i{t}_i}-{\displaystyle \underset{i=1}{\overset{\infty }{\sum }}{c}_i{t}_i}={\displaystyle \underset{i=k+1}{\overset{\infty }{\sum }}}{c}_i{t}_i$ (82)

We calculate the norm of $X-X_k$ using (81), (82):

${\Vert X-X_k\Vert }^2=\underset{{\Vert \alpha \Vert }_{H\le 1}}{\sup }{\Vert {\displaystyle \underset{i=k+1}{\overset{\infty }{\sum }}}{c}_i{t}_i\Vert }_H^2=\underset{{\Vert \alpha \Vert }_{H\le 1}}{\sup }{\displaystyle \underset{i=k+1}{\overset{\infty }{\sum }}}{c}_i{\Vert t_i\Vert }_K^2=\underset{{\Vert \alpha \Vert }_{H\le 1}}{\sup }{\displaystyle \underset{i=k+1}{\overset{\infty }{\sum }}}{c}_i{\sigma }_i^2,$ (83)

because

$\begin{array}{c}{\Vert t_i\Vert }_K^2={\left(t_i,t_i\right)}_K={\left(X{p}_i,X{p}_i\right)}_K={\left(X^{*}X{p}_i,p_i\right)}_H\\ ={\left({\sigma }_i^2{p}_i,p_i\right)}_H={\sigma }_i^2{\left(p_i,p_i\right)}_H={\sigma }_i^2{\Vert p_i\Vert }_H={\sigma }_i^2\end{array}$ (84)

and

${\left(t_i,t_j\right)}_H=\left(X{p}_i,X{p}_j\right)=\left(X^{*}X{p}_i,p_j\right)=\left({\sigma }_i^2{p}_i,p_j\right)=0\text{if}i\ne j$ (85)

As $\alpha =p_0+{\displaystyle \underset{i=1}{\overset{\infty }{\sum }}{c}_i{p}_i}$ , $p_0\perp p_i$ ( $i\in \mathbb{N}$ ) and ${\Vert \alpha \Vert }_H^2={\Vert p_0\Vert }^2+{\displaystyle \underset{i=1}{\overset{\infty }{\sum }}{c}_i^2}\le 1$ , we obtain ${\displaystyle \underset{i=1}{\overset{\infty }{\sum }}{c}_i^2}\le 1$ . As ${\sigma }_{k+1}\ge {\sigma }_i$ for all $i>k+1$ ,

${\displaystyle \underset{i=k+1}{\overset{\infty }{\sum }}}{c}_i{\sigma }_i^2\to \max ={\sigma }_{k+1},$ (86)

if $c_{k+1}=1,c_{k+2}=c_{k+3}=\cdots =0$ and $p_0=0$ .

Hence,

$\Vert X-X_k\Vert ={\sigma }_{k+1}$ (87)

Secondly, we prove that

${\Vert X-Y\Vert }_H\ge {\Vert X-X_k\Vert }_H\text{for}\text{all}Y\in {\mathcal{L}}_k\left(H,K\right)$ (88)

Let $y_1\mathrm{,}\cdots ,y_k$ be a basis in $\text{Im}Y$ . Then there exist some $z_1\mathrm{,}\cdots ,z_k$ from H such that

$Y\alpha ={\displaystyle \underset{i=1}{\overset{k}{\sum }}}{\left(\alpha ,z_i\right)}_H{y}_i$ (89)

We want to prove that

$\text{span}{\left\{z_1\mathrm{,}\cdots \mathrm{,}{z}_k\right\}}^{\perp }\cap \text{span}\left\{p_1\mathrm{,}\cdots \mathrm{,}{p}_{k+1}\right\}\ne \left\{0\right\}$ (90)

If $\alpha \in \text{span}{\left\{z_1\mathrm{,}\cdots \mathrm{,}{z}_k\right\}}^{\perp }\mathrm{,}$ then $Y\alpha =0.$

If $\alpha \in \text{span}\left\{p_1\mathrm{,}\cdots \mathrm{,}{p}_{k+1}\right\}\mathrm{,}$ then $\alpha ={\alpha }_1{p}_1+\cdots +{\alpha }_{k+1}{p}_{k+1},{\alpha }_i\in ℝ,1\le i\le k+1.$

Therefore

$\text{span}{\left\{z_1,\cdots ,z_k\right\}}^{\perp }\cap \text{span}\left\{p_1,\cdots ,p_{k+1}\right\}\ne \left\{0\right\}\iff \exists z_1,\cdots ,z_k\text{suchthatthesystem}$

${\alpha }_1\left(p_1,z_i\right)+\cdots +{\alpha }_{k+1}\left(p_{k+1},z_i\right)=0,1\le i\le k\text{hasnon}-\text{trivialsolutions}\mathrm{.}$ (91)

This homogeneous system has $k+1$ unknowns and $k$ equations, so that there is $\alpha ={\displaystyle \underset{i=1}{\overset{k+1}{\sum }}{c}_i{p}_i}$ such that ${\displaystyle \underset{i=1}{\overset{k+1}{\sum }}{c}_i^2}=1$ and $Y\alpha =0$ . Therefore

${\Vert X-Y\Vert }^2\ge {\Vert \left(X-Y\right)\alpha \Vert }_K^2={\Vert {\displaystyle \underset{i=1}{\overset{k+1}{\sum }}{c}_i{t}_i}\Vert }_K^2={\displaystyle \underset{i=1}{\overset{k+1}{\sum }}{c}_i^2{\Vert t_i\Vert }_K^2}\ge {\sigma }_{k+1}^2,$ (92)

as ${\Vert t_i\Vert }_K={\sigma }_i\ge {\sigma }_{k+1}={\Vert t_{k+1}\Vert }_K$ for $i>k+1.$

$\square$

2. Tensor product of operators in Hilbert spaces

Let $H_1\mathrm{,}{H}_2$ and $K_1\mathrm{,}{K}_2$ be real separable Hilbert spaces, where

• $H_1$ has an orthonormal basis $\left\{e_1^{\left(1\right)}\mathrm{,}{e}_2^{\left(1\right)}\mathrm{,}\cdots \mathrm{,}{e}_i^{\left(1\right)}\mathrm{,}\cdots \right\}\mathrm{.}$

• $H_2$ has an orthonormal basis $\left\{e_1^{\left(2\right)}\mathrm{,}{e}_2^{\left(2\right)}\mathrm{,}\cdots \mathrm{,}{e}_j^{\left(2\right)}\mathrm{,}\cdots \right\}\mathrm{.}$

• $K_1$ has an orthonormal basis $\left\{{\hat{e}}_1^{\left(1\right)}\mathrm{,}{\hat{e}}_2^{\left(1\right)}\mathrm{,}\cdots \mathrm{,}{\hat{e}}_i^{\left(1\right)}\mathrm{,}\cdots \right\}\mathrm{.}$

• $K_2$ has an orthonormal basis $\left\{{\hat{e}}_1^{\left(2\right)}\mathrm{,}{\hat{e}}_2^{\left(2\right)}\mathrm{,}\cdots \mathrm{,}{\hat{e}}_j^{\left(2\right)}\mathrm{,}\cdots \right\}\mathrm{.}$

Let

$h_{\tau }={\displaystyle \underset{i=1}{\overset{\infty }{\sum }}{c}_i^{\left(\tau \right)}{e}_i^{\left(\tau \right)}},c_i^{\left(\tau \right)}\in ℝ,\tau =1,2$ (93)

Now, we define the tensor product $H=H_1\otimes H_2$ of the spaces $H_1$ and $H_2$ as the real separable Hilbert space, which has the basis $e_{ij}$ consisting of all ordered pairs $\left(e_i^{\left(1\right)}\mathrm{,}{e}_j^{\left(2\right)}\right)$ , and we put $e_{ij}\equiv e_i^{\left(1\right)}\otimes e_j^{\left(2\right)}\mathrm{.}$ By definition, any $h\in H$ can be uniquely represented as

$h={\displaystyle \underset{i,j=1}{\overset{\infty }{\sum }}}{c}_{ij}{e}_{ij},{\displaystyle \underset{i,j=1}{\overset{\infty }{\sum }}}{c}_{ij}^2<\infty$ (94)

Definition 2. The scalar product $\left(\cdot \mathrm{,}\cdot \right)$ in $H$ is defined as

$\left(g,h\right)={\displaystyle \underset{i,j=1}{\overset{\infty }{\sum }}}{c}_{ij}{d}_{ij},$ (95)

where $g={\displaystyle \underset{i,j=1}{\overset{\infty }{\sum }}}{c}_{ij}{e}_{ij}\in H,h={\displaystyle \underset{i,j=1}{\overset{\infty }{\sum }}}{d}_{ij}{e}_{ij}\in H$ .

Evidently, the set $e_i\otimes e_j$ is an orthonormal basis of the space $H_1\otimes H_2$ and therefore

${\Vert h\Vert }^2={\displaystyle \underset{i,j=1}{\overset{\infty }{\sum }}}{\left|\left(h,e_{ij}\right)\right|}^2={\displaystyle \underset{i,j=1}{\overset{\infty }{\sum }}}{\left|c_{ij}\right|}^2$ (96)

is the norm on $H$ . The series

${\displaystyle \underset{i,j=1}{\overset{\infty }{\sum }}{c}_{ij}{e}_{ij}},c_{ij}\in ℝ$

converges in this norm. It is also straightforward to check that

$\Vert h_1\otimes h_2\Vert ={\Vert h_1\Vert }_{H_1}{\Vert h_2\Vert }_{H_2}$ (97)

for all $h_1\in H_1$ , $h_2\in H_2$ .

Let us consider two compact linear operators

$X^{\left(1\right)}\mathrm{<mo>\; :\; </mo>}{H}_1\to K_1\mathrm{,}{X}^{\left(2\right)}\mathrm{<mo>\; :\; </mo>}{H}_2\to K_2$ (98)

For all $h_1\in H_1\mathrm{,}{h}_2\in K_2$ we have

$X^{\left(1\right)}{h}_1={\displaystyle \underset{i=1}{\overset{\infty }{\sum }}{c}_i^{\left(1\right)}{X}^{\left(1\right)}{e}_i^{\left(1\right)}},X^{\left(2\right)}{h}_2={\displaystyle \underset{i=1}{\overset{\infty }{\sum }}{c}_i^{\left(2\right)}{X}^{\left(2\right)}{e}_i^{\left(2\right)}}$ (99)

We define the tensor product $X^{\left(1\right)}\otimes X^{\left(2\right)}:H_1\otimes H_2\to K_1\otimes K_2$ of $X^{\left(1\right)}$ and $X^{\left(2\right)}$ as

$Xh={\displaystyle \underset{i,j=1}{\overset{\infty }{\sum }}}{c}_{ij}\left(X^{\left(1\right)}{e}_i^{\left(1\right)}\otimes X^{\left(2\right)}{e}_j^{\left(2\right)}\right)\mathrm{,}$ (100)

where $h\in H$ is given by (94).

Proposition 8. If $X^{\left(1\right)}\mathrm{<mo>\; :\; </mo>}{H}_1\to K_1\mathrm{,}{X}^{\left(2\right)}\mathrm{<mo>\; :\; </mo>}{H}_2\to K_2$ are linear compact ope- rators, then so is the operator $X^{\left(1\right)}\otimes X^{\left(2\right)}:H_1\otimes H_2\to K_1\otimes K_2$ .

Proof. Linearity of $X\equiv X^{\left(1\right)}\otimes X^{\left(2\right)}$ follows directly from the definition. Taking an arbitrary $h\in H$ satisfying (94) we obtain

$\begin{array}{c}{\Vert Xh\Vert }^2={\Vert {\displaystyle \underset{i,j=1}{\overset{\infty }{\sum }}}\left(c_{ij}\left(X^{\left(1\right)}{e}_i^{\left(1\right)}\otimes X^{\left(2\right)}{e}_j^{\left(2\right)}\right)\right)\Vert }^2\le {\displaystyle \underset{i,j=1}{\overset{\infty }{\sum }}}{\left|c_{ij}\right|}^2{\Vert X^{(1)}{e}_i^{\left(1\right)}\otimes X^{\left(2\right)}{e}_j^{\left(2\right)}\Vert }^2\\ \le {\displaystyle \underset{i,j=1}{\overset{\infty }{\sum }}}{\left|c_{ij}\right|}^2{\Vert X^{\left(1\right)}\Vert }^2{\Vert X^{\left(2\right)}\Vert }^2={\Vert X^{\left(1\right)}\Vert }^2{\Vert X^{\left(2\right)}\Vert }^2{\Vert h\Vert }^2\end{array}$ (101)

Therefore $X$ is bounded, and in particular,

$\Vert X\Vert \le \Vert X^{\left(1\right)}\Vert \Vert X^{\left(2\right)}\Vert$ (102)

To prove compactness we choose an arbitrary $\epsilon >0$ and linear bounded finite dimensional operators $Y_{\tau }\mathrm{<mo>\; :\; </mo>}{H}_{\tau }\to K_{\tau }$ for which $\Vert X_{\tau }-Y_{\tau }\Vert <\epsilon \left(\tau =1,2\right)$ .

Evidently,

$\begin{array}{c}{X}^{\left(1\right)}\otimes X^{\left(2\right)}-Y^{\left(1\right)}\otimes Y^{\left(2\right)}=\left(X^{\left(1\right)}-Y^{\left(1\right)}\right)\otimes \left(X^{\left(2\right)}-Y^{\left(2\right)}\right)\\ +\left(X^{\left(1\right)}-Y^{\left(1\right)}\right)\otimes Y^{\left(2\right)}+Y^{\left(1\right)}\otimes \left(X^{\left(2\right)}-Y^{\left(2\right)}\right)\end{array}$ (103)

Using (102) we obtain

$\begin{array}{c}\Vert X^{\left(1\right)}\otimes X^{\left(2\right)}\text{}-\text{}{Y}^{\left(1\right)}\otimes Y^{\left(2\right)}\Vert \le \Vert X^{\left(1\right)}\text{}-\text{}{Y}^{\left(1\right)}\Vert \Vert X^{\left(2\right)}\text{}-\text{}{Y}^{\left(2\right)}\Vert \\ +\Vert X^{\left(1\right)}-\text{}{Y}^{\left(1\right)}\Vert \Vert Y^{\left(2\right)}\Vert +\Vert Y^{\left(1\right)}\Vert \Vert X^{\left(2\right)}\text{}-\text{}{Y}^{\left(2\right)}\Vert \\ <{\epsilon }^2+\epsilon \left(\Vert X^{\left(1\right)}\Vert +\epsilon \right)+\epsilon \left(\Vert X^{\left(2\right)}\Vert +\epsilon \right)\end{array}$ (104)

Therefore, the operator $X^{\left(1\right)}\otimes X^{\left(2\right)}$ can be approximated in norm by finite dimensional operators of the form $Y^{\left(1\right)}\otimes Y^{\left(2\right)}$ with an arbitrary precision. Thus, $X^{\left(1\right)}\otimes X^{\left(2\right)}$ is compact.

$\square$

Proposition 9. For all linear compact operators $X^{\left(1\right)}\mathrm{<mo>\; :\; </mo>}{H}_1\to K_1$ and $X^{\left(2\right)}\mathrm{<mo>\; :\; </mo>}{H}_2\to K_2$ we have

${\left(X^{\left(1\right)}\otimes X^{\left(2\right)}\right)}^{*}={\left(X^{\left(1\right)}\right)}^{*}\otimes {\left(X^{\left(2\right)}\right)}^{*}$ (105)

Proof. The set of linear combinations ${\displaystyle {\sum }_i{f}_1\otimes f_2}$ is dense in $H_1\otimes H_2$ , i.e. for all $h\in H_1\otimes H_2$ there is a sequence of linear combinations of $h_1\otimes h_2$ which converges to $h$ in the norm. As the operators $X^{\left(1\right)}$ and $X^{\left(2\right)}$ are linear and bounded, it is sufficient to prove the equality in the lemma for the special case of $h=h_1\otimes h_2\in H_1\otimes H_2$ , where we by definition have the formula

$\left(X^{\left(1\right)}\otimes X^{\left(2\right)}\right)\left(h_1\otimes h_2\right)=\left(X^{\left(1\right)}{h}_1\right)\otimes \left(X^{\left(2\right)}{h}_2\right)$ (106)

Let $\alpha ={\alpha }_1\otimes {\alpha }_2,\beta ={\beta }_1\otimes {\beta }_2$ . where ${\alpha }_1\mathrm{,}{\alpha }_2\in H_1$ and ${\beta }_1\mathrm{,}{\beta }_2\in H_2$ . Then

$\begin{array}{c}\left(\left(X^{\left(1\right)}\otimes X^{\left(2\right)}\right)\alpha ,\beta \right)=\left(X^{\left(1\right)}{\alpha }_1\otimes X^{\left(2\right)}{\alpha }_2,{\beta }_1\otimes {\beta }_2\right)=\left(X^{\left(1\right)}{\alpha }_1,{\beta }_1\right)\left(X^{\left(2\right)}{\alpha }_2,{\beta }_2\right)\\ =\left({\alpha }_1,{\left(X^{\left(1\right)}\right)}^{*}{\beta }_1\right)\left({\alpha }_2,{\left(X^{\left(2\right)}\right)}^{*}{\beta }_2\right)=\left({\alpha }_1\otimes {\alpha }_2,{\left(X^{\left(1\right)}\right)}^{*}{\beta }_1\otimes {\left(X^{\left(2\right)}\right)}^{*}{\beta }_2\right)\\ =\left(\alpha ,{\left(X^{\left(1\right)}\right)}^{*}\otimes {\left(X^{\left(2\right)}\right)}^{*}\beta \right)\end{array}$ (107)

Hence ${\left(X^{\left(1\right)}\otimes X^{\left(2\right)}\right)}^{*}={\left(X^{\left(1\right)}\right)}^{*}\otimes {\left(X^{\left(2\right)}\right)}^{*}$ .

$\square$

Proposition 10. If $\left({\lambda }^{\left(\tau \right)}\mathrm{,}{q}^{\left(\tau \right)}\right)$ is the eigenpair of the operator $X^{\left(\tau \right)}\mathrm{<mo>\; ;\; </mo>}{H}_{\tau }\to K_{\tau }$ ( $\tau =1,2$ ), then $\left({\lambda }^{\left(1\right)}{\lambda }^{\left(2\right)}\mathrm{,}{q}^{\left(1\right)}\otimes q^{\left(2\right)}\right)$ is the eigenpair of the operator $X^{\left(1\right)}\otimes X^{\left(2\right)}$ .

Proof.

$\begin{array}{c}\left(X^{\left(1\right)}\otimes X^{\left(2\right)}\right)(q^{\left(1\right)}\otimes q^{\left(2\right)})=\left(X^{\left(1\right)}{q}^{\left(1\right)}\right)\otimes \left(X^{\left(2\right)}{q}^{\left(2\right)}\right)\\ =\left({\lambda }^{\left(1\right)}{q}^{\left(1\right)}\right)\otimes \left({\lambda }^{\left(2\right)}{q}^{\left(2\right)}\right)\\ ={\lambda }^{\left(1\right)}{\lambda }^{\left(2\right)}{q}^{\left(1\right)}\otimes q^{\left(2\right)}\end{array}$ (108)

$\square$

Submit or recommend next manuscript to SCIRP and we will provide best service for you:

Accepting pre-submission inquiries through Email, Facebook, LinkedIn, Twitter, etc.

A wide selection of journals (inclusive of 9 subjects, more than 200 journals)

Providing 24-hour high-quality service

User-friendly online submission system

Fair and swift peer-review system

Efficient typesetting and proofreading procedure

Display of the result of downloads and visits, as well as the number of cited articles

Maximum dissemination of your research work

Submit your manuscript at: http://papersubmission.scirp.org/

Or contact [e-mail]

Conflicts of Interest

The authors declare no conflicts of interest.

References

[1]	Tafintseva, V., Tøndel, K., Ponosov, A. and Martens, H. (2014) Global Structure of Sloppiness in a Nonlinear Model. Journal of Chemometrics, 28, 645-655. https://doi.org/10.1002/cem.2651
[2]	Tøndel, K., Gjuvsland, A., Mage, I. and Martens, H. (2010) Screening Design for Computer Experiments: Metamodelling of a Deterministic Mathematical Model of the Mammalian Circadian Clock. Journal of Chemometrics, 24, 738-747. https://doi.org/10.1002/cem.1363
[3]	Martens, H., Mage, I., Tøndel, K., Isaeva, J., Gjuvsland, A., Høy, M. and Saebø, S. (2010) Multi-Level Binary Replacement (MBR) Design for Computer Experiments in High-Dimensional Nonlinear Systems. Journal of Chemometrics, 24, 748-756. https://doi.org/10.1002/cem.1366
[4]	Martens, H., Veflingstad, S.R., Plahte, E., Martens, M., Bertrand, D. and Omholt, S.W. (2009) The Genotype-Phenotype Relationship in Multicellular Pattern-Generating Models—The Neglected Role of Pattern Descriptors. BMC Systems Biology, 3, 87. http://bmcsystbiol.biomedcentral.com/articles/10.1186/1752-0509-3-87
[5]	Isaeva, J., Saebø, S., Wyller, J.A., Liland, K.H., Faergestad, E.M., Bro, R. and Martens, H. (2010) Using GEMANOVA to Explore the Pattern Generating Properties of the Delta-Notch Model. Journal of Chemometrics, 24, 626-634. https://doi.org/10.1002/cem.1348
[6]	Isaeva, J., Saebø, S., Wyller, J.A., Wolkenhauer, O. and Martens, H. (2012) Nonlinear Modelling of Curvature by Bi-Linear Metamodeling. Chemometrics and Intelligent Laboratory Systems, 117, 2-12.
[7]	Konevskikh, T., Blümel, R., Lukacs, R., Ponosov, A. and Kohler, A. (2015) Fringes in FTIR Spectroscopy Revisited: Understanding and Modelling Fringes in Infrared Spectroscopy of Thin Films. Analyst, 140, 3969-3980. https://doi.org/10.1039/C4AN02343A
[8]	Gohberg, I.C. and Krein, M.G. (1969) Introducion to the Theory of Linear Nonselfadjoint Operators in Hilbert Space. American Mathematical Society, Providence.
[9]	Hutson, V., Pym, J.S. and Cloud, M.J. (2005) Applications of Functional Analysis and Operator Theory. Elsevier Science, Amsterdam.
[10]	Voit, E.O. (2000) Computational Analysis of Biochemical Systems. A Practical Guide for Biochemists and Molecular Biologists. Cambridge University Press, Cambridge.
[11]	Burlakov, E., Ponosov, A., Wyller, J. and Zhukovskii, E. (2015) Existence, Uniqueness and Continuous Dependence on Parameters of Solutions to Neural Field Equations. Memoirs on Differential Equations and Mathematical Physics, 65, 35-55.
[12]	Mestl, T., Plahte, E. and Omholt, S.W. (1995) A Mathematical Framework for Describing and Analysing Gene Regulatory Networks. Journal of Theoretical Biology, 176, 291-300.