A General Method for Construction of Bivariate Stochastic Processes Given two Marginal Processes

I. INTRODUCTION

Suppose two random variables Y, Z are given, with some "real life" interpretation.

Mainly we consider a pair Y, Z to have bio-medical, reliability or econometric interpretation. For example, Y, Z may be (stochastically dependent) levels of some chemicals (hormones, for example) in a human or an animal body. In particular, one may consider Y to describe random level of "bad" cholesterol and Z an accompanying sugar level (both measured at the same time). Apparently, the two (random) quantities are stochastically dependent on each other. Another tandem of random quantities Y, Z can be daily salt consumption Y and blood pressure Z. There are a countless number of such pairs of random quantities that have biomedical meanings. They mostly are stochastically dependent. This means that a measured value $z^{\prime }$ of Z has an influence on the probabilities $P(Y\ge y_0)$ (if, for a fixed value z, the random event $Z\ge z$ occurs and $z^{\prime }$, such that $z^{\prime }\ge z$, is its elementary realization). In particular, $y_0$ may be considered a 'critical value' of Y (for example, the critical cholesterol level $y_0$ ) so that occurrence of the random event $Y\ge y_0$ means occurrence of a disease.

Again, the probability of occurring of the so defined disease may depend on the measured value $z^{\prime }$ of the random quantity $Z$ when, for a given measurement $z^{\prime }$, we observe that $z^{\prime }\ge z$, for some fixed value $z$. Here notice that, in this framework, one may disregard an obtained by a measurement particular value $z^{\prime }$ and only notice if $z^{\prime }\ge z$ or otherwise. The more essential value, in this case, is $z$.

In the general case of two random variables Y, Z we will be interested in finding analytical formulas for all the conditional probabilities $P(Y\ge y\mid Z\ge z)$ for any arbitrary value y of Y and not only for a critical one $y_0$. To achieve this goal we will outline methodology for construction of the joint probability distribution of the random variables Y, Z under various situations.

A similar but different development of methodology of construction of bivariate probability distributions based on Cox and Aalen ideas (see, [2] and [1]) can be found in [3].

The constructions so far rely on building bivariate probability distributions of random vectors. However, the main goal of this work is to extend the obtained methods to methods where the random variables Y, Z are replaced by t-dependent stochastic processes $\{Y_t\},\{Z_t\}$. Thus, given two marginal (originally independent) univariate stochastic processes, we provide tools for constructing a class of bivariate processes $\{(Y_t,Z_t)\}$ such that the "initial" processes $\{Y_t\},\{Z_t\}$ and their probability distributions, for each t, remain as the marginals within the constructed joint. The obtained methods of the constructions turn out to be quite general and are governed by some (linear, in particular) integral equations.

As an example of the construction of processes with continuous time we consider construction of some classes of "bivariate Wiener processes" based on two given Wiener univariates. In applications, the underlying univariate Wiener processes may have econometric meanings. For example, one of the two marginal processes may describe the stock market level time evolution while the other the accompanying gross national product rate at time t. In another application, a process describes the level of inflation at a given time and the other processes the level of employment at the same time. In all cases one is interested in the stochastic dependence between the two marginal processes. That need motivates constructions of the 'joint stochastic processes'. In general, the marginal processes, as applied in econometry, are normal. However, among all the mutual stochastic dependencies at any given time epoch t we choose not to be those described by the classical bivariate normal models, since such dependencies are very well known. Our method provides a large variety of stochastic dependencies, some possibly overlooked in literature.

In another example we consider discrete time bivariate processes associated with first Pareto distributions. All the bivariate stochastic processes considerations are included in section 4.

Random Vectors, A General Approach

2.1 Suppose we are given two arbitrary random variables Y, Z with known marginal survival functions $S_1(y)=P(Y\ge y)$ and $S_2(z)=P(Z\ge z)$. Our aim is to provide a general methodology for constructions of various joint survival functions $S(y,z)=P(Y\ge y,Z\ge z)$ such that all their marginal survival functions are invariant with respect to the performed constructions, always remaining the same as the originally given $S_1(y)$ and $S_2(z)$. In general, we restrict our considerations to the cases where both the probability distributions, given by the survival functions $S_1(y)$, $S_2(z)$, possess probability densities so that the corresponding hazard rates ${\lambda }_1(y)$, ${\lambda }_2(z)$ exist too. In this case, according to the common rule, we can express the considered marginal survival functions as follows:

In the simplest case, i.e., when the random variables Y, Z are independent, their joint survival function $S^{\ast }(y,z)$ satisfies:

To impose a dependence structure for the above "initially" independent random variables Y, Z, we multiply the right hand side of formula (2) by a "dependence factor" $\phi (y,z)$. This factor we propose to call the "Aalen factor".

2.2 At this point realize that for any joint survival function $S(y,z)$ having the same fixed marginals $S_1(y)$, $S_2(z)$ there exist unique Aalen factor $\phi (y,z)=S(y,z)/S_1(y)S_2(z)$. However, in most situations we may encounter, we do not know in advance the joint survival function $S(y,z)$ but rather we have the marginals $S_1(y)$, $S_2(z)$. We aim to choose a proper Aalen factor $\phi (y,z)$ in order to construct the corresponding $S(y,z)$. To give some more intuitive meaning to the Aalen factor we express it in exponential form:

The function $a(y,z)$ we call the "joiner" of the two marginal survival functions $S_1(y)$, $S_2(z)$.

Of course, the joiner $\alpha (y,z)$ is uniquely determined by a given Aalen factor $\phi (y,z)$ since we have $\alpha (y,z)=-\log \phi (y,z)$. However, the joiner $\alpha (y,z)$ not always is represented by the integral ${\int }_0^y{\int }_0^z\psi (t;u)$ dudt. The 'integral representation' of a joiner is only valid when such a function $\psi (t;u)$, considered as a kind of an "additional hazard rate", exists. Existence of $\psi (t;u)$ together with the existence of the hazard rates ${\lambda }_1(t)$, ${\lambda }_2(u)$ involves existence of the joint probability density of the considered random vector $(Y,Z)$. Using the function $\psi (t;u)$ we define the conditional hazard rate of Y | Z = u to be equal to: ${\lambda }_1(t\mid Z=u)={\lambda }_1(t)+\psi (t;u)du$, and the conditional hazard rate of Z | Y = t as:

For the meaning of the differentials $\psi (t;u)du$ and $\psi (t;u)dt$ see section 2.4 and the associated "(microshok □ microdamage) □ microchange in hazard rate" paradigm. (These infinitesimal "microchanges" correspond to the above considered differentials.)

Since we are mostly dealing with the survival functions (not with probability densities), we will rather be interested in the conditional hazard rates of$Y\mid Z\ge z$, which involve the form:

and in the conditional hazard rates of $Z\mid Y\ge y$ using the form

The non-negativity assumption for the function $\psi (t;u)$ is not mandatory but simplifies many considerations. More general and also necessary assumption for the class of functions $\psi (t;u)$ are given by the inequalities:

for any nonnegative t, u. As we have already defined it, the left hand sides of the above inequalities represent the (conditional) hazard rates, so they must be non-negative.

Also recall that ${\lambda }_1(t)$, ${\lambda }_2(u)$ are always nonnegative.

Restricting the function $\psi (t;u)$ to nonnegative values implies considering positive stochastic dependences only between the considered random variables Y, Z.

2.3 Assuming existence of the underlying hazard rates, let us analyze the general form of the joint survival function under the investigation:

with $\psi (t;u)\ge 0$.

First realize that upon setting on the right hand side of (4) z = 0, the expression reduces to the marginal $\exp [-{\int }_0^y{\lambda }_1(t)\mathrm{d}t]=S_1(y)$. Likewise, upon the setting y = 0 in (4) turns the expression to the marginal $\exp [-{\int }_0^z{\lambda }_2(u)\mathrm{d}u]=S_2(z)$.

Second, upon dividing both sides of (4) by $S_2(z)$ one obtains the conditional survival function

Also dividing (4) by $S_1(y)$ yields the conditional survival function

2.4 Let us explain more the structure of the conditional hazard rate ${\lambda }_1$(y \mid Z \geq z), given "initially" the marginal (baseline) hazard rate ${\lambda }_1$(y). The case of ${\lambda }_2$(z \mid Y \geq y) can be understood in an analogous way. Example 1. Suppose we have two different objects O1, O2 whose "behaviors" are exhibited by the random quantities Y, Z respectively. Take, at first, object O1 as the object "of main interest", characterized by the quantity Y. Now consider the "activity" of object O2, measured by the random quantity Z. The activity Z is regarded as "stress" that the object O1 is subjected to.

(Such an "activity" may be understood as temperature, for example.) In short, the stress Z is an explanatory variable for the variable of the main interest Y. In the physical absence of the object O2 the variable Y is independent on Z, so the corresponding hazard rate for Y is simply equal to ${\lambda }_1(t)$, and the (unconditioned) survival function of Y equals

(The following 'micro-shocks □ micro-damages' mechanism that yields the stochastic dependence is described in more detail in [4].)

When we consider the case where the object O2 is physically accompanying ("connected to") object O1, its activity "produces" at each time epoch 'physical' micro-shock on O1. These micro-shocks result in corresponding micro-damages in the physical structure of O1 which in turn result in micro-changes in the hazard rate of the corresponding random quantity Y. Every such a micro-change (as related to a quantity u) at a given moment t relies on adding to the Y's hazard rate the infinitesimal quantity $\psi (t;u)du$. This change is, actually, too small to be recognized by any physical measurement, but mathematically we can express it as an infinitesimal differential $\psi (t;u)du$ as proportional to the given "intensity" $\psi (t;u)$. The microchanges cumulate as u (for Z = u) "runs" through the interval [o,z].

This phenomenon of the cumulation of the micro-changes (as corresponding to an 'elementary' random events Y = t) results in the conditional hazard rate

which, evidently, is different from the "original" (baseline) ${\lambda }_1(t)$.

The foregoing integration relates to the transfer from an elementary event Z = u to the random event [Here realize that since we apply, as the analytic tool, the survival functions, instead of the distribution functions, we eventually will 'calculate' the probability of its complement, i.e., the probability of $Z\ge z$ ].

Integrating (7) with respect to t over the interval [0,y] and applying formula (5), we obtain the conditional survival function P(Y\geq y|Z\geq z). The latter, once multiplied by the marginal distribution in the form P(Z\geq z), results again in the joint survival function of the random vector (Y,Z).

Setting ${\int }_0^z\psi (t;u)du=\phi (t,z)$ we obtain the formula

which corresponds to the Aalen additive model [1] being the modification of the famous Cox proportional hazards model [2].

Remark 1: In a more general case one may consider the second differential $\psi (t;u)dtdu$ of the joiner $\alpha$ (,) as a microchange (corresponding to both elementary events ["actions"] t and u [or u at the time epoch t]) in both hazard rates caused by mutual interaction of the "micro-shocks" described by t and u, caused by activities of the objects O1 and O2. Realize that neither t nor u needs to represent time (they may, for example, be levels of hormones in a human body).

In this framework, that interaction may be better understood if the intensity $\psi (t;u)$ is chosen to have separated variables, i.e., $\psi (t;u)={\psi }_1(t){\psi }_2(u)$ (see examples in below). Notice also that the secondary differential $\psi (t;u)$ dtdu is added to both the original hazard rates ${\lambda }_1(t)$ and ${\lambda }_2(u)$, which agrees with the idea of mutual interactions.

On Representation of Bivariate Survival Functions

3.1 The Representation

Write formula (4) in a slightly more general form:

We now depart from the Aalen-like model towards a more general paradigm. We even may drop the assumption on the existence of the hazard rates ${\lambda }_1(y)$, ${\lambda }_2(z)$, and the function $\alpha (y,z)$ need not to be expressed by the integral ${\int }_0^y{\int }_0^z\psi (t;u)$ dudt. Instead of formulas (4) and (4*) for the joint survival function of (Y,Z) we may use the following, more general, formula:

where $\alpha (y,z)$ is assumed to be a real function, defined for $y\ge 0$, $z\ge 0$.

We will require that it satisfies the conditions specified as follows. Thus the function $\alpha (y,z)$ is:

From the last condition we directly obtain the following Property.

Property 1: If the latter condition (3) is satisfied, then both the marginal probability distributions of the joint distribution given by formula (8) are preserved in the sense that they are the same as the baseline distributions $S_1(y)$, $S_2(z)$ originally given.

If the marginal and the joint probability densities of the considered random variables Y, Z exist, the concern is on non-negativity of them. While the marginal densities, say, $f(y)$, $g(z)$ are non-negative due to the common assumption, we need some special condition for $\alpha (y,z)$ to assure non-negativity of the corresponding joint density $h(y,z)$.

For the joint density $h(y,z)$ to exist, the function $\alpha (y,z)$ must have continuous partial derivatives of first order ${\alpha }_y(y,z)$, ${\alpha }_z(y,z)$ and the continuous second order mixed partial derivative ${\alpha }_{y,z}(y,z)=\psi (y,z)$.

An additional condition that must be satisfied by $\alpha (y,z)$ has the form of the following inequality:

This follows from the form of the joint density (if it exists):

which must be nonnegative. Here, $\mathrm{d}/\mathrm{dy}{\mathrm{\Lambda }}_1(\mathrm{y})={\lambda }_1(\mathrm{y})$, and $\mathrm{d}/\mathrm{dz}{\mathrm{\Lambda }}_2(\mathrm{z})={\lambda }_2(\mathrm{z})$, and ${\alpha }_{\mathrm{y},\mathrm{z}}(\mathrm{y},\mathrm{z})=\psi (\mathrm{y},\mathrm{z})$. A simpler condition than (9) for the non-negativity of $\mathrm{h}(\mathrm{y},\mathrm{z})$, which is sufficient and necessary too, for the existence of the joint survival function $(4^{\ast })$ is the following, obtained from (9):

The condition (9) or $(9^{\ast })$ together with the conditions (1), (2), and (3) are sufficient and necessary for "connecting" the two survival functions $S_1(y)$, $S_2(z)$ by a given joiner $\alpha (y,z)$ into the bivariate model $S(y,z)$.

As it was pointed out above, there may be a numerous of such models.

3.2 Particular Cases of the Bivariate Models

A. Obviously, when $\alpha (y,z)$ reduces to zero for all y, z, then model (8) describes independent random variables.

B. If the baseline hazard rates exist and are constant, we call this model "exponential".

In this case we may choose $\psi (y,z)=a=$ constant to obtain the following special case of model $(4^{\ast })$:

where $0\le a\le {\lambda }_1{\lambda }_2$.

This, apparently, represents the first bivariate exponential Gumbel probability distribution (see [5]). C. One also obtains the following 'Weibullian version' of the above bivariate Gumbel:

where ${\gamma }_1$ and ${\gamma }_2$ are positive reals.

The latter two models are special cases of representation $(4^{\ast })$.

We only need to check when condition $(9^{\ast })$ is satisfied, i.e.,

To check for that, we properly differentiate the terms of the expression $-{\lambda }_1{y}^{\gamma 1}-a{y}^{\gamma 1}{z}^{\gamma 2}-{\lambda }_2{z}^{\gamma 2}$ ( ${\lambda }_1{y}^{\gamma 1}$ over $y$ and ${\lambda }_2{z}^{\gamma 2}$ over $z$ and $a{y}^{\gamma 1}{z}^{\gamma 2}$ over $y$ and $z$ ), and set inequality (11*) in the form:

It holds, for every nonnegative y and z, whenever

Thus, if the latter condition is satisfied then model (11) is properly defined.

Remark 2: As for the above given Weibullian version (11) of the Gumbel exponential model (with $a\le {\lambda }_1{\lambda }_2$ ), this can be 'partially' generalized by the following model:

Now inequality (12) takes the form:

The necessary condition for (14) to hold for all nonnegative values y, z is that

together with

However, condition (15) makes inequality (14) not true for . The remedy for this is to shift the variables y, z by imposing the additional conditions:

1 = c = y, 1 = c = z for some real c.

Now model (13) is well defined with $1\le {\delta }_i\le {\gamma }_i$.

Remark 3: The form of model (8) and Property 1 allows us to construct numerous of bivariate probability distributions. Namely, realize that:

1. For any given fixed 'joiner function' \alpha(y,z,\text{‘any’ pair of two probability distributions [not necessarily both from the same class of the distributions]},\text{determined by }S_{1}(y),S_{2}(z)} (such that inequality (9) or (9*) is satisfied), may be "connected" by formula (8) to "become" the bivariate survival (distribution) function in which they remain the marginals.

2. For any fixed pair of probability distributions $S_1(y)$, $S_2(z)$, there is a wide class of possible joiners $\alpha (y,z)$ [determined by the conditions (1), (2), (3) from section 3.1 together with inequality (9) or $(9^{\ast })$ ] such that the two distributions can be "connected" into the bivariate model (8) in as many ways as there are possible proper functions $\alpha (y,z)$.

Finally notice that the above methods for "connecting" any two probability distributions into bivariate distributions, resembles the idea of the copula methodology [6]. It is, however, different. For more remarks on that see [3].

II. ON CONSTRUCTION OF BIVARIATE STOCHASTIC PROCESSES GIVEN ANY TWO UNIVARIATE MARGINAL PROCESSES WHICH SHARE THE SAME TIME

Suppose the (marginal) stochastic processes $\{Y_t\}$, $\{Z_t\}$ are completely defined. So, for every time epoch $t\in T$ (T is a nonempty set) two random variables $Y_t$, $Z_t$ have known in advance survival functions $S_t(y)=P(Y_t\ge y)$,

If the corresponding hazard rates ${\lambda }_1(y,t)$, ${\lambda }_2(z,t)$ exist, then, according to what was pointed out in previous sections, for any joiner function ${\alpha }_t(y,z)=\alpha (y,z,t)$ [satisfying, for each t, inequality (9) with respect to the ${\lambda }_1(y,t)$, ${\lambda }_2(z,t)$ as well as conditions (1)–(3) from section 3.1] there exist a unique joint survival function which is also a function of time t, given by: