Weiss-Weinstein bound for change-point estimation

—We compute the Weiss-Weinstein bound in the context of change-point estimation in a multivariate time series whatever the considered distribution of the data as well the prior. Closed-form expressions are then given in the case of Gaussian observations with change of mean and variance and in the case of parameter change in a Poisson distribution. The proposed bound is shown to be tighter than the previous bounds which were originally derived in the deterministic context and provides a better approximation of the maximum a posteriori estimator global mean square error.


I. INTRODUCTION
Change-point detection and estimation in a multivariate time series is an important topic due to the wide range of underlying applications.One can cite speech processing, fault detection, medical imaging or radar [1].Particularly, in radar signal processing, such problem arise in SAR image edge detection [2], clutter map segmentation [3], meteorological calibration [4], detection of target signals in clutter [5], etc.
A plethora of algorithms have been proposed in the literature to address this problem, but their statistical performances are generally conducted only by way of Monte-Carlo simulations.Indeed, few results are available concerning the consistency, the bias or the mean square error (MSE) of change point estimators and are only given in an asymptotic context [6]- [8].
Concerning the MSE, the problem comes from the fact that the classical Cramér-Rao bound cannot be computed due to the discrete nature of the parameters.Some authors have tried to overcome this difficulty by using approximations (see e.g.[9], [10]).A more natural way to compute a lower bound on the MSE in this context is to use other bounds than the Cramér-Rao bound which require less regularity assumptions (mainly the existence of the derivative of the likelihood function).To the best of our knowledge, this has been done first in [11] where the Chapman-Robbins bound has been studied.Then, this result has been extended in [12] to the case of multiple change point estimation by using the McAulay-Seidman bound.In both the aforementioned papers, it has been shown that the obtained bounds was quite optimistic with respect to the maximum likelihood estimator empirical MSE.
In this paper, we propose to analyze the problem in the Bayesian context.As well as in the deterministic context, the Bayesian Cramér-Rao bound cannot be computed due to the discrete nature of the parameters and the Bayesian Barankin bound (a Bayesian version of the Chapman-Robbins and McAuly-Seidman bounds) neither due to some regularity assumptions about the used parameter a priori support.We propose to study the Weiss-Weinstein bound which is known to be one of the tightest bound among the Bayesian bounds.We give a general form of this bound whatever the considered distribution of the data (and the prior) and we then apply it in the case of Gaussian observations with change of mean and variance and in the case of parameter change in a Poisson distribution.Finally, throughout simulation results, the proposed bound is shown to be tighter than the previous bounds which were originally derived in the deterministic context.

II. PROBLEM SETUP AND BACKGROUND
We consider the general case of N independent observation vectors Y = [y 1 , y 2 , . . ., y N ] ∈ Ω ⊂ R M ×N , which can be obtained, for example, from a multiple sensor system.In the context of a single change-point estimation, these observations are modeled as follows [11], [12]: where M is the size of the sample vector (e.g. the number of sensors), and p j is a probability density function with parameters η j ∈ R L , j ∈ {1, 2}.Note that if M = 1, the problem is reduced to the estimation of a single change point in a time series context.We assume that the probability density functions p j , j ∈ {1, 2}, belong to a common distribution.The unknown parameter of interest is the change-point location denoted θ ∈ Θ = Z.The observations before and after the change point are assumed to be stationary.The change point θ is assumed to be a discrete random variable over the parameter space Θ with a known prior distribution Pr (θ = i), i ∈ Z.
Besides, we recall that the Weiss-Weinstein bound (WWB) for a parameter θ, denoted W W B, satisfies the following inequality w.r.t. the mean square error of any Bayesian estimator θ (Y) [13] in which and and where i + h is the so-called test-point such that i + h ∈ Θ, h = 0 and 0 < s < 1.We notice that we have to optimize the bound over the test point and over parameter s.Nevertheless, it has been shown that choosing s = 1/2 is generally the best choice in terms of tightness (see e.g.[13], [14]).Consequently, we will only optimize the bound over h.We first simplify the aforementioned expression in the context of a discrete parameter, then we derive a closed-form expression for change point estimation in the following.
The expectation of the numerator of (4) can then be simplified as follows: in which denotes the Bhattacharyya distance between p (Y | θ = i) and p (Y | θ = j) [15].
Concerning the denominator of ( 4), one has: The three terms of the right hand side of ( 7) can be calculated as follows: given the constraint In the same way, the second term is given by: Finally, the third term can be written: Consequently, from ( 4), ( 7)- (10), one obtains the WWB expression as: where Given the above expression of the WWB, its final closedform expression will be obtained by deriving B (i + h, i) and B (i + h, i − h) which depend on the probability density functions p 1 and p 2 .Note that B (i + h, i − h) can be simplified by using a change of variable i − h = j, keeping in mind that j ∈ Θ: meaning that finding a closed-form expression of B (i + h, i) is enough since we can easily deduce B (i + h, i − h) by replacing h with 2h.

III. PROPOSED BOUND FOR CHANGE POINT ESTIMATION
First, let us calculate the closed-form expression of B (i + h, i).By using the observation model of (1) and the fact that the observations are assumed to be independent, one can write We investigate both cases h > 0 and h < 0. For h > 0: where Ω ⊂ R M denotes the observation space w.r.t.y i , i ∈ {1, . . ., N } (i.e.Ω N ∆ = Ω × . . .× Ω = Ω).In the same way, for h < 0: Consequenlty, for any h we obtain: and Since B (i + h, i) and B (i + h, i − h) given above do not depend on i, the general WWB expression is now given by: The previous bound holds for any probablity density function p 1 (y) and p 2 (y) and any prior distribution on θ.A natural distribution in the change point estimation problem is the discrete uniform distribution, i.e., In this case, it is easy to evaluate the remaining parts of the WWB.We have Consequently, the WWB for the estimation of one change point using a discrete uniform prior distribution is given by The only term that remains to be calculated in (24) is ρ 1,2 , which will be done in the next section in the particular cases of Gaussian and Poisson distributed series.

A. Gaussian distributions
We now assume that the observations y i follow a Gaussian distribution before and after the change point θ.The means and covariance matrices before and after θ are denoted µ 1 , µ 2 , σ 2 1 I, and σ 2 2 I, respectively.It is then possible to obtain the term ρ 1,2 defined in (16): due to the identification of a M-dimensional Gaussian density with mean and covariance matrix In the case of a single time series (M = 1), this expression can be simplified and written as a function of two signal-to-noise ratios1 defined in [11] as , and The expression of ρ 1,2 in (25) then becomes: In the following, we treat the two cases of mean jump and variance jump separately.
1) Gaussian mean change: In this case, the variance before and after the change point remains the same: σ 2 1 = σ 2 2 i.e. snr v 1,2 = 1.As a consequence, (26) can be simplified as follows: The associated Weiss-Weinstein bound is then obtained by plugging (27) into (24).
2) Gaussian variance change: In that case, the mean remains the same before and after the change point, i.e., µ 1 = µ 2 , which implies snr m 1,2 = 0.The expression of ρ 1,2 then reduces to: and as in the previous case, the Weiss-Weinstein bound is obtained thanks to (24).

B. Poisson distributions
Another case of interest in signal processing occurs with Poisson distributed observations.We denote λ 1 and λ 2 as the parameters of the distribution before and after the changepoint respectively.The corresponding ρ 1,2 term in (16) can then be obtained by taking into account the discrete nature of the distribution2 : Finally, as in the Gaussian case, the WWB expression is obtained by substituting (29) into (24).We use a similar definition of the signal-to-noise ratio as in [11], i.e., snr 1,2 = (λ1−λ2) 2 λ 2 1 .

V. SIMULATION RESULTS AND CONCLUSION
We present in this section the simulation results in the cases of a Gaussian mean change (Fig. 1) and of a Poisson parameter change (Fig. 2).Due to the lack of space, we do not present the result for the case of a Gaussian variance change, for which the same behavior is observed.On these figures, the empirical global mean square error (GMSE) of the maximum a posteriori (MAP) estimator is obtained by averaging the square errors over 1024 Monte-Carlo simulations.We compare them with the proposed theoretical WWB, as well as the averaged deterministic Barankin bound (BRB) adapted from [11], so that the comparison with the WWB makes sense.We recall that due to some regularity assumptions about the used parameter a priori support, the original Bayesian Barankin bound (and the Bayesian Cramér-Rao bound) cannot be computed for this problem.
We notice that, as expected, the proposed bound is in good agreement with the MAP estimator GMSE.