Journal of Global Positioning Systems (2002)
Vol. 1, No. 2: 122-131
A General Criterion of Integer Ambiguity Search
Guochang Xu
GeoForschungsZentrum Potsdam (GFZ), Telegrafenberg, 14473 Potsdam, Germany
Received: 9 October 2002 / Accepted: 6 January 2003
Abstract. A general criterion for integer ambiguity
searching is derived in this paper. The criterion takes into
account not only the residuals caused by ambiguity
parameter changing, but also the residuals caused by
coordinates changing through ambiguity fixing. The
search can be carried out in a coordinate domain, in an
ambiguity domain or in both domains. The three
searching scenarios are theoretically equivalent. The
optimality and uniqueness properties of the proposed
criterion are also discussed. A numerical explanation of
the general criterion is outlined. The theoretical
relationship between the general criterion and the
commonly used least squares ambiguity search (LSAS)
criterion is derived in an equivalent case in detail. It
shows that the LSAS criterion is just one of the terms of
the equivalent criterion. Numerical examples are given to
illustrate the behaviour of the two components of the
equivalent criterion.
Key words: Integer Ambiguity Searching Criterion
1 Introduction
It is well-known that the ambiguity resolution is a key
problem which has to be solved in GPS static and
kinematic precise positioning. Some well-derived
ambiguity fixing and searching algorithms have been
published during the last decade. These methods can be
generally classified as four types. The first type includes
Remondi's static initialisation approach (Remondi 1984;
Hofmann-Wellenhof et al. 1997; Wang et al. 1988),
which requires a static survey time to solve the ambiguity
unknowns after any complete loss of lock. Normally, the
results are good enough to take a round up ambiguity
fixing. The second type includes the so-called phase-code
combined methods (Han & Rizos 1995, 1997; Sjoeberg
1998, 1999); the phase and code have to be used in the
derivation as if they have the same precision, and in case
of anti-spoofing (AS), the C/A code has to be used. A
search process is still needed in this case. The third type
is the so-called ambiguity function method (Remondi
1984; Hofmann-Wellenhof et al. 1997); its search domain
is a geometric one. The fourth type includes approaches,
their search domain is only in domain of ambiguity,
including some optimal algorithms to reduce the search
area and to accelerate the search process (Euler & Landau
1992; Teunissen 1995; Leick 1995; Han & Rizos 1995,
1997). Because of the statistic character of validation
criteria, sometimes no valid result is obtained at the end
of the search processes.
The effort to develop the KSGSoft (Kinematic/Static
GPS Software) at the GeoForschungsZentrum (GFZ)
Potsdam began at the beginning of 1994 due to the
requirement of kinematic GPS positioning in
aerogravimetry applications (Xu et al. 1998, 1999). An
optimal ambiguity resolution method is needed in order
to implement it into the software; however, selecting the
published algorithms has turned out to be a difficult task.
This has led to the independent development of this so-
called integer ambiguity search method (cf. Xu 2003). It
turns out to be a very promising algorithm; the search
domain could be in the domain of coordinate or
ambiguity or both, and it is reliable and fast. Using this
general criterion, an optimal ambiguity vector can be
searched for and found out. The searched result is the
optimal one under the least squares principle and integer
ambiguity property.
The theoretical background of this method is the well-
known conditional least squares adjustment and will be
outlined below in the section 2. The well-known least
squares ambiguity search (LSAS) criterion is derived in
section 3. An analogue derivation of using coordinate
condition is outlined in section 4. A general criterion is
presented in section 5. Properties of the general criterion
are discussed in section 6. The relationship between the
general criterion and the least squares ambiguity search
criterion is derived in an equivalent case in section 7.

Xu: A General Criterion of Integer Ambiguity Search 123
where i is the element index of a vector or a matrix, sqrt()
is the square root operator, sd is the standard deviation (or
sigma) of unit weight, p[i] is the i-th element of the
precision vector, Qc[i][i] is the i-th diagonal element of
the quadratic matrix Qc, and
Numerical examples are given in section 8. Conclusions
and comments are given in the last section.
2 Conditional Least Squares Adjustment
Qc = Q − QCTQ2CQ (7)
The principle of least squares adjustment with condition
equations can be summarised as below (Cui et al. 1982;
Leick 1995; Gotthardt 1978; Xu 2003):
Q2 = (CQCT)−1 (8)
sd = sqrt((VTPV)c/(m
−
n+r)) if(m > n
−
r) (9)
1). Linearised observation equation system can be
represented by: 6). For recursive convenience, (VTPV)c can be calculated
by using:
V = L − AX, P (1) (VTPV)c = LTPL
−
(ATPL) TXc
−
WTK (10)
where Above are the complete formulas of conditional least
squares adjustment. The application of such an algorithm
for the purpose of integer ambiguity search will be further
discussed in later sections.
L: observation vector of dimension m,
A: coefficient matrix of dimension m×n,
X: unknown vector of dimension n,
V: residual vector of dimension m,
n: number of unknowns,
m: number of observations, 3 Integer Ambiguity Search in Ambiguity Domain
P: symmetric and quadratic weight matrix of
dimension mm. ×GPS observation equations can be represented with (1).
Considering the case without conditions (2), i.e., C = 0
and W = 0, the above equations are the same as the
results of normal least squares adjustment. So the least
squares solution of (1) is
2). The condition equation system can be written as:
CX − W = 0 (2)
where
X0 = Q(ATPL) = QW1 (11)
C: coefficient matrix of dimension r×n,
W: constant vector of dimension r, and
r: number of conditions. (VTPV)0 = LTPL
−
(ATPL)TX0 (12)
3). The least squares criterion for solving the observation
equations with condition equations is well-known as: sd = sqrt((VTPV)0/(m
−
n)), if(m > n) (13)
VTPV = min (3)
p[i] = sdsqrt(Q[i][i]) (14)
where
VT: the transpose of the related vector V.
4). The solution of the conditional problem (1) and (2)
under the least squares principle (3) is then:
Xc = (ATPA)
− 1(ATPL)
− (ATPA)
− 1CTK
= (ATPA)
− 1(ATPL
−
CTK) (4)
and
K = (CQCT)
− 1(CQW1
− W) (5)
Where index 0 is used for convenience to denote the
variables related to the normal least squares solution
without conditions. X0 is the complete unknown vector
including coordinates and ambiguities and is called a
float solution later on. Solution X0 is the optimal one
under least squares principle. However, because of the
observation and model errors as well as method
limitations, float solution X0 may not be exactly the right
one, e.g the ambiguity parameters are real numbers and
do not fit to the integer property. Therefore one
sometimes needs to search for a solution, say X, which
not only fulfils some special conditions, but also
meanwhile keeps the deviation of the solution as small as
possible (minimum). This can be represented by
where AT, CT are the transpose matrices of A, C, the
superscript
− 1 is an inversion operator, Q = (ATPA) − 1, K is
a gain vector (of dimension r), index c is used to denote
the variables related to the conditional solution, and W1 =
ATPL.
Vx
TPVx = min (15)
In (15) the Vx is the residuals vector in case of solution X.
For simplification, let:
5). The accuracies of the solutions are then:
p[i] = sdsqrt(Qc[i][i]) (6)

124 Journal of Global Positioning Systems
where (VTPV)0 is the value obtained without condition
(17). The second term on the right-hand side of (23) is the
often-used least squares ambiguity search criterion, (cf.
e.g. Teunissen 1995; Euler & Landau 1992; Hofmann-
Wellenhof et al. 1997), which can be expressed as








==








=








=
12
11
1
2221
1211
W
W
PLAW
QQ
QQ
Q
N
Y
X
T
δ (dN) = (N0 – N)T(Q22)
− 1(N0 – N) (24)








==
2221
1211
MM
MM
PAAM T (16)
1 −
=QM
where Y is the coordinate vector, N is the ambiguity
vector (generally, a real vector). To use the conditional
adjustment algorithm for integer ambiguity searching in
ambiguity domain, the condition shall be selected as N =
W, here W is, of course, an integer vector. Generally,
letting C = (0, E), then condition (2) turns out to be:
N = W (17)
It indicates that any ambiguity fixing will cause an
enlargement of the standard deviation. However, one may
also notice that here only the enlargement of the standard
deviation caused by ambiguity parameter changing has
been considered. Any ambiguity fixing will lead to a
related coordinate changing (cf. (20)). Furthermore, the
condition (17) does not really exist. Ambiguities are
integers, however, they are unknowns. The formula to
compute the accuracy vector of the ambiguity does not
exist too, because the ambiguity condition is considered
exactly known in conditional adjustment (cf. Xu 2003).
Using definitions of C and Q, one has:
()
2221 QQCQ =
4 Standard Deviation Enlargement Caused by Coordi-
nate Changing
CQC T = Q22
The float solution is denoted as
Analogous to above discussion, the condition could be
selected as Y = W, here W is a coordinate vector.
Generally, letting C = (E, 0), where E is an identity
matrix with dimension of rr, C has dimension r×n,
condition (2) turns out to be:
×
X0 = =








0
0
N
Y







+
+
12221121
12121111
WQWQ
WQWQ
where X0 is the solution of (1) without condition (17).
The gain vector KN can be computed by:
Y = W (25)
KN = (Q22)
− 1(CQW1
− W) = (Q22)
− 1(N0
− W) (18)
Using definitions of C and Q, one has:
So under the condition (17), the conditional least squares
solution (4) can be written as:
()
1211 QQCQ=
CQC T = Q11








−








=








=
Nc
c
cKW
W
QQ
QQ
N
Y
X
12
11
2221
1211
The float solution is denoted by
N
K
Q
Q
N
Y







−








=
22
12
0
0 (19)



=




=
0
Y
X



+
+



12221121
12121111
0
0
WQWQ
WQWQ
N
Simplifying (19), one gets:
Yc = Y0 – Q12KN (20)
where X0 is the solution of (1) without condition (25).
The gain vector KY can be computed by using (5):
KY = (Q11)
− 1(CQW1
− W) = (Q11)
− 1(Y0
− W) (26)
and
So under the condition (25), the conditional least squares
solution (4) can be written as:
Nc = N0 − Q22KN = N0 − Q22(Q22)−1(N0−W) = W (21)
The precision computing formulas under condition (17)
can be derived as below: 






−








=








=
12
11
2221
1211
W
KW
QQ
QQ
N
Y
XY
c
c
c
CQQQCQQT
c
1
22 )( −
−=







−
=−
00
0)( 21
1
221211 QQQQ (22)



=Y K
Q
Q
N
Y







−



 
21
11
0
0 (27)
Simplifying (27), one gets:
0
)()( PVVPVV T
c
T= Yc = Y0 – Q11KY
)()()( 0
1
220WNQWN T−−+− (23) = Y0 – Q11(Q11)
− 1(Y0
− W) = W (28)

Xu: A General Criterion of Integer Ambiguity Search 125
K = Q
− 1(CQW1
− W) = Q
− 1(X0
− W) (34)
and
Nc = N0 − Q21KY (29)
So under the condition (33), the conditional least squares
solution (4) can be written as:
For any given constant coordinate vector W, an ambiguity
vector Nc can be found out (or computed). In such a case,
Nc is a float vector and Yc is exactly the same as that
given in condition (25). If the Yc is a correct one, the
computed Nc should be very close to an integer vector
under the assumptions made at the beginning. The
searched integer ambiguity vector is then Fix(Nc), where
Fix() is a round up function for rounding up a real
number to its nearest integer number. A more detailed
discussion on the use of the rounding function to the
computed vector Nc will be made in section 6. The
precision computing formula under condition (25) can be
derived by using definitions:
Xc = X0
−
QK = X0
−
QQ
− 1(X0
−
W) = W (35)
Precision computing formulas under condition (33) can
be derived as below:
Qc = 0 (36)
(VTPV)c = (VTPV)0 + (X0
−
W)TQ
− 1(X0
−
W) (37)
where (VTPV)0 is the value obtained without condition
(33).
The second term on the right side of (37) can be used as a
general criterion for integer ambiguity search, i.e.:
δ = (X0 − X)TQ
− 1(X0 – X) (38)
CQQQCQQ T
c
1
11 )( −
−=








−
=−
12
1
112122 )(0
00
QQQQ (30)
0
)()( PVVPVV T
c
T=
)()()( 0
1
110WYQWY T−−+ − (31)
It indicates the enlargement of the standard deviation
caused by fixed solution X. A minimum value of (38) is
equivalent to a minimum value of (VTPV)c. Therefore an
optimal fixed solution has to be searched for so that (38)
has the minimum value. To be noticed is that the
minimum value of (38) is not a minimization process, but
just a searching process to find out the optimal X. (38)
has obviously a more general form than the least squares
ambiguity search criterion (24) does.
where (VTPV)0 is the value obtained without condition
(25). The second term on the right-hand side of (31)
cannot be used directly as a criterion for an ambiguity
search; however, it indicates that any coordinate change
will cause an enlargement of the standard deviation. For
convenience, we denote
δ
1(dY) = (Y0 – Y)T(Q11)
− 1(Y0 – Y) (32)
In all above three derivations, to be noticed is that in the
precision vector the condition related elements are not
defined. This is because in the conditional adjustment
conditions are considered exactly known. However, in
integer ambiguity searching, to be tested candidates (e.g.
integer ambiguity) are indeed not exactly known or say,
known with uncertainty (float solution with its precision).
The uncertainty of the computed ambiguity and selected
coordinate vectors (related to the searching in coordinate
domain), and the uncertainty of the computed coordinate
and selected ambiguity vectors (related to the searching
in ambiguity domain), as well as the uncertainty of the
selected vector in both domains (related to the searching
in both domains) should be taken into account in any
cases. Therefore (38) is a more reasonable criterion and
should be used generally in ambiguity searching no
matter in which domain the search will be made. Under
such criterion, the deviation of the result vector X related
to the float vector X0 is homogenously considered. For
computing the precision of the searched X, the formulas
of least squares adjustment shall be further used, and
meanwhile the enlarged residuals shall be taken into
account by
It is obvious that such an effect has to be taken into
account in the ambiguity fixing. This will be further
discussed in next section.
5 Integer Ambiguity Search in Coordinate and Ambi-
guity Domains
Even the to be fixed solution is an unknown vector,
however, in order to see the enlargement of the standard
deviation caused by the fixed solution, the condition
could be selected as X = W, here W consists of two sub-
vectors (coordinate and ambiguity parameter related sub-
vectors). And only the ambiguity parameter related sub-
vector is an integer one. Letting C = E, condition (2) is
then:
X = W (33)
p[i] = sd sqrt(Q[i][i]) (39)
One has: sd = sqrt((VTPV)c/(m
−
n)) if(m > n) (40)
CQ = CQC T = Q (VTPV)c = (VTPV)0 +
δ
(41)
Denote X0 = QW1; here X0 is the solution of (1) without
condition (33). The gain K can be computed by:

126 Journal of Global Positioning Systems
In other words, the original Q matrix and (VTPV)0 of the
least squares problem (1) are further used. The
δ
has the
function of enlarging the standard deviation. The
formulas of (38), (39--41) are partly derived from the
conditional adjustment, however, the formulas have
nothing to do with the conditions. Searching for a
minimum δ leads to a minimum of sd and therefore the
best precision vector p[i]. The geometric explanation of
here proposed integer ambiguity searching criterion is
discussed in section 6.
The general criterion of (38) is used for all three
searching scenarios, where X0 is the float solution, Q is
the inversion of the complete normal matrix of (1). X is
the selected candidate vector in case of searching in both
coordinate and ambiguity domains. In case of searching
in coordinate domain, X consists of the selected sub-
vector of Yc in (28) and the computed sub-vector of
Fix(Nc) in (29), i.e.:








=)( c
c
NFix
Y
X (42)
The reason why the Fix(Nc) is used here will be discussed
theoretically in next section. In the case of searching in
ambiguity domain, X consists of the selected sub-vector
of Nc in (21) and the computed coordinate sub-vector Yc
in (20), i.e.:








=
c
c
N
Y
X (43)
6 Properties of the General Criterion
1). Equivalence of the Three Searching Processes
To be emphasised is that the same searching criterion
(38) and the same formulas of precision estimation (39—
41) are used in the three integer ambiguity search
scenarios. And the same normal equations of (1) is used
to compute the vector Nc using selected Yc or to compute
the Yc using selected Nc if necessary. The three searching
processes indeed deal with the same problem, just as
different ways of searching are used.
Suppose by searching in ambiguity domain, the vector X
= (Yc Nc)T is found so that δ reaches the minimum, where
Nc is selected integer sub-vector and Yc is the computed
one. In the case of searching in coordinate domain, if the
selected coordinate sub-vector Y is exactly the same as Yc,
then integer sub-vector N obtained by computation
should be exactly the same as Nc. Taking the computing
errors into account, the computed N could be a real
vector, however, the errors must be very small and the
rounding vector Fix(N) must be the same as Nc. (This is
also the reason why the rounding function is used for the
computed vector Nc in the case of search in coordinate
domain). We see now the same results will be obtained
theoretically in the both searching cases. Therefore, the
searching methods in coordinate domain or in ambiguity
domain are theoretically equivalent.
Suppose by searching in ambiguity domain, again, the
vector (Yc Nc)T is obtained. And in the case of searching
in both coordinate and ambiguity domains, a candidate
vector X = (Y N)T is selected so that
δ
reaches the
minimum, where N is selected integer sub-vector and Y is
selected coordinate vector. Because of the optimality and
uniqueness properties of the vector X in (38) (please refer
to 2, which is discussed next), here selected (Y N)T must
be equal to (Yc Nc)T. So the theoretical equivalency of the
three searching processes is confirmed.
In practice, it could be difficult to have a selected Y that
exactly equals the computed Yc (computed by searching
in ambiguity domain). However, it is always possible to
get a Y that is as close as required to Yc by selecting
smaller search steps.
2). Optimality and Uniqueness Properties
The float solution X0 is the optimal and unique solution of
(1) under the least squares principle. Using the integer
ambiguity search criterion (38), analogously, the searched
vector X is the optimal solution of (1) under the least
squares principle and integer ambiguity properties. A
minimum of δ in (38) will lead to a minimum of (VTPV)c
in (41). The uniqueness property is obvious. If X1 and X2
are such that
δ
(X1) =
δ
(X2) = min., or
δ
(X1) –
δ
(X2) = 0,
then by using (38), one may assume that X1 must be equal
to X2.
3). Geometric Explanation of the General Criterion
Geometrically, δ = (X0 – X)TQ
− 1(X0 – X) is the “distance”
between the vector X and float vector X0. The distance
contributed to enlarge the standard deviation sd (cf. (40)).
Ambiguity searching is then the search for the vector,
which own the integer ambiguity property and has the
minimum distance to the float vector.
In the next section, the relationship between above
proposed general criterion and the common used least
squares ambiguity search criterion (derived in §3) will be
discussed.
7 Relationship Between the Two Criteria
We are going to prove theoretically that LSAS criterion
(24) is just one of the terms of an equivalent criterion of
the general criterion (38) as follows.
The normal equation of (1) can be denoted by (use
notation of (16)):

Xu: A General Criterion of Integer Ambiguity Search 127








=
















12
11
2221
1211
W
W
N
Y
MM
MM (44) 




MM







=

−
2221
1211
1
2221
1211
QQ
QQ
MM (56)
or where (cf. e.g. Cui et al. 1982; Leick 1995; Gotthardt
1978)
M11Y + M12N = W11 (45)
M21Y + M22N = W12 (46)
Q11 = (M11 – M12(M22)
− 1M21) − 1 (57)
Q22 = (M22 – M21(M11)
− 1M12) − 1 (58)
The normal equation (45) and (46) can be solved by
block-wise elimination as follows. From (45), one has: Q12 = (M11)
− 1(–M12Q22) (59)
Q21 = (M22)
− 1(–M21Q11) (60)
Y = (M11)
− 1(W11 – M12N) then after comparing (57) and (58) with (52) and (48) one
has
Setting Y into (46), one gets a normal equation related to
the second block of unknowns:
M2N = B2 (47)
Q11 = (M1)
− 1, Q22 = (M2) − 1
or
where
M2 = M22 – M21(M11)
− 1M12 (48)
M1 = (Q11)
− 1, M2 = (Q22) − 1 (61)
B2 = W12 – M21(M11)
− 1W11 (49)
Then (55) turns out to be
δ
1 = (Y0 – Y)T(Q11)
− 1(Y0 – Y)
+ (N0 – N)T(Q22)
− 1(N0 – N) (62)
Similarly, from (46), one has
N = (M22)
− 1(W12 – M21Y) (50)
Setting N into (45), one gets a normal equation related to
the first block of unknowns:
M1Y = B1 (51)
Note that the second term on the right-hand side of (62) is
exactly the same as the criterion of the least squares
ambiguity search (24). In other words, the criterion of
least squares ambiguity search is just one term of the
equivalent criterion (62) (q.e.d).
where
M1 = M11 – M12(M22)
− 1M21 (52)
B1 = W11 – M12(M22)
− 1W12 (53)
It should be emphasised that the consistency between the
coordinate sub-vector Y and ambiguity sub-vector N is
implicitly used by the proof. Therefore (62) is only valid
if the Y and N are consistent each other. The first term on
the right-hand side of (62) is the same as the (32), which
indicates an enlargement of the standard deviation due to
the coordinate change caused by ambiguity fixing.
Then the normal equation of (44) can be written by
combining (51) and (47) as








=
















2
1
2
1
0
0
B
B
N
Y
M
M (54)
Now, it is obvious that
1). only if one may lead from a minimum value of (24)
δ (dN) = (N0 – N)T(Q22)
− 1(N0 – N) (63)
(44) and (54) are two equivalent normal equations,
therefore the integer ambiguity search using (44) or (54)
are also equivalent. Using the notation (16), the normal
equation of (1) is MX = W1 and the general criterion is
(38). Because of M = Q
− 1, (38) is the same as: (X0 –
X)TM(X0 – X). So for the normal equation of (54), the
related general criterion (38) turns out to be (put the
diagonal M into above formula!):
to get a minimum value of (62)
δ
1 = (Y0 – Y)T(Q11)
− 1(Y0 – Y)
+ (N0 – N)T(Q22)
− 1(N0 – N) (64)
then the least squares ambiguity search is equivalent to
the general method proposed in §5. However, such a
generality does not exist. Therefore, the LSAS criterion is
generally not equivalent to the criterion (64) (which is
equivalent to the general criterion (38)). Furthermore,
using (20) and (18) one has
δ
1 =








−
−
















−
−
NN
YY
M
M
NN
YYT
0
0
2
1
0
0
0
0
Y0 – Y = Q12(Q22)
− 1(N0 – N) (65)
or
δ
1 = (Y0 – Y)TM1(Y0 – Y) + (N0 – N)TM2(N0 – N) (55) Putting (65) into (64), one has
It has to be emphasised that search criterion (55) is
equivalent to the criterion (38), however, they are not
identical, or generally, δ ≠ δ
1. Furthermore, denote
⋅+⋅−= ]})(QQ)(QQ[E){(QN)(N -1
2212
-1
1121
-1
22
T
01
δ
N)(N0− (66)

128 Journal of Global Positioning Systems
One may see clearly now the differences between the two
criteria (63) and (66).
2). If one may not lead from a minimum value of (63) to
get a minimum value of (64), then the least squares
ambiguity search may not find the optimal results in view
point of the criterion (62). In this case, only criterion (62)
reaches a minimum with a unique and optimal vector X.
3). The coordinate change due to the ambiguity fixing has
not been taken into account in the least squares ambiguity
search criterion.
A by-product of above derivation is that we have now a
criterion (62) which is equivalent to the criterion (38). By
computing the precision vector of (39)—(41), the δ has to
be computed using (38), because the δ is not equal δ
1 in
general.
8 Numerical Examples of General Criterion and
LSAS Criterion
Several numerical examples are given here to illustrate
the behaviour of the two terms of the criterion. For
convenience, we denote the first and second terms of the
right-hand side of (62) as δ (dY) and δ (dN) respectively.
δ
1 =
δ
(dY)+
δ
(dN) is the equivalent criterion of the
general criterion and is denoted as δ (total). The term
δ (dN) is the LSAS criterion. Of course, the search is
made in the ambiguity domain. Analogues, the general
criterion is also used for search in the ambiguity domain.
The search area is determined by the precision vector of
the float solution. All possible candidates are tested one
by one, and the related δ1 are compared to each other to
find out the minimum.
In the first example, precise orbits and dual-frequency
GPS data of 15 April 1999 at station Brst (N 48.3805°, E
355.5034°) and Hers (N 50.8673°, E 0.3363°) are used.
Session length is 4 hours. The total search candidate
number is 1020. Results of the two sigma components are
illustrated as 2-D graphics with the 1st axis of search
number and the 2nd axis of sigma in Fig. 1. The red and
blue lines represent
δ (dY) and δ (dN), respectively. δ (dY)
reaches the minimum at the search number 237, and
δ (dN) at 769. δ (total) is plotted in Fig. 2, and it shows
that the general criterion reaches the minimum at the
search number 493. For more detail, a part of the results
are listed in the Tab. 1.
Tab. 1 Sigma values of searching process
Search No. δ(dN) δ(dY) δ(total)
237 183.0937 97.8046 280.8984
493 181.7359 97.9494 279.6853
769 93.3593 315.2760 408.6353
771 96.0678 343.5736 439.6414
0200400600800 1000
Search number
100
200
300
400
500
Sigma
Two components of general ambiguity search criterion
Sigma_dN
Sigma_dY
min(Sigma_dY) min(Sigma_dN) min(total)
Fig. 1 Two components of the general ambiguity search criterion

Xu: A General Criterion of Integer Ambiguity Search 129
0200400600800 1000
Search number
200
300
400
500
600
700
Sigma
General ambiguity search criterion
min(total)
min(total)
Fig. 2 General ambiguity search criterion
050100 150 200250
search number
0
100
200
300
400
Sigma
min(Sigma_dN) min(Sigma_dY)
Example of general ambiguity search criterion
Fig. 3 Example of general ambiguity search criterion

130 Journal of Global Positioning Systems
The δ (dN) reaches the second minimum at search No.
771. This example shows that the minimum of
δ
(dN) may
not lead to the minimum of total sigma, because the
related δ (dY) is large. If the sigma ratio criterion is used
in this case, the LSAS method will reject the found
minimum and explain that no significant ambiguity fixing
can be made. However, because of the uniqueness
principle of the general criterion, the search reaches the
total minimum uniquely.
The second example is very similar to the first one. The
sigmas of the search process are plotted in Figure 3,
where δ (dY) is much smaller than δ (dN). δ (dN) reaches
the minimum at the search number 5 and
δ
(dY) at 171.
δ
(total) reaches the minimum at the search number 129.
The total 11 ambiguity parameters are fixed and listed in
Table 2. Two ambiguity fixings have just one cycle
difference at the 6th ambiguity parameter. The related
coordinate solutions after the ambiguity fixings are listed
in Table 3. The coordinate differences at component x
and z are about 5 mm. Even the results are very similar,
however, two criteria do give different results.
Tab. 2 Two kinds of ambiguity fixing due to two criteria
Ambiguity No. 1 2 3 4 5 6 7 8 9 10 11
LSAS fixing 0 0 1 0 0 0 -1 0 0 -1 -1
General fixing 0 0 1 0 0 -1 -1 0 0 -1 -1
Tab. 3 Ambiguity fixed coordinate solutions (in meter)
Coordinates x y z
LSAS fixng 0.2140 -0.0449 0.1078
General fixing 0.2213 -0.0465 0.1127
Tab. 4 Sigmas of ambiguity search process
Search No. δ(dN) δ(dY) δ(total)
1 248.5681 129.0555 377.6236
2 702.6925 58.9271 761.6195
3 889.5496 107.9330 997.4825
4 452.1952 42.3226 494.5178
5 186.7937 112.3030 299.0967
6 739.0487 55.9744 795.0231
7 931.4125 89.9074 1021.3199
8 592.1887 38.0969 630.2856
In the third example, real GPS data of 3 October 1997 at
station Faim (N 38.5295°, E 331.3711°) and Flor (N
39.4493°, E 328.8715°) are used. The sigmas of the
search process are listed in Table 4. Both δ (dN) and
δ (total) reach the minimum at the search number 5. This
indicates that the LSAS criterion may sometimes reach
the same result as that of the equivalent criterion being
used.
9 Conclusions and Comments
1). Conclusions
A general criterion of integer ambiguity search is
proposed in this paper. The search can be carried out in a
coordinate domain, in an ambiguity domain or in both
domains. The criterion takes the both coordinate and
ambiguity residuals into account. The equivalency of the
three searching processes are proved theoretically. The
searched result is optimal and unique under the least
squares minimum principle and under the condition of
integer ambiguities. The criterion has a clear numerical
explanation. The theoretical relationship between the
general criterion and the common used least squares
ambiguity search (LSAS) criterion is derived in detail. It
shows that the LSAS criterion is just one of the terms of
the equivalent criterion of the general criterion (does not
take into account the coordinate change due to the
ambiguity fixing). Numerical examples shown that, a
minimum δ(dN) may have a relatively large δ(dY), and
therefore a minimum δ(dN) may not guarantee a
minimum δ(total).
2). Comments
The float solution is the optimal solution of the GPS
problem under the least squares minimum principle.
Using the general criterion, the searched solution is the
optimal solution under the least squares minimum
principle and under the condition of integer ambiguities.
However, the ambiguity searching criterion is just a
statistic criterion. Statistic correctness does not guarantee
correctness in all applications. Ambiguity fixing only
makes sense when the GPS observables are good enough
and the data processing models are accurate enough.
References
Cui, X.; Yu, Z.; Tao, B.; Liu, D. (1982): Adjustment in
Surveying, Surveying Publishing House, Peking
Euler, H.J.; Landau, H. (1992): Fast GPS ambiguity resolution
on-the-fly for real-time applications. Proceedings of 6th
Int. Geod. Symp. on satellite Positioning. Columbus, Ohio,
17-20.
Gotthardt, E. (1978): Einfuehrung in die
Ausgleichungsrechnung, Herbert Wichmann Verlag
Karlsruhe
Han, S.; Rizos, C. (1995): On-The-Fly Ambiguity Resolution
for Long Range GPS Kinematic Positioning, IAG
Symposia 115, edited by Beutler, Hein, Melbourne and
Seeber.
Han, S.; Rizos, C. (1997): Comparing GPS Ambiguity
Resolution Techniques, GPS World, Oct. 1997, 54-61
Hofmann-Wellenhof, B; Lichtenegger, H.; Collins, J. (1997):
GPS Theory and Practice, Springer-Verlag, Wien

Xu: A General Criterion of Integer Ambiguity Search 131
Leick, A. (1995). GPS Satellite Surveying. New York: John
Wiley & Sons.
Remondi, B. (1984): Using the Global Positioning System
(GPS) phase observable for relative geodesy: modelling,
processing, and results. University of Texas at Austin,
Center for Space Research.
Sjoeberg, L. (1998): On the estimation of GPS phase
ambiguities by triple frequency phase and code data, ZfV,
123(1998)5, pp. 162-163, Stuttgart, 1998.
Sjoeberg,L. E. (1999): Unbiased vs biased estimation of GPS
phase ambiguities from dual-frequency code and phase
observables, Journal of Geodesy (1999) 73: 118-124
Teunissen, P.J.G. (1995): The least-squares ambiguity
decorrelation adjustment: a method for fast GPS integer
ambiguity estimation. Journal of Geodesy, 70, 65-82.
Wang, G.; Chen, Z.; Chen, W.; Xu, G. (1988): The Principle of
the GPS Precise Positioning System, Surveying
Publishing House, Peking
Xu, G.; Bastos, L.; Timmen, L. (1997): GPS Kinematic
Positioning in AGMASCO Campaigns-Strategic Goals
and Numerical Results, Proceedings of ION GPS-97
Conference in Kansas City (USA), p1173-1183
Xu, G.; Schwintzer, P.; Reigber, Ch. (1998): KSGSoft
(Kinematic/Static GPS Software) --- Software User
Manual, Scientific Technical Report 19/1998,
GeoForschungsZentrum Potsdam
Xu, G. (2003): GPS - Theory, Algorithms and Applications,
Springer Heidelberg, in English, in print