Asymptotic boundary forms for tight Gabor frames and lattice localization domains

We consider Gabor localization operators $G_{\phi,\Omega}$ defined by two parameters, the generating function $\phi$ of a tight Gabor frame $\{\phi_\lambda\}_{\lambda \in \Lambda}$, parametrized by the elements of a given lattice $\Lambda \subset \Bbb{R}^2$, i.e. a discrete cocompact subgroup of $\Bbb{R}^2$, and a lattice localization domain $\Omega \subset \Bbb{R}^2$ with its boundary consisting of line segments connecting points of $\Lambda$. We find an explicit formula for the boundary form $BF(\phi,\Omega)=\text{A}_\Lambda \lim_{R\rightarrow \infty}\frac{PF(G_{\phi,R\Omega})}{R}$, the normalized limit of the projection functional $PF(G_{\phi,\Omega})=\sum_{i=0}^{\infty}\lambda_i(G_{\phi,\Omega})(1-\lambda_i(G_{\phi,\Omega}))$, where $\lambda_i(G_{\phi,\Omega})$ are the eigenvalues of the localization operators $G_{\phi,\Omega}$ applied to dilated domains $R\Omega$, $R$ is an integer and $\text{A}_\Lambda$ is the area of the fundamental domain of the lattice $\Lambda$.

Abstract. We consider Gabor localization operators G φ,Ω defined by two parameters, the generating function φ of a tight Gabor frame {φ λ } λ∈Λ , parametrized by the elements of a given lattice Λ ⊂ R 2 , i.e. a discrete cocompact subgroup of R 2 , and a lattice localization domain Ω ⊂ R 2 with its boundary consisting of line segments connecting points of Λ . We find an explicit formula for the boundary form BF (φ, Ω) = A Λ lim R→∞ P F (G φ,RΩ ) R , the normalized limit of the projection functional P F (G φ,Ω ) = ∞ i=0 λ i (G φ,Ω )(1 − λ i (G φ,Ω )), where λ i (G φ,Ω ) are the eigenvalues of the localization operators G φ,Ω applied to dilated domains RΩ, R is an integer and A Λ is the area of the fundamental domain of the lattice Λ . Although the lattice Λ is also a parameter of the localization operator we assume that it is fixed and we do not list it explicitly in our notation. The boundary form expresses quantitatively the asymptotic interactions between the generating function φ of a tight Gabor frame and the oriented boundary ∂Ω of a lattice localization domain from the point of view of the projection functional, which measures to what degree a given trace class operator fails to be an orthogonal projection. It provides an evaluation framework for finding the best asymptotic matching between pairs consisting of generating functions φ and lattice domains Ω. It takes into account directional information involving outer normal vectors of the linear segments constituting the boundary of Ω and the weighted sums over the corresponding half spaces of the absolute value squares of the reproducing kernels obtained out of φ. In the context of tight Gabor frames and Gabor localization domains placing an upper bound on the square of the L 2 norm of the generating function φ divided by the area of the fundamental domain of lattice Λ corresponds to keeping the relative redundancy of the frame bounded. Keeping the area of the localization domain Ω bounded above corresponds to controlling the relative dimensionality of the localization problem.

Main Results and Their Context
We start by formulating the main results of this paper. Let Λ ⊂ R 2 be a lattice, i.e. a discrete cocompact subgroup of R 2 , satisfying condition A Λ < 1, where A Λ denotes the area of the fundamental domain of Λ. For a function φ ∈ L 2 (R) and λ = (q, p) we define φ λ (x) = e 2πipx φ(x − q). Let the system {φ λ } λ∈Λ , where φ ∈ L 2 (R) is a fixed function, be a tight Gabor frame, i.e. we assume that for every f ∈ L 2 (R) where the convergence of the sum is understood in the unconditional norm sense. Function φ is called the generating function of a tight Gabor frame {φ λ } λ∈Λ . An operator G φ,b , acting on L 2 (R), and of the form is called a Gabor multiplier of localization type, if its symbol b, defined on the lattice Λ, is non-negative and summable. It can be easily verified that Gabor multipliers of localization type are non-negative, trace class, and that tr(G φ,b ) = ||φ|| 2 and the operator norm of G φ,b satisfies The projection functional P F is defined on positive definite, trace class operators T , with their operator norm bounded above by 1, via the formula where {λ i (T )} ∞ i=0 are the eigenvalues of T . The projection functional measures the extend by which the operator T fails to be an orthogonal projection. It takes nonnegative values and it vanishes on the space consisting of finite dimensional orthogonal projections. A collection of line segments l i ⊂ R 2 , i = 1, 2, ..., C each of them starting and ending at a point of Λ is called a Λ cycle if the union l 1 ∪ l 2 ∪ ... ∪ l C forms a closed continuous line without self intersections. A bounded, connected and closed subset Ω ⊂ R 2 is called a Λ domain if its boundary consists of a finite family of Λ cycles C i , i = 1, 2, ..., B satisfying condition dist(C i , C j ) > 0 for i = j . We say that a function φ ∈ L 2 (R) satisfies condition Φ if Φ: For a Λ domain Ω ⊂ R 2 and a generating function φ of a tight Gabor frame {φ λ } λ∈Λ satisfying condition Φ the boundary form BF(φ, Ω) is defined by the formula where l i , i = 1, ..., N are the line segments constituting the boundary of Ω, n i is the unit vector orthogonal to l i directed outside Ω, P i = {w ∈ R 2 | w · n i = 0}, The projective metaplectic representation defined on SL(2, R) provides the most natural way to deal with linear changes of coordinates of the time-frequency plane R 2 . Its definition and its basic properties are reviewed in Section 4. It came to us as a surprise, that the boundary form BF , representing the limit value of the projection functional P F with respect to dilation factors of the localization domain tending to infinity, is invariant with respect to the action of the projective metaplectic representation. Many geometric and numerical studies of lattices in R 2 distinguish the hexagonal lattice as being special, but this is not the case in the context of the projection functional P F and its limit value BF .
In our first result we describe the invariance properties of P F and BF with respect to the projective metaplectic representation. Although the invariance of P F is standard we include it as well for the sake of completeness. Theorem 1.1. Let µ be the projective metaplectic representation of SL(2, R) acting on L 2 (R).Let φ be a generating function of a tight Gabor frame {φ λ } λ∈Λ satisfying condition Φ with respect to Λ. Then for any A ∈ SL(2, R) Γ = A(Λ) is a lattice, {(µ(A)φ) γ } γ∈Γ is a tight Gabor frame with the generating function µ(A)φ satisfying condition Φ with respect to Γ, and (i) for any Gabor multiplier G φ,b of localization type with the symbol bounded above by 1, G µ(A)φ,b•A −1 is a Gabor multiplier of localization type with the symbol bounded above by 1, and (ii) for any Ω a Λ domain, A(Ω) is a Γ domain, and BF (φ, Ω) = BF (µ(A)φ, A(Ω)).
Our second result is the principal result of the current paper. It describes the limit behavior of the projection functional P F applied to a Gabor multiplier with the symbol of the form b R = χ RΩ , i.e. the characteristic function of a dilated lattice domain Ω by a factor R ∈ Z, with R → ∞. Gabor multipliers of this special form are called Gabor localization operators and they are denoted as G φ,RΩ . We will see later on that the dilation factor R can in fact take any real values. Theorem 1.2. Let φ be a generating function of a tight Gabor frame parametrized by lattice Λ and satisfying condition Φ, and Ω a Λ domain contained in R 2 . Then Formula (7) expresses the limit behavior of the projection functional PF in terms of the boundary form BF. It provides a very explicit, quantitative way of describing the interactions between the boundary ∂Ω and the reproducing kernel obtained out of the generating function φ.
The invariance of the projection functional PF and the boundary form BF with respect to the action of the projective metaplectic representation expressed in Theorem 1.1, taken together with the limit result of Theorem 1.2, has its important consequences. We can conclude that no lattice Λ is distinguished, neither from the point of view the value of the limit of P F (G φ,RΩ ) as R → ∞, nor from the point of view of its rate of convergence. Corollary 1.3. For any lattice Λ ⊂ R 2 satisfying condition A Λ < 1, any generating function φ of a tight Gabor frame {φ λ } λ∈Λ , any Λ lattice domain Ω, and any a, b > 0 satisfying ab = A Λ , there are a generating functionφ of a tight Gabor frame {φ λ } λ∈aZ×bZ and a aZ × bZ lattice domainΩ, satisfying ||φ|| L 2 (R) = ||φ|| L 2 (R) , Area(Ω) = Area(Ω), P F (G φ,RΩ ) = P F (Gφ ,RΩ ), for all R > 0, and also BF (φ, Ω) = BF (φ,Ω). The rates of convergence of Many important mathematical theories started in the one dimensional setup, where a multitude of additional tools is available, and then through various stages of evolution came up into their full form in any finite dimension. This was the case of the representation theory of semi-simple Lie groups, which started with the listing of all irreducible representations of SL(2, R). Classical time-frequency localization operators allow explicit diagonalization in one dimension, but not in higher dimensions, yet it was possible to transfer a large portion of one dimensional results to higher dimensions. In one dimension, for several classes of potentials, Schrödinger operators can be treated via explicit formulae, yet many results that follow the guidelines of the one dimensional setup are also true in higher dimensions, although explicit formulae are no longer available. The development of the theory of Gabor frames follows a similar path. As this is the case of many relatively recent theories, some of its branches are still at an early, one dimensional stage, e.g. the treatment of Gabor frames with maximal lattice parameter frame set via totally positive functions of finite type done recently by Gröchenig and Stöckler in [23]. In many instances the distinction between one and higher dimensions is related to the differences between the theories of one and several complex variables. The development of the phase space theory of reproducing formulae shows many similarities with the development of the theory of Gabor frames. The semidirect product R 2n ⋊ Sp(n, R), where Sp(n, R) is the symplectic group consisting of 2n × 2n invertible matrices preserving the symplectic form, together with the extended projective metaplectic representation defined on it, constitute the group of affine transformations of the phase space, acting geometrically on R 2n , and analytically on L 2 (R n ). The affine transformations of the phase space provide a natural framework for the constructions of reproducing formulae. All reproducing formulae coming out of connected Lie subgroups of R 2n ⋊ Sp(n, R) were characterized by De Mari and Nowak for n = 1 in [8], but no analogous results are known in higher dimensions. In dimension one Sp(1, R) = SL(2, R) and it is possible to obtain the list of all connected Lie subgroups of R 2 ⋊ SL(2, R) out of the classical structure results describing the orbits of inner automorphisms of SL(2, R). No analogous lists of subgroups are available in higher dimensions. In the current paper we deal with tight Gabor frames parametrized by a lattice Λ ⊂ R 2 . Any tight Gabor frame parameterized by a lattice Λ ⊂ R 2 can be transferred into a tight Gabor frame parameterized by a separable lattice aZ × bZ, a, b > 0, by an appropriate SL(2, R) linear transformation of the time-frequency plane. We treat the case of a separable lattice via an explicit computation and then we transfer the formula we obtain to an arbitrary lattice with the help of the projective metaplectic representation. This approach does not generalize to higher dimensions.
The principal results of the current paper deal with discrete one-dimensional setup of tight Gabor frames, therefore for the sake of consistency we formulate definitions and reference results only in one dimension. Operators of composition of convolution with g followed by a pointwise multiplication by f , where both functions f and g are defined on the real line R, have integral kernels of the form They are commonly called convolution-product operators. Historically three dimensional convolution-product operators played an important role in the study of Schrödinger operators. The Birman-Schwinger principle allows a transition from a Schrödinger operator to a convolution-product operator. Out of that transition it was possible to obtain sharp estimates for the number of bound states of the Schrödinger operator. Classical time-frequency localization operators are one dimensional convolution-product operators with f andǧ characteristic functions of intervals, whereǧ is the inverse Fourier transform of g . Their spectral properties were carefully studied many years ago by Landau, Pollak, Slepian and Widom (see [6], [29], [28], and the references provided within), yet till now these classical results bring important ingredients for both theoretical and applied components occurring in many recent developments. There were also extensive studies of convolution-product operators with one of the function parameters f or g fixed and of prescribed potential type (see the book by Mazya [33]), but the dependence on both parameters, and the mutual interaction between f and g in the general case seems to be a difficult problem that is to a large degree still open.
We are interested in operators with integral kernels of the form (8), where translations constituting the convolution with g are extended to combined actions of translations and modulations of the Schrödinger representation, and applied to a function φ. As a consequence of this extension we need to introduce two arguments of the multiplier f , one corresponding to translations, the other to modulations, and the extended integral kernel (8) becomes where φ q,p (x) = e 2πipx φ(x − q). We add complex conjugate over φ q,p , because we want our operators to be exactly square roots of Gabor-Toeplitz operators. The Gabor reproducing formula has the form f, φ q,p φ q,p dq dp, where f, φ ∈ L 2 (R), ||φ|| L 2 (R) = 1, and the convergence of the integral is understood in the weak sense. Introducing into (10) a weight function b(q, p), called a symbol we obtain a Gabor-Toeplitz operator Gabor-Toeplitz operators generalize Fock space Toeplitz operators. The Bargmann transform provides their mutual unitary equivalence, in case the normalized Gaussian is chosen for the generating function. Books by Folland [19] and Zhu [40] are very good references on the subject. In the field of phase space analysis Gabor-Toeplitz operators were introduced by Ingrid Daubechies. Her book [6] provides a comprehensive account of the background and the initial results. It is possible to study Gabor-Toeplitz operators in a very general context, with various classes of symbols and acting on a wide range of function spaces. It is convenient to discuss Gabor multipliers and Gabor-Toeplitz operators in parallel. In this paper we restrict attention to Gabor-Toeplitz operators of localization type acting on L 2 (R), i.e. we assume that the symbol b is non-negative, bounded and integrable with respect to the Lebesgue measure on R 2 . It is straightforward to verify that under these assumptions T φ,b is non-negative, trace class, and that if |f (q, p)| 2 = b(q, p), then the composition of the operator defined by the integral kernel (9) with its conjugate equals T φ,b defined in (11). Kernels of the form (9) represent square roots of Gabor-Toeplitz operators (11). They provide a link between Gabor-Toeplitz operators and generalized convolution-product operators, where translations are substituted by actions of unitary representations.
Asymptotic properties, as R → ∞, of the symbolic calculus of Gabor-Toeplitz op- [13]. For h a continuous function defined on the closed interval [0, 1] the operator h(T φ,b R ) is defined via the spectral decomposition of T φ,b R . A Szegö type formula for operators of the form h(T φ,b R ) was obtained in [13], showing that We need to multiply h(T φ,b R ) by T φ,b R in formula (12) in order to ensure that the operator is trace class. The Bohr correspondence principle was concluded as a consequence of (12). It asserts that in the normalized limit with factor R 2 in the denominator both the distribution of the eigenvalues of and the size of their plunge region are expressed directly via the corresponding quantities of the symbol function, the distribution of b |{(q, p) | b(q, p) > δ}| , (15) and the Lebesgue measure of its plunge region In the passage to the normalized limit it is necessary to assume that the level sets {(q, p) | b(q, p) = τ }, τ = δ, δ 1 , δ 2 , have Lebesgue measure 0. We were able to deduce that asymptotically the best localization properties, i.e. asymptotically there are no eigenvalues in the open interval (0, 1), occur for symbols being characteristic functions of measurable sets Ω ⊂ R 2 .
In the next step operators T φ,Ω , with symbols b = χ Ω , were studied directly without the asymptotic limit. Two-sided estimates of the size of the eigenvalue plunge region expressed in terms of the area of a strip of fixed size R around the boundary ∂Ω and uniform with respect to generating functions φ and localization domains Ω, of the form where (∂Ω) R = {(q, p) | dist((q, p), ∂Ω)| < R}, were obtained in [7]. In order to get twosided uniform positive constants c 1 , c 2 it was necessary to assume uniform decay and non-degeneracy of the reproducing kernels obtained out of generating functions φ and uniform access to localization domains Ω and their complements Ω c from the points near their boundaries ∂Ω. The size R of the strip (∂Ω) R around ∂Ω is one of the uniform parameters controlling mutual interactions between the generating functions φ and domains of localization Ω.
The next step in the study of mutual interactions between generating functions and domains of localization from the point of view of estimating the size of the eigenvalue plunge region was accomplished in [34]. The eigenvalues of Gabor-Toeplitz lo- ) provides an exact, quantitative way of measuring the size of the eigenvalue plunge region. It has the same form as in (3) in the discrete setup. We assume that Ω is a bounded domain with C 1 boundary. Symbol σ denotes the arc length defined on ∂Ω. Function n is the Gauss map, i.e. n(r, s) is the unit normal vector at (r, s) ∈ ∂Ω directed outside Ω. By P v = {w ∈ R 2 | w · v = 0} we denote the linear subspace of R 2 consisting of vectors orthogonal to v , and U v = {w ∈ R 2 | w · v ≥ 0} is the half space inside R 2 with v being the inner normal vector. The boundary form is defined as BF (φ, Ω) = ∂Ω U n(r,s) | φ, φ q,p | 2 dist((q, p), P n(r,s) )dq dp dσ(r, s).
Under an appropriate integrability condition imposed on φ, condition Φ of the current paper is its adaptation to the discrete setup, that makes formula (19) well defined it has been shown in [34] that Formula ( 20 ) expresses the limit behavior of the projection functional in terms of the boundary form. It describes quantitatively the interactions between the boundary ∂Ω and the reproducing kernel obtained out of the generating function φ. Boundary form (4) is an adaptation of (19) to the discrete setup.
Let us fix the area A and a generating function φ for which the boundary form (19) is well defined, and let us ask for what localization domains Ω of area A the boundary form BF (φ, Ω) takes the smallest possible value. Let us introduce surface tension M φ , defined on the unit sphere of the time-frequency plane R 2 , given by the formula Function M φ , defined in (21), determines the Wulff shape For the given generating function φ let us construct K φ , defined in (22), and let us scale it with a constant c, so that the area of cK φ is A. It occurs, that, up to translation, the domain cK φ is the only domain among all domains Ω with finite perimeter and area A for which BF (Ω, φ) is minimal. The shape cK φ is the optimal domain of localization.
The principal results of the current paper deal with the discrete setup of tight Gabor frames defined on L 2 (R). On the level of convolution-product operators with kernels of the form (8) the transition to the discrete setup translates to making the range variables discrete and keeping the domain variables continuous. The multiplication parameter f is now defined on the group of integers, but the convolution parameter g is still defined on the real line R. After the modification the integral kernel (8) becomes In the discrete setup the Gabor reproducing formula (10) is substituted by a tight Gabor frame expansion and the Gabor-Toeplitz operator (11) by a Gabor multiplier (2) Gabor multipliers were introduced as a phase space analysis tool parallel to Gabor expansions. Both Gabor-Toeplitz operators and Gabor multipliers are currently very actively investigated, mostly from the point of view of their usage in phase space analysis. They were applied as phase space partitioning operators in [10] , leading to a new characterization of modulation spaces, as isomorphism maps, in [24], [25], in the context of modulation spaces and weighted Bargmann-Fock spaces, in [9], [21], [11], [5], [4] as approximation blocks for the representation of Hilbert-Schmidt, pseudodifferential, and Fourier integral operators. A recent survey [15] provides an overview of the results on both Gabor-Toeplitz operators and Gabor multipliers from the point of view of phase space localization. Not all of the phenomena are the same for Gabor-Toplitz operators and Gabor multipliers. The cut-off phenomenon happens only in the continuous setup (see [35]), the Berezin transform is invertible in a stable way only in the discrete setup (see [14]).
We restrict attention to Gabor multipliers of localization type, i.e. we assume that the symbol b defined on Λ is non-negative and summable. The operator given by the kernel a discrete analogue of (9), acting from A discrete analogue of the Szegö type formula (12) applied to G φ,b R required us to assume that the symbol function b is defined on R 2 , that it is Riemann integrable, has compact support, and that 0 ≤ b(q, p) ≤ 1. Also in the discrete setup the Szegö type formula captures essential asymptotic properties of the symbolic calculus of Gabor multipliers of localization type as . For any continuous function h defined on [0, 1] the operator h(G φ,b R ) is defined in terms of the spectral decomposition of G φ,b R . The discrete analogue of (12) was obtained in [14]. It states that The motivation for both formulae (12) and (24) came out of Widom's paper [38]. The Bohr correspondence principle, i.e. the normalized limits, the same as for the eigenvalue distribution of (13), and the size of their plunge region of (14), can be concluded for Gabor multipliers of localization type. In the discrete setup we need to insert the multiplicative constant (15) and (16), the same constant as in (24), but otherwise the normalized limits are same as in the case of Gabor-Toeplitz operators of localization type. In particular for a bounded, Riemann measurable set Ω we obtain for any ǫ > 0, where in case b R = χ RΩ we use the notation G φ,b R = G φ,RΩ . According to the principles of quantum mechanics we interpret the area |Ω| as the dimension of the space of functions localized in phase space to Ω. The constant compensates the relative redundancy of the tight frame.
Two-sided uniform estimates of the size of the eigenvalue plunge region, the discrete analog of (17), were also obtained in [14]. In the discrete context localization domains Ω are finite subsets of Λ, the strips of size R around the boundary ∂Ω occurring in (18) are substituted by and |(∂Ω) R | by the number of points of Λ inside (∂Ω) R .
Out of Theorems 1.1, 1.2 we were able to conclude Corollary 1.3, expressing the fact that from the point of view of the projection functional P F and the boundary form BF no lattice Λ is distinguished. It would be interesting to isolate those phase space phenomena that make distinction between lattices parameterizing tight Gabor frames. The book by Martinet [32] is a comprehensive resource on lattices in Euclidean spaces.
We would like to indicate intuitive reasons explaining the existence of the normalized limit of the projection functional applied to G φ,RΩ . Let F be the absolute value square of the reproducing kernel, i.e. F (ν) = | φ, φ ν | 2 . The properties of the symbolic calculus of Gabor multipliers allow us to express the normalized projection functional We observe that instead of stretching the domain R times and keeping the lattice fixed we may make the lattice R times denser and keep the domain fixed. What counts are the relative sizes. Expression ( 25 ) may be rewritten as 1 R Form ( 26 ) of ( 25 ) is more convenient for a conceptual geometric interpretation. We see that for large values of R the main contribution comes from pairs ν 1 , ν 2 , ν 1 ∈ Ω c , ν 2 ∈ Ω, which lie close together. Therefore, we expect that the resulting limit will be interpretable as an appropriate boundary form.
In the continuous case the Wulff shape (22) is the optimal localization domain. There is no analogue of it for lattice domains. We do not know how large is the class of localization domains for which the asymptotic boundary forms exist. We expect that in the general case the problem of existence of normalized limits of projection functionals has to be considered in parallel with the asymptotic properties of the counting function |Λ ∩ RΩ| as R → ∞. Harmonic analysis background on the counting problem |Λ ∩ RΩ| as R → ∞, together with the original Hlawka result, and several developments that followed afterwards are presented in Stein's book [37]. The paper by Nowak [36] discusses more recent approaches to the topic.
The major part of this work was done during the special semester at the Erwin Schrödinger Institute in Vienna devoted to Time-Frequency Analysis. The authors express many thanks to the the Schrödinger Institute, for the hospitality, and for the great creative atmosphere. They also thank Monika Dörfler for numerous insightful discussions on the subject of the current paper.

Tight Gabor Frames, their Constructions and Properties
Let H be a Hilbert space. A family of functions For any frame of H it is possible to choose the largest constant A and the smallest constant B for which (27) holds. These two extreme values of A and B are called the lower and the upper frame bounds of the frame {φ j } j∈J . If the lower and the upper frame bounds are equal then the frame is called tight. Any tight frame gives rise to a discrete reproducing formula. We may renormalize a tight frame {φ j } j∈J and obtain the norm equality which in turn via the polarization identity may be interpreted as a reproducing formula with the convergence of the sum understood in the unconditional norm sense. There is a canonical way of constructing a tight frame out of a frame. For any frame {φ j } j∈J we define a frame operator via Condition (27) guarantees that the above sum is unconditionally convergent for any f ∈ H and that the linear operator S defined in (29) is positive definite, bounded and invertible on H. It is straightforward to verify that the family S −1/2 φ j j∈J is a tight frame on H.
Gabor frames have the form , and Λ is a lattice in R 2 . We assume that the system {φ λ } λ∈Λ is a tight Gabor frame normalized in such a way that the reproducing formula (28) holds. If necessary we multiply the generating function φ by an appropriate normalization constant. The principles of quantum mechanics allow us to interpret Gabor frames as coverings of the phase space. Each frame element φ λ corresponds to a phase space block of area 1. Let A Λ denote the area of the fundamental domain of lattice Λ. If A Λ > 1 then the lattice Λ is too sparse and it is not possible to cover the phase space with blocks of area 1. There are no Gabor frames if A Λ > 1. If A Λ = 1, this is the so called critical case, it is possible to cover the phase space with blocks of area 1, but without any overlapping. It occurs that in this case a Gabor frame is automatically a Riesz basis, and a normalized tight Gabor frame is an orthonormal basis. A well known Balian-Low theorem applies to the critical case. It states that the generating function of a Gabor frame has strictly limited smoothness and localization properties. It is possible to interpret it as the outcome of the covering property with no overlapping, i.e. a partitioning of the phase space. From the point of view of modulation spaces, which require smoothness and localization, there are no Gabor frames in the critical case A Λ = 1. If A Λ < 1 then the lattice Λ is dense enough and it is possible to cover the phase space with blocks of area 1. There is also a sufficient overlap between the blocks that allows constructions of generating functions of Gabor frames with arbitrary smoothness and localization properties as measured by the scale of modulation spaces. The book by Gröchenig [20] is a comprehensive resource on all these facts. Further on we assume that A Λ < 1.
Let W s (q, p), s ≥ 0, be a weight function defined on R 2 by the formula Let S(R 2 ) denote the space of Schwartz class functions and S ′ (R 2 ) the space of tempered distributions. Let us select a non-zero function g ∈ S(R 2 ) and define the the modulation space M p,q Ws , 1 ≤ p, q ≤ ∞, as the space consisting of those tempered distributions f ∈ S ′ (R 2 ) for which the norm is finite. If p = ∞ or q = ∞ the integral norm is substituted by the essential supremum. The definition of the modulation space does not depend on the choice of function g . For different functions g the corresponding norms are equivalent. If p = q we write M p Ws instead of M p,p Ws . Let us recall that a function φ ∈ L 2 (R) satisfies condition Φ with respect to lattice Λ if and only if λ∈Λ | φ, φ λ | 2 |λ| < ∞. Condition Φ is necessary and sufficient for the boundary form BF (φ, Ω) of formula (4) to be well defined for any Λ lattice domain Ω. The next proposition formulates a sufficient condition on a function φ ∈ L 2 (R), expressed in terms of modulation spaces, for condition Φ to hold. Our sufficient condition follows by a direct application of the Young's inequality and the standard theory of modulation spaces presented in [20]. We do not include its proof.
The first construction of tight Gabor frames in dimension 1 was obtained by Daubechies, Grossmann and Meyer in 1986. Tight Gabor frames were called painless nonorthogonal expansions back then. The construction produced generating functions with compact support either in position or in momentum and with arbitrary smoothness measured by the number of continuous derivatives, any C k or C ∞ was possible. The book by Daubechies [6] provides an excellent account of the initial stages of the constructions of tight Gabor frames. Then came Wexler-Raz biorthogonality relations, Walnut, Janssen representations, and Ron-Shen duality principle dealing with Gabor frames and the frame operator in any finite dimension. All these are very nicely presented in Gröchenig's book [20]. Tight Gabor frames and in particular canonical tight Gabor frames obtained via the action of S −1/2 , where S is the frame operator, are of the principal interest from the point of view of this paper. Generating functions being members of the modulation space M 1 W , where W is a subexponential weight, are the building blocks for all other modulation spaces. Subexponential weights are a natural generalization of the standard weights W s defined in (30) (see [22] for the definition, provided on page 10, and for explanations of their usage in time-frequency analysis). It is important to know how to construct generating functions of tight Gabor frames, which are members of M 1 W . The major result in this direction was obtained by Gröchenig and Leinert in [22], where they proved that, for any lattice Λ ⊂ R 2n , the canonical tight frame operator S −1/2 is bounded on M 1 W . If we have a Gabor frame with the generating function in M 1 W , then we also have a tight Gabor frame with the generating function in M 1 W . The existence of a generating function in M 1 W of a Gabor frame, for any lattice Λ ⊂ R 2n , with the volume of the fundamental domain satisfying Vol Λ < 1, was established recently by Luef [31]. Feichtinger and Kaiblinger examined, from the point of view modulation spaces, continuity properties of the canonical dual generating function, i.e.g = Sg , where S is the frame operator, with respect to the perturbations of the lattice. They proved in [12] that the set Gabor frame} is open and that the map (φ, A) →φ is continuous from F M 1 W into M 1 W . This result was recently extended to canonical tight generating functionsφ = S −1/2 (φ) by Leinert and Luef [31].

Proofs of the Main Results
We start by recalling the definitions of our principal objects of interest. A tight Gabor frame parametrized by a lattice Λ ⊂ R 2 , and normalized in such a way that the reproducing formula (1) holds, is given. A Gabor multiplier of localization type (2) is constructed out of the reproducing system {φ λ } λ∈Λ , and a non-negative, summable, and bounded above by 1 symbol b, defined on Λ. It is defined by the formula The projection functional (3) applied to G φ,b has the form are the eigenvalues of G φ,b . Condition Φ, the additional requirement imposed on the generating function φ, λ∈Λ | φ, φ λ | 2 |λ| < ∞, is a necessary and a sufficient condition for the boundary form (4) to be well defined for any Λ localization domain Ω, where l i , i = 1, ..., N are the line segments constituting the boundary of Ω, n i is the unit vector orthogonal to l i directed outside Ω, Now we are ready to present the proof of Theorem 1.1. Proof. The transformation rule (ii) of Lemma 4.1, describing the effect of the conjugation by the projective metalplectic representation, shows that (i), i.e. formula (5) In the remaining part of the proof we deal with (ii), i.e. with formula (6) BF (φ, Ω) = BF (µ(A)φ, A(Ω)).
The group SL(2, R) acts transitively on the collection of all lattices Γ ⊂ R 2 satisfying condition A Γ = A Λ . Therefore, we may assume that A(Λ) = aZ × bZ, where a, b > 0 and ab = A Λ . It is clear that for any A ∈ SL(2, R) A(Ω) is an A(Λ) domain. The proof of the BF invariance rule (6), describing the transition from Λ to aZ × bZ via A, requires two ingredients, the transformation rule for the reproducing kernel and for the geometric atoms where λ ∈ Λ, s is a line segment with its endpoints being the points of lattice Λ, constituting a portion of the boundary of Ω, l is the line parallel to s, and passing through (0, 0). These two, (32) and (33), constitute the basic building blocks of the boundary form. Lemma 4.1 (i) provides the formula describing the transformation of the reproducing kernel (32) under the action of the projective metaplectic representation, it is enough to apply it. However, we need to deal with the geometric atoms (33), via a direct computation.
The linear map A ∈ SL(2, R) providing the transition from Λ to aZ × bZ satisfies condition det(A) = 1, therefore it preserves the orientation of the boundary. It transforms a line segment l i of the boundary of Ω to the corresponding line segment A(l i ) of A(Ω), and the half lattice U i of Λ to the corresponding half lattice A(U i ) of A(Λ). The fact that A might not preserve orthogonality does not matter. Lemma 4.1 (i) allows us to conclude that the transformation of the boundary form BF is correct as far as the reproducing kernel (32) is concerned. It also shows that condition Φ transforms properly. We need to show that the geometric atoms a s,λ of (33) also transform properly, i.e. that a s,λ = length(As) · dist(γ, Al), where γ = Aλ. The Iwasawa decomposition of SL(2, R) (see [30]) allows us to represent A as P K , where P is upper triangular and K is a rotation matrix, both in SL(2, R). Since K is unitary it preserves distances and angles. It is clear that the geometric atoms properly transform under K . It is therefore enough to consider the upper triangular P . In order to show that (34) holds, we consider R, the inverse of P , providing a transition from aZ × bZ to Λ. We verify that length(Rs an,bm ) · dist(R(ak, bl), Rl an,bm ) = ab|nl − km|, where R = d e 0 d −1 , d > 0, e ∈ R, s an,bm is the line segment connecting points (0, 0) and (an, bm), l an,bm is the line containing s an,bm , and n, m, k, l ∈ Z. Translation invariance of the Euclidean length allows us to assume that the line segment under consideration, constituting a portion of the boundary of the domain of localization Ω, starts at the origin (0, 0). We observe that R(an, bm) = (dan + ebm, bmd −1 ), length 2 (Rs an,bm ) = (dan + ebm) 2 + (bmd −1 ) 2 , the normalized vector v orthogonal to Rl an,bm equals v = (dan + ebm) 2 + (bmd −1 ) 2 −1/2 (−bmd −1 , dan + ebm), therefore we obtain length 2 (Rs an,bm )dist 2 (R(ak, bl), Rl an,bm ) = = (dak + ebl, bld −1 ), (−bmd −1 , dan + ebm) The above calculation verifies (35). It also shows (34), since plugging R = I , i.e. taking d = 1, e = 0 produces the geometric atom on aZ × bZ.
Let us recall that Gabor multipliers with symbols of the form b = χ Ω are called Gabor localization operators and that they are denoted G φ,Ω . The proof of Theorem 1.2, the principal result of the current paper, is very lengthy and it makes use of all the auxiliary facts collected in Section 4. In order to facilitate the reading, we list the steps of the proof first, and then we present their proofs. Theorem 1.2. Let φ be a generating function of a tight Gabor frame parametrized by lattice Λ and satisfying condition Φ, and Ω a Λ domain contained in R 2 . Then Steps of Proof: Step 1. We make a transition from lattice Λ to Z 2 , where the computation is easier to handle. We construct a matrix A ∈ SL(2, R) and scaling parameters a, b > 0, such that A Λ = ab and A(Λ) = S a,b (Z 2 ), where S a,b is the scaling matrix with numbers a, b on its diagonal and zeros elsewhere. We will work with the Z 2 domain S −1 a,b (A(Ω)) instead of the Λ domain Ω and with the generating function µ(A)φ of the tight Gabor frame {(µ(A)φ) γ } γ∈S a,b (Z 2 ) instead of the generating function φ of the tight Gabor frame {φ λ } λ∈Λ . Symbol µ denotes the projective metaplectic representation defined on SL(2, R). We substitute the Gabor multiplier G φ,Ω by the operator L µ(A)φ,S −1 a,b (A(Ω)) , which is unitary equivalent to it, and defined by the formula Once the unitary equivalence is verified it is clear that We substitute the boundary form BF (φ, Ω) defined in (4) by the boundary form SF (µ(A)φ, S −1 a,b (A(Ω))) defined in terms of the Z 2 domain S −1 a,b (A(Ω))) and the image of the original generating function φ of the tight Gabor frame {φ λ } λ∈Λ under µ(A), where l i , i = 1, ..., N are the line segments constituting the boundary of S −1 a,b (A(Ω)), n i is the unit vector orthogonal to l i directed outside S −1 a,b (A(Ω)), P i = {w ∈ R 2 | w · n i = 0}, U i = {ν ∈ Z 2 | ν · n i ≥ 0}. We show that in view of condition Φ the form (38) is well defined and that Step 2. We use tight Gabor frames versions of Toeplitz and Hankel operators in order to express the projection functional as the square of the Hilbert-Schmidt norm of the matrix where ν 1 , ν 2 ∈ Z 2 . As the result we obtain the equality Next, we change variables and let them range over the dilated lattice 1 R Z 2 . The right hand side of ( 41 ) becomes 1 R Step 3. We split the boundary of S −1 a,b (A(Ω)) into its component lattice cycles, then into their individual line segments, and then we reduce the computation of the limit to the sum of the limits over the line segments constituting the boundary. We may assume that each boundary segment connects lattice points and does not have lattice points in its interior. If necessary we divide it into subsegments. Each sufficiently small neighborhood of a boundary segment gets represented as an unbounded vertical strip domain, for which expression ( 43 ) becomes and function H representing a given segment of the boundary is a linear function with rational slope of the form H(x) = m n x, with n ≥ m, m ≥ 0, n > 0, and m, n relatively prime. The graph of H constitutes the top portion of the boundary of the unbounded vertical strip domain {(x, y) ∈ R 2 | 0 ≤ x < n, y ≤ H(x)}. Points x 1 R , y 1 R satisfying condition y 1 R > H x 1 R lie above the graph of H , and points x 2 R , y 2 R for which y 2 R ≤ H x 2 R are located below or on the graph of H . Function F is obtained out of | µ(A)φ, (µ(A)φ) S a,b (ν) | 2 via a lattice transformation of Lemma 4.5 that brings the selected fragment of the boundary to the form described above.
Step 4. We adjust the form of variables x 1 , x 2 to the arithmetic form of the slope of H . We represent x 1 , x 2 as x 1 = nk 1 + r 1 , x 2 = nk 2 + r 2 , where k 1 , k 2 are integers and r 1 , r 2 ∈ {0, 1, 2, ..., n − 1}. With new variables k 1 , r 1 , k 2 , r 2 expression ( 44 ) becomes Step 5. Invariance of expression ( 45 ) with respect to variables y 1 , y 2 allows us to substitute the double summation with a single summation. We count the number of repetitions in the representation y = y 1 − y 2 and we place an appropriate factor that compensates them. Expression ( 45 ) becomes where square brackets denote the integer part of a rational number.
Step 6. Form ( 46 ) is convenient for making the passage to the limit. It occurs that taking the limit in ( 46 ) with respect to R is just the same as performing summation with respect to k . We obtain n−1 Step 7. We write expression ( 47 ) in the form involving function R t (s) where Step 8. We interpret expression ( 48 ) geometrically. We make use of the structural features of lattice lines and we show that in fact ( 48 ) equals where G = {(x, H(x)) | x ∈ R} is the graph of H , U is the part of Z 2 lying above G and dist is the Euclidean distance inside the plane R 2 containing lattice Z 2 . Constant √ n 2 + m 2 represents the contribution of vertical strip domain {(x, y) ∈ R 2 | 0 ≤ x < n, y ≤ H(x)} to the total length of the boundary of S −1 a,b (A(Ω)).
Step 9. We put together the boundary forms (50) of vertical strip domains corresponding to all line segments of the boundary of S −1 a,b (A(Ω)) and we obtain In view of (37), and (39), formula (51) concludes the proof.

Proofs of Steps:
Proof of Step 1. We construct matrix A ∈ SL(2, R), satisfying property A(Λ) = aZ × bZ, a, b > 0, ab = A Λ , by assigning values a(0, 1), b(1, 0) to the generators of Λ and then extending the assignment by linearity. The inverse of the scaling matrix S a,b provides a transition from aZ × bZ to Z 2 , S a,b (Z 2 ) = aZ × bZ. Matrices A and S a,b are the arithmetic tools needed for the transition from Λ to Z 2 . The unitary operator µ(A), where µ is the projective metaplectic representation defined on SL(2, R), is the analytic tool responsible for the transition needed on the level of the generating functions of tight Gabor frames. We observe that in view of Lemma 4.1 (ii) and definition (36) . We conclude that (37) holds. The transition from Λ to Z 2 is justified as far the operator properties, i.e. the values of the projection functional, are concerned.
The justification of the transition from Λ to Z 2 as far as the geometric properties, i.e. the boundary form, are concerned, follows from Theorem 1.1 (ii). We obtain BF (φ, Ω) = BF (µ(A)φ, A(Ω)), but we still need to switch to SF (µ(A)φ, S −1 a,b (A(Ω))) defined in (38). In order to do that, we observe that the boundary form scales with respect to the action of S a,b via the area factor ab, and this allows us to finish the proof of (39) Proof of Step 2. Since where M R (ν 1 , ν 2 ) is defined in (40).

Proof of
Step 3. This is the most tedious step of the proof. For the sake of notational convenience we will use symbol Ω for the current domain under consideration. It will be clear from the context what it is at a given stage of the proof. Initially Ω denotes S −1 a,b (A(Ω)). Our first target is to cut the kernel into pieces with the help of a partition of unity of R 2 obtained out of quadrilaterals formed around the boundary segments constituting ∂Ω and two open sets isolated from the boundary and representing the exterior and the interior of Ω. We traverse each component of the boundary of Ω according to the orientation, keeping the interior on the right and the exterior on the left, and at each initial lattice point of the boundary segment l i we place a sufficiently small rational line segment s i with its middle being the initial lattice point. We chose s i in such a way that it is transversal to both l i and the boundary segment l l preceding it, and that both exterior and interior angles with l i and l l are smaller than π . We assume that segments s i , i = 1, ..., N are so small so that they intersect the boundary ∂Ω exactly at one point. We assume that they are positioned in such a way that one of the endpoints of s i is inside the interior of Ω and the other in the exterior of Ω. Let l r be the segment following l i . We form quadrilaterals W i out of consecutive segments s i , s r attached at the beginning and at the end of l i , and the segments joining their endpoints, both endpoints inside the interior of Ω or both in the exterior of Ω. We also assume that the sizes of the transversal segments are so small so that the segments joining the endpoints of s i and s r are contained either in the exterior or in the interior of Ω. We include s i inside W i , but not s r . Adding s r to W i we would obtain a closed quadrilateral, but we need to form a family of pairwise disjoint sets, and s r is already included in W r .
. We observe that We do not need terms with factors m I (ν 1 ) and m E (ν 2 ) in the above decomposition, since U I ∩ Ω c = ∅ and U E ∩ Ω = ∅. We know that dist(U I , Ω c ) > 0, dist(U E , Ω) > 0, therefore Lemma 4.6 allows us to conclude for J = II, III, IV . Since dist(W k , W l ) > 0 for k = l such that l k , l l are not neighboring line segments, again by Lemma 4.6 we obtain If k = l , but l k , l l are neighboring line segments, then Lemma 4.7 applies and we conclude again contribute to the limit.
Let us recall that we assumed that each boundary segment l k connects lattice points and does not have lattice points in its interior. We may apply Lemma 4.5 in order to bring each segment l k to the form (x, H(x)), with 0 ≤ x < n, H(x) = m n x, n > 0, m ≥ 0, n ≥ m, n, m relatively prime, with the image of Ω placed below the graph of H , and the image of Ω c placed above the graph of H . Under the transformation of Lemma 4.5 quadrilateral W k becomesW k . With the help of Lemma 4.7 we bring W k to a form of a bounded vertical strip domain with the top portion of the boundary represented by a line t k and the bottom portion by a line b k . Lemma 4.6 allows us to substitute the bounded vertical strip domain we have just obtained by a rectangle with the top boundary represented by a horizontal line y = r , with r ∈ Z, placed above t k , and the bottom boundary represented by a horizontal line y = s, with s ∈ Z, placed below b k . Figure 5. Illustrates the usage of quadrilateralW k .
In the last step we switch to the unbounded strip domain with its complement We have the splitting We need to show that the normalized sums coming out of terms (57), (58), (59) have zero limits. Lemma 4.6 applies to both (57), (58), but (59) needs to be treated separately. After the change of variables the normalized sum of (59) becomes In the proof that its limit is zero as R → ∞ we may assume that R is an integer. Let u = min{r, −s} and let * denote the convolution on Z. We have 1 R 0≤x 1 ,x 2 <Rn y 1 >rR Proof of Step 4. With the new representation of x 1 , x 2 the summation done in (44) becomes (45). It is enough to perform substitutions and then simplify the resulting expressions.
Proof of Step 6. We move summations over k 1 , k 2 in (46) inside and summations over r 1 , r 2 , i outside. Our target is to identify convolution kernels defined in terms of variables k 1 , k 2 and then, keeping variables r 1 , r 2 , i fixed, take the limit with respect to R. For a rational number q by ⌈q⌉ we denote the smallest integer larger or equal to q . We know that 0 ≤ nk i + r i R < n, i = 1, 2.
We observe that k i ranges over the integer interval 0, ⌈R − r i n ⌉ − 1 , i = 1, 2.
We define convolution kernels K r 1 ,r 2 ,i as tends to k K r 1 ,r 2 ,i (k). We proved that (47) is the limit of (46).
Proof of Step 8. We fix summation variables k, t of expression (48) and we consider the effect of the summation done with respect to s, i. By Lemma 4.3 we know that function R t (s) takes two values m n t + 1 and m n t + 1 + 1, therefore we conclude that points (kn + t, km + R t (s) + i), s = 0, 1, ..., n − 1, i = 0, 1, 2, ..., represent a discrete vertical half line consisting of lattice points of Λ starting directly above the graph G. Lemma 4.3 also tells us that value m n t + 1 is taken n m n t + 1 − m n t times and value m n t + 1 + 1 the remaining n − n m n t + 1 − m n t times. The first point of the discrete half line, located right above the graph G, corresponds to parameter values i = 0, R t (s) = m n t + 1 , and it is repeated inside formula (48) n m n t + 1 − m n t times. The j th point of it, j ≥ 2, with counting done upwards, corresponds to values i = j − 1, R t (s) = m n t + 1 and i = j − 2, R t (s) = m n t + 1 + 1, and in (48) it comes with multiplicity n j − 1 + m n t + 1 − m n t . Let d j , v j represent the Euclidean and the vertical distances from the j th point, j ≥ 1 of the discrete half line to the graph G. Similarity relation of the triangle representing distances d i , v i with the triangle with vertices (0, 0), (n, 0), (n, m) gives Since the multiplicity inside (48) of the j th point of the discrete half line equals nv j , and variables k, t parameterize all discrete vertical half lines of U we conclude that (48) may be expressed as (50). Proof. It is enough to take A ∈ SL(2, R) transferring Λ onto aZ × bZ,φ = µ(A)φ, Ω = A(Ω), and apply Theorem 1.1. The existence and the form of the limits follows from Theorem 1.2.

Auxiliary Facts and their Proofs
Transformation properties with respect to the metaplectic representation.
A comprehensive presentation of the metaplectic representation from the point of view of phase space analysis is contained in Folland's book [19]. The book by Lang [30] is an extensive reference on SL(2, R). The Heisenberg group H 1 is the group obtained by defining on R 3 the product where z, w ∈ R 2 , t, s ∈ R and ω is the symplectic form defined on R 2 , i.e.
The Schrödinger representation of the group H 1 , acting on L 2 (R), is then defined by ρ(x, ξ, t)f (y) = e 2πit e 2πiξy f (y − x).
We write z = (x, ξ) when we separate the position component x from the momentum component ξ of a point z of the phase space R 2 . The group SL(2, R), consisting of 2 × 2 matrices with real entries and determinant 1, acts on H 1 via automorphisms that leave the center {(0, t) | t ∈ R} of H 1 pointwise fixed, For any fixed A ∈ SL(2, R) there is a unitary representation of H 1 , acting on L 2 (R), defined as the composition ρ A (z, t) = ρ(A · (z, t)), with its restriction to the center of H 1 being a multiple of the identity. By the Stonevon Neumann theorem ρ A is unitary equivalent to ρ, i.e. there is an intertwining unitary operator µ(A) acting on L 2 (R) such that for all (z, t) ∈ H 1 By Schur's lemma, µ is determined up to a phase factor e is , s ∈ R. It turns out that the phase ambiguity is really a sign, so that µ lifts to a representation of the double cover of the group SL(2, R). The constructed representation of the double cover of SL(2, R) is called the metaplectic representation.
The representations ρ and µ can be combined and give rise to the extended metaplectic representation µ e , the composition of ρ, defined on H 1 , with µ, defined on the double cover of SL(2, R). From the point of view of the interpretation as a phase space action, the phase factors do not matter, therefore we remove them and treat µ as a projective representation of SL(2, R), and µ e as a projective representation of the semidirect product R 2 ⋊ SL(2, R) with the group law (z, A) · (w, B) = (z + Aw, AB) . (60) The extended metaplectic representation provides all affine transformations of the phase space R 2 . For (z, A) ∈ R 2 ⋊ SL(2, R) the unitary operator µ e (z, A), defined up to a phase factor, expresses the analytic action on L 2 (R). The affine geometric action on R 2 is expressed by the law The extended metaplectic representation is a convenient setup for performing computations involving compositions of the Schrödinger and the metaplectic representations.
Both Gabor multipliers (2) and lattice boundary forms (4) have natural transformation properties with respect to the projective metataplectic representation. These properties are important ingredient of our proofs. We will deduce them out of the the fundamental lemma formulated below.
Lemma 4.1. Let Λ ⊂ R 2 be a lattice, {φ λ } λ∈Λ a tight Gabor frame with the generating function φ, A ∈ SL(2, R), and b ∈ l ∞ (Λ). Then (i) Γ = AΛ ⊂ R 2 is a lattice, {(µ(A)φ) γ } γ∈Γ a tight Gabor frame with the generating function µ(A)φ, and Proof. Clearly Γ = AΛ is a lattice, since A is linear and invertible. Group law (60) and affine action rule (61) allow us to identify the phase space by substituting γ = Aλ we obtain The above calculation shows that {(µ(A)φ) γ } γ∈Γ is a tight Gabor frame, it is enough to take the constant function equal to 1 for b. It also verifies formula (63). Formula (62) follows by a similar calculation, Symbolic calculus of Gabor multipliers. Let us assume that Λ ⊂ R 2 is a lattice, and {φ λ } λ∈Λ a tight Gabor frame defined on it. Let us define the mapping W φ : L 2 (R) → l 2 (Λ) by the formula Tight frame properties of {φ λ } λ∈Λ imply that W φ is an isometry, and that the operator P φ : l 2 (Λ) → l 2 (Λ) defined as is the orthogonal projection onto the range of W φ . Gabor multiplier G φ,b is parametrized by the generating function φ of a tight Gabor frame {φ λ } λ∈Λ and a symbol b ∈ l ∞ (Λ). Let us recall that it is a bounded operator acting on L 2 (R) defined as It is convenient to describe Gabor multiplier G φ,b in terms of the Toeplitz operator G φ,b = P φ M b P φ acting on l 2 (Λ), M b denotes the operator of multiplication by b.
The isometry W φ allows us to identify G φ,b with the upper left corner of the matrix representation of the operator G φ,b with respect to the orthogonal decomposition l 2 (Λ) = W φ (L 2 (R)) ⊕ W φ (L 2 (R)) ⊥ . Basic properties of the symbolic calculus of Gabor multipliers can be deduced out of the properties of the symbolic calculus of Toeplitz operators. Hankel operator H φ,b = (I − P φ )M b P φ acting on l 2 (Λ) measures to what degree the mapping b → G φ,b fails to be a homomorphism. The algebraic formula expresses this relationship quantitatively and it is the main conceptual ingredient of the argument that allows us to write down the projection functional as the square of the Hilbert-Schmidt norm the matrix where ν 1 , ν 2 ∈ Λ.
Lemma 4.2. Let {φ λ } λ∈Λ be a tight Gabor frame and Ω ⊂ Λ a finite set. Then Proof. The non-zero eigenvalues of the localization operator G φ,Ω coincide with the non-zero eigenvalues of the Toeplitz operator G φ,Ω = P φ M χ Ω P φ and the non-zero eigenvalues of the operator R φ,Ω = M χ Ω P φ M χ Ω . The first fact follows from the identification of G φ,Ω with the upper left corner of the matrix representation of G φ,Ω with respect to the decomposition of l 2 (Λ) into the range of W φ and its orthogonal complement. The second fact follows since for a compact operator T the non-zero eigenvalues of T T * and T * T are the same. We observe that P F (G φ,Ω ) = P F (G φ,Ω ) = tr (G φ,Ω (I − G φ,Ω )) = tr (R φ,Ω (I − R φ,Ω )) and that Therefore we obtain and this finishes the proof.
Lattice slopes of rational lines. For an integer t we define the t-slope of the line y = ax + b at an integer argument s as where U(x) is the smallest integer larger than x, and l(x) is the largest integer smaller or equal to x. The t-slope at s is simply the smallest difference between integer values above the graph at t + s and below or on the graph at s. We interpret it as the lattice slope corresponding to making t-steps to the right of s. We do not normalize the t-slope, i.e. we do not divide it by the number of steps. As we have already seen function R t defined in (49) is the principal analytic component of the boundary form. It occurs that it may be interpreted as the t-slope of the rational line y = m n x. Indeed, let S t (s) be the t-slope of the line y = m n x at s. Direct inspection shows, that values R t (s) and S t (s) coincide for s ∈ {0, 1, 2, ..., n − 1}. Function S t is periodic with period n and we may regard it as defined on the cyclic group Z n .
Our primary geometric concern are the values of lattice slopes and the frequencies with which they occur. Observe that R 0 (s) = 1 for all s ∈ {0, 1, 2, ..., n − 1}. If n = 1, then the only possible value of t is 0 and again 1 is the only value of R t . If however n ≥ 2 and t = 0, then R t takes precisely two values. The next lemma describes those values and the frequencies with which they occur. Lemma 4.3. Let m, n be a pair of relatively prime integers. Assume that n ≥ 2. Let t ∈ {0, 1, 2, ..., n − 1} be a fixed number. Function R t (s) defined for arguments s ∈ {0, 1, 2, ..., n − 1} by formula (49) takes two distinct values m n t + 1 and m n t + 1 + 1. Value m n t + 1 is taken n m n t + 1 − m n t times, and value m n t + 1 + 1 is taken the remaining n − n m n t + 1 − m n t times.
Proof. We know that R t and S t are equal. It is therefore enough to prove Lemma 4.3 with S t instead of R t . Let s ∈ {0, 1, 2, ..., n − 1}. Line y = m n x crosses vertical lines x = s at points of the form s, l + r n , where l is an integer and r ∈ {0, 1, 2, ..., n − 1}. Numbers m, n are relatively prime, therefore each value of r occurs precisely once for an appropriate value of s ∈ {0, 1, 2, ..., n − 1}. Observe that the set of values S t (s), s ∈ {0, 1, 2, ..., n − 1} is the same as the set of t-slopes at 0 of the lines y = m n x + r n , r ∈ {0, 1, 2, ..., n − 1}. Indeed an integer shift of coordinates allows us to view each segment starting at s, m n s and ending at t + s, m n (t + s) as a segment starting at 0, r n and ending at t, m n t + r n for an appropriate value of r . Line y = m n x crosses the vertical line x = t at a point l + u n with l an integer and u ∈ {1, 2, ..., n − 1}. All lines y = m n x + r n with r ∈ {0, 1, 2, ..., n − u − 1} have the same t-slope at 0 as the line y = m n x. The t-slope jumps up by 1 for r = n − u and keeps this value for all all the remaining r ∈ {n − u, n − u + 1, ..., n − 1}. We conclude that function S t takes two values m n t + 1 and m n t + 1 + 1. The first value is taken n − u times and the second value u times. We need to verify that n − u = n m n t + 1 − m n t . Indeed, m n t = l + u n , therefore mt = nl +u, m n t + 1 = l +1 and n m n t + 1 − m n t = n(l +1)−mt = n−u. Approximation to the identity by Fejér's kernel. In the lemma that follows we quote a well known approximation to the identity property of the Fejér's kernel. We translate the original property from the group of one dimensional torus to the group of integers. For f, g ∈ l 1 (Z) by f * g we denote the convolution of f and g , defined as where ·, · stands for the inner product of l 2 (Z).
Proof. The proof of the first part, the formula for the coordinate change is a straightforward computation which makes use of the fact that τ −1 = (−h −1 (µ), h −1 ). The proof of the second part follows the process of inspecting the list of all possible positions of the segment l , which takes into account the placement of Ω and Ω c relative to l , and indicating in each case the coordinate change τ needed to accomplish the target. It is also a direct computation.
Asymptotic limits of the restricted kernel K R . The following two lemmas are the main technical tools behind the reduction process of general lattice domains to vertical strip domains. In the first lemma we deal with separated supports of variables ν 1 , ν 2 . The second lemma is more delicate, it treats the case of variables ν 1 , ν 2 restricted to bounded cones located outside and inside Ω, with their sides being segments of rational lines, and their common vertex being a lattice point of the boundary of Ω. Lemma 4.6. Suppose that there is δ > 0 such that for all sufficiently large R nonnegative kernel G R (ν 1 , ν 2 ), ν 1 , ν 2 ∈ 1 R Z 2 satisfies G R (ν 1 , ν 2 ) ≤ K R (ν 1 , ν 2 ), and G R (ν 1 , ν 2 ) = 0 for |ν 1 − ν 2 | < δ.
Proof. We observe that in view of Lemma 4.6 we may substitute cones C 1 , C 2 by restricted cones C δ is the Euclidean disk with radius δ and center v , the common vertex of C 1 , C 2 . We may take the radius δ arbitrarily small. We choose a lattice line l passing through v and separating C 1 and C 2 . Lemma 4.5 allows us to assume that line l has the form H(x) = m n x, m ≥ 0, n > 0, m ≤ n, gcd(m, n) = 1, that the origin is the common vertex of C 1 , C 2 , and that C 1 lies above l and C 2 below l .
This concludes the proof since constants M,σ depend only on cones C 1 , C 2 , condition Φ guaranties that x,y∈ZF 1 σ (x, y) (y + 1) < ∞, and we are allowed to take δ arbitrarily small.