Multiple Solutions for a Class of Concave-Convex Quasilinear Elliptic Systems with Nonlinear Boundary Condition ()
1. Introduction
In this article, we are interested in the existence of two nontrivial nonnegative solutions of the following problem:

(1.1)
where
is a bounded domain with smooth boundary,
is the critical Sobolev exponent for the embedding
.

is the outer normal derivative,
, the weight m(x) is a positive bounded function and
are smooth functions which may change sign in Ω. By Nehari manifold, fibering method and analytic techniques, the existence of multiple positive solutions to this equation is verified.
In recent years, there have been many papers concerned with the existence and multiplicity of positive solutions for semilinear elliptic problems. Some interesting results can be found in Garcia-Azorero et al. [1], Wu [2-4] and the references therein. More recently, Hsu [5] has considered the following elliptic system:
(1.2)
By variational methods, he proved that problem (1.2) has at least two positive solutions if the pair of the parameters
belongs to a certain subset of
. However, as far as we know, there are few results of problem (1.1) in addition to concave-convex nonlinearities, i.e.,
, including nonlinear boundary condition. We focus on the existence of at least two nontrivial nonnegative solutions for problems (1.1) in the present paper.
Set
(1.3)
where
satisfy
(1.4)
The main result of this paper is summarized in the following theorem.
Theorem 1.1. If the parameters
satisfy

then problem (1.1) has at least two solutions
and
satisfy
in
and 
It should be mentioned that the similar results about the existence of multiplicity of positive solutions for the Laplace problem with critical growth and sublinear perturbation have been discussed in the recent paper [6-8] and the reference therein.
This paper is organized as follows. Some preliminaries and properties of the Nehair manifold are established in Sections 2, and Theorems 1.1 is proved in Sections 3.
2. Preliminaries
Let
denotes the usual Sobolev space. In the Banach space
we introduce the norm which is equivalent to the standard one:

First, we give the definition of the weak solution of (1.1).
Definition 2.1. We say that
is a weak solution to (1.1) if for all
, we have


It is clear that problem (1.1) has a variational structure. Let
be the corresponding energy functional of problem (1.1), and it is defined by

where


It is not difficult to verify that the functional I is not bounded neither from below nor from above. So it is convenient to consider I restricted to a natural constraint, the Nehari manifold, that contains all the critical points of I. First we introduce the following notation: for any functional
we denote by
the Gateaux derivative of F at
in the direction of
and


Define the Nehari manifold
. Note that N contains all solutions of (1.1) and
if and only if
(2.1)
Lemma 2.1.
is coercive and bounded below on N.
Proof. Suppose
From (2.1), the Holder inequality and the Sobolev embedding theorem, it follows that
(2.2)
Thus
is coercive and bounded below on
since
Define
Then for all
we have
(2.3)
Arguing as that in [9,10], we split
into three parts:



Lemma 2.2. Suppose
is a local minimizer of
on
and
Then
in 
Proof. If
is a local minimizer for I on N, then
is a solution of the optimization problem minimize
subject to

Hence, by the theory of Lagrange multipliers, there exists
such that
in
.
Here
is the dual space of the Sobolev space
. Thus,

But
since
Hence 
Lemma 2.3.
for all

Proof. We argue by contradiction. Suppose that for all
there is
then (2.3) and the Sobolev embedding theorem imply that
(2.4)
and

(2.5)
Thus from (2.4), (2.5) we have
(2.6)
and

Consequently,

which is a contradiction.
By Lemma 2.3, we can write
for all

Define
.
Lemma 2.4. (i)
for all

(ii) There exists a positive constant d0 depending on
such that
for all

Proof. (i) Suppose
, then we have

for 
Thus we get that 
(ii) Suppose

and
. Then (2.4) implies that
(2.7)
and (2.5) implies that
(2.8)
From (2.7) and (2.8) it follows that


which shows that

since 
where
is a positive constant.
For all
such that
, set

Lemma 2.5. Suppose that

and
is a function satisfying 
(i) If
, then there exists a unique 
such that
and
.
(ii) If
, then there exist
and
such that

Furthermore, 

Proof. Fix
with
For all
, let

then it is obvious that Ψ(0) = 0, Ψ(t) → −∞ as t → +∞,
as
small enough. So we can deduce that Ψ′(t) = 0 at
for
,
for
Then Ψ(t) that achieves its maximum at
is increasing for
and decreasing for
Moreover,

(i) If
, then there exists a unique
such that
Note that

thus we get 
From

we have
. For all
it follows that



So we get that 
(ii) If
for


then there exist
and
such that
and
By the similar argument in (i), we get
and

for 
for 
Then it follows that


The proof of this Lemma is completed.
For each
with
, we write
(2.9)
Lemma 2.6. Suppose that

and
is a function satisfying
.
(i) If
then there exists a unique
such that
and

(ii) If
, then there exist
and
such that
and
. Furthermore,


Proof. Fix
with
For all
let
(2.10)
then it is obvious that
. So we can deduce that
at 

for
.
Then
that achieves its maximum at
is increasing for
and decreasing for
Using the similar argument in Lemma 2.5, we can obtain the result of Lemma 2.6.
3. Proof of Theorem 1.1
Lemma 3.1. Suppose that

then the functional
has a minimizer
and it satisfies
(i) 
(ii)
is a nontrivial solution of (1.1).
Proof. Let
be a minimizing sequence such that
(3.1)
Since I is coercive on N, we get that
is bounded on
. Passing to a subsequence (still denoted by
), there exists
such that
weakly in
,
a.e. in
, (3.2)
strongly in
and in
. This implies

Since
, we get

By Lemma 2.4 (i) we get
and then
. Now we prove that 
strongly in
Suppose otherwise, then either
(3.3)
Fix
with
. Let
, where
is as in (2.10).
Clearly,
as
, and
as
Since
by an argument similar to the one in the proof of Lemma 2.6, we have that the function
achieves its maximum at
, is increasing for
and decreasing for
where
is as in (2.9). Since
by Lemma 2.6, there is unique
such that

Then
(3.4)
By (3.3) and (3.4), we obtain
for n sufficiently large for the sequence
Since
we have
Moreover,

and
is increasing for
. This implies
for all
and
sufficiently large. We obtain
. But 
and
this implies

which is a contradiction. Hence
strongly in W. This implies
as
. Thus
is a minimizer for
on
Since
and
, by Lemma 2.2 we may assume that
is a nontrivial nonnegative solution of Equation (1.1).
Next we prove
Arguing by contradiction, without loss of generality, we may assume that v ≡ 0. Then as u is a nonzero solution of
(3.5)
we have
(3.6)
Choose
such that
(3.7)
then

By Lemma 2.6, there is a unique
such that
. Moreover, from (3.6) and (3.7), it follows that

and

This implies

which contradict with that (u,0) is the minimizer and hence
. So
is a nontrivial nonnegative solution of Equation (1.1).
Lemma 3.2. Suppose that

Then the functional
has a minimizer
and it satisfies
(i) 
(ii)
is a nontrivial solution of (1.1).
Proof. Let
be a minimizing sequence such that
(3.8)
Since I is coercive on N, we get that
is bounded on
. Passing to a subsequence (still denoted by
), there exists
such that
weakly in
,
a.e. in
, (3.9)
strongly in
and in 
This implies

Moreover, by (2.3) we obtain
then
Now we prove that
strongly in W. Suppose otherwise, then either
(3.10)
By Lemma 2.6, there is unique to such that
Since
for all
, we have

and this is a contradiction. Hence 
strongly in W. This implies 
as
. Thus
is a minimizer for I on
.
Since
and
, by Lemma 2.4 and the similar argument as that in Lemma 3.1 we can get
is also a nontrivial nonnegative solution of Equation (1.1).
Proof of Theorem 1.1. From Lemma 3.1 and Lemma 3.2, we obtain that Equation (1.1) has two nontrivial nonnegative solutions
and
satisfy
and
. It remains to show that the solutions found in Lemma 3.1 and Lemma 3.2 are distinct. Since
this implies that
and
are distinct. This concludes the proof.
4. Acknowledgements
The author is indebted to the referees for carefully reading this paper and making valuable comments and suggestions.