Q-Learning-Based Adaptive Waveform Selection in Cognitive Radar

Bin WANG, Jinkuan WANG, Xin SONG, Fulai LIU
DOI: 10.4236/ijcns.2009.27077                

Abstract

Cognitive radar is a new framework of radar system proposed by Simon Haykin recently. Adaptive waveform selection is an important problem of intelligent transmitter in cognitive radar. In this paper, the problem of adaptive waveform selection is modeled as stochastic dynamic programming model. Then Q-learning is used to solve it. Q-learning can solve the problems that we do not know the explicit knowledge of state-transition probabilities. The simulation results demonstrate that this method approaches the optimal wave-form selection scheme and has lower uncertainty of state estimation compared to fixed waveform. Finally, the whole paper is summarized.

Keywords

Waveform Selection; Q-Learning; Space Division; Cognitive Radar

Conflicts of Interest

The authors declare no conflicts of interest.

References

[1]	S. Haykin, “Cognitive radar: A way of the future,” IEEE Signal Processing Magazine, Vol. 23, No. 1, pp. 30–40, 2006.
[2]	C. Rago, P. Willett, and Y. Bar-Shalom, “Detecting- tracking performance with combined waveforms,” IEEE Transactions on Aerospace and Electronic Systems, Vol. 34, No. 2, pp. 612–624, 1998.
[3]	D. J. Kershaw and R. J. Evans, “Waveform selective probabilistic data association,” IEEE Transactions on Aerospace and Electronic Systems, Vol. 33, No. 4, pp. 1180–1188, 1997.
[4]	Y. He and E. K. P. Chong, “Sensor scheduling for target tracking in sensor networks,” 43rd IEEE Conference on Decision and Control, Paradise, Island, Bahamas, pp. 743–748, 2004.
[5]	V. Krishnamurthy, “Algorithms for optimal scheduling of hidden Markov model sensors,” IEEE Trans. on Signal Processing, Vol. 50, No. 6, pp.1382–1397, 2002.
[6]	C. O. Savage, and B. Moran, “Waveform selection for maneuvering targets within an IMM framework,” IEEE Transactions on Aerospace and Electronic Systems, Vol. 43, No. 3, pp. 1205–1214, 2007.
[7]	C. T. Capraro, I. Bradaric, G. T. Capraro, and T. K. Lue, “Using genetic algorithms for radar selection,” 2008 IEEE Radar Conference, Inc., Utica, NY, pp. 1–6, May 2008.
[8]	B. F. La Scala and R. J. Moran Wand Evans, “Optimal adaptive waveform selection for target detection,” The International Conference on Radar, Adelaide, SA, Aus-tralia, pp. 492–496, Sept. 2003.
[9]	La Scala, Rezaeian, and Moran, “Optimal adaptive wave-form selection for target tracking,” International Confer-ence on Information Fusion, pp. 552–557, 2005.
[10]	D. Bertsekas, “Dynamic programming and optimal con-trol,” Athena Scientific, Second Edition, Vol. 1, 2001.
[11]	V. Krishnamurthy, “Algorithms for optimal scheduling of hidden Markov model sensors,” IEEE Transactions on Signal Processing, Vol. 50, No. 6, pp. 1382–1397, 2002.
[12]	W. S. Lovejoy, “Computationally feasible bounds for partially observed Markov decision processes,” Opera-tions Research, Vol. 39, No. 1, pp. 162–175, 1991.