This work describes a novel adaptive matrix/vector gradient (AMVG) algorithm for design of IIR filters and ARMA signal models. The AMVG algorithm can track to IIR filters and ARMA systems having poles also outside the unit circle. The time reversed filtering procedure was used to treat the unstable conditions. The SVD-based null space solution was used for the initialization of the AMVG algorithm. We demonstrate the feasibility of the method by designing a digital phase shifter, which adapts to complex frequency carriers in the presence of noise. We implement the half-sample delay filter and describe the envelope detector based on the Hilbert transform filter.