1-3 of 3 Books
Markov Decision Processes
By Martin L. Puterman
Markov-modulated processes & semiregenerative phenomena
By António Pacheco
An introduction to Markov processes
By Daniel W. Stroock