This tutorial presents the main characteristics, evaluations, and applications of various Markov modeling approaches including the discrete-time Markov models (DTMM), continuous-time Markov models (CTMM), hidden Markov models, and item-response theory (IRT) model with Markov sub-models.