Mathematical modeling is a useful tool used to understand the outbreak, spread and influence of epidemics on a population. Mathematical models can predict the number of infections and deaths, as well as the duration and speed of disease spread. One of the most important parameters in disease modeling is the basic reproductive number defined as the number of secondary infections that are caused by one infectious person in a completely susceptible population. This number also represents a threshold quantity that determines whether there will be an outbreak of epidemics or not. Mathematical epidemic models can be classified into deterministic and stochastic models. Deterministic models cannot handle randomness and thus give the same output for every input given. Stochastic models, on the other hand, can handle randomness and thus give different outputs for every input specified. The most recognizable deterministic model is the SIR model, where the population is divided into three compartments: susceptible, infected and recovered. Another two well-known deterministic compartmental models covered in this thesis are SIS and SEIR epidemic models. Stohastic SIS and SIR models are modeled using Markov chains in discrete and continuous time. Memoryless property which states that the state of the stochastic process in the nearest future depends only on the current state of the same process makes Markov chains an important tool in modeling naturally occurring events such as epidemics