In this thesis we study Brownian motion, a random process named by Robert Brown who discovered that the pollen particles dispersed in the liquid move constantly and randomly. The basic characteristics of this random process are independent and normally distributed increments. Even though the applications and implications of Brownian motion are numerous and significant, it is not suitable for modeling a phenomenon with an unimaginable probability of extreme events. This is the motivation for introducing fractal Brownian motion, as a generalization of Brownian motion and model for stable processes. In the thesis we discuss the concepts of fractal and fractal dimensions. In examples, we show self similarity as one of the basic characteristics of a fractal. We consider the application of fractal Brownian motion to the financial market. First of all we consider the historical problem of cotton prices which is characterized by self similarity in time series of prices. Benoit Mandelbrot noticed that cotton prices do not follow the normal distribution but one of the distributions with the heavy tail. We also define terms multifractal and multifractal time, on the basis of which we are conducting a multifractal model of the market.