Title MCMC metoda i primjene Author Elio Bartoš Mentor Bojan Basrak (mentor)Committee member Bojan Basrak (predsjednik povjerenstva)Committee member Marcela Hanzer (član povjerenstva)Committee member Ilja Gogić (član povjerenstva)Committee member Tina Bosner (član povjerenstva)Granter University of Zagreb Defense date and country 2016-09-23, Croatia Scientific / art field, NATURAL SCIENCES Abstract U ovom radu govorimo o MCMC algoritmima i ilustriramo njihovu primjenu u bayesovskoj statistici. MCMC algoritmi služe za simuliranje uzorka iz distribucije s gustoćom f koju moramo znati samo do na konstantu. Oni konstruiraju Markovljev lanac čija se putanja uzima kao uzorak iz distribucije s gustoćom f, što nam omogućuje ergodski teorem. Nakon motivacijskog uvoda, u prvom poglavlju obradujemo nužnu teoriju koja je u pozadini MCMC algoritama, a to je teorija Markovljevih lanaca na općenitom
... More skupu stanja. Prvo definiramo osnovne pojmove poput prijelazne jezgre i Markovljevog lanca, a zatim neka svojstva Markovljevih lanaca poput ireducibilnosti, povratnosti, Harrisove povratnosti i stacionarnu distribuciju Markovljevog lanca. Iskazujemo teorem koji nam daje dovoljne uvjete da marginalne distribucije lanca konvergiraju prema stacionarnoj distribuciji f i ergodski teorem koji nam omogućuje da putanju lanca koristimo kao uzorak dobiven iz stacionarne distribucije f. Nakon toga obradujemo Metropolis-Hastings algoritam, jedan od najvažnijih MCMC algoritama. Dokazujemo da putanju lanca dobivenog ovim algoritmom možemo koristiti kao uzorak iz zadane ciljne distribucije f. Uzorak iz distribucije f služi nam kako bismo procijenili karakteristike distribucije f pa smo uveli efektivnu veličinu uzorka i Monte Carlo standardnu grešku kao mjere preciznosti dobivenih procjena. Kao primjere Metropolis-Hastings algoritma obradili smo Nezavisni Metropolis-Hastings algoritam i Metropolis-Hastings algoritam sa slučajnom šetnjom. Na primjeru smo vidjeli da je kod Nezavisnog Metropolis-Hastings algoritma najbolje odabrati proposal distribuciju što sličniju ciljnoj distribuciji, ukoliko je to moguće. Kod Metropolis-Hastings algoritma sa slučajnom šetnjom uočili smo da za efikasnost algoritma moramo odgovarajuće skalirati proposal distribuciju. Na kraju smo spomenuli i Gibbsovo uzorkovanje koji je vrlo važan i u primjenama čest algoritam. U zadnjem dijelu ovoga rada objašnjavamo glave ideje bayesovske statistike - pojmove poput apriori i aposteriori distribucije. Na primjeru bacanja novčića ilustriramo kako se aposteriori distribucija ponekad može izračunati egzaktno, no uočavamo i da se to ponekad neće moći zbog čega onda koristimo MCMC algoritme. Nakon toga ukratko objašnjavamo generalizirane linearne modele kako bismo mogli napraviti model linearne regresije s teškim repovima na bayesovski način. Kao primjer uzeli smo određivanje potrošnje goriva automobila. Nakon definiranja modela komentiramo izbor apriori distribucija te koristimo MCMC algoritme kako bismo dobili uzorak iz aposteriori distribucije. Prije nego što analiziramo rezultate, radimo dijagnostiku kako bismo provjerili da dobiveni uzorak doista predstavlja uzorak iz aposteriori distribucije. U dodatku objašnjavamo kako smo tehnički proveli ranije opisan postupak pomoću programskog jezika R i programa JAGS. Less Abstract (english) In this paper we talk about Markov chains Monte Carlo (MCMC) algorithms, and we illustrate their application in Bayesian statistics. MCMC algorithms are used to generate sample from distribution with density f, which is known up to a normalising constant only. They produce Markov chains whose trajectory represents a sample from distribution with the density f, which is justified by Ergodic theorem. After motivational introduction, in chapter one, we introduce necessary background theory of
... More MCMC algorithms; that is the theory of Markov chains on general state spaces. First, we define basic terms such as transition kernels and Markov chains, and after that we define some properties of Markov chains such as irreducibility, recurrence, Harris recurrence and stationary distribution of Markov chain. We state a theorem which gives us necessary conditions on the chain, so that its marginal distributions converge to its stationary distribution f, and Ergodic theorem. Secondly, we define Metropolis-Hastings algorithm, one of the most important MCMC algorithms. We prove that we can use a trajectory generated with Metropolis-Hastings algorithm as sample from target distribution f. Sample from distribution f is used to approximate distribution f and its characteristics, so we introduced effective sample size and Monte Carlo standard error as measures of accuracy. Furthermore, we give examples of Metropolis-Hastings algorithm: the Independent Metropolis-Hastings and Random walk Metropolis-Hastings. On a simple example we demonstrate that in the Independent Metropolis-Hastings algorithm optimal proposal distribution should be as much proportional to target distribution as possible. In the Random walk Metropolis-Hastings algorithm we noticed that we need to scale proposal distribution for effectiveness. Finally, we mention Gibbs sampler which is very important and widely used algorithm. In the last part of this paper, we explain the main ideas of Bayesian statistics. We introduce terms like prior and posterior distribution. On a simple coin tossing example we illustrate that sometimes one can exactly calculate posterior distribution, but we notice that in cases when that is not possible, one can use MCMC algorithms. After that, we briefly explain generalised linear models, so that we can introduce heavy-tailed linear regression model from Bayesian perspective. As an example we are estimating car fuel consumption. After defining the model, we comment on choosing prior distributions, and we use MCMC algorithms to get sample from posterior distribution. Before we analize posterior distribution, we check if our sample is representative of prior distribution and accurate enough. In the appendix, we explain how we used programming language R and JAGS to give Bayesian analysis of the heavy-tailed linear regression model. Less Keywords
MCMC algoritmi
Markovljev lanac
Markovljevi lanci
Metropolis-Hastings algoritam
Nezavisni Metropolis-Hastings algoritam
Metropolis-Hastings algoritam sa slučajnom šetnjom
Gibbsovo uzorkovanje
bayesovska statistika
apriori distribucija
aposteriori distribucija
Keywords (english)
Markov chains Monte Carlo algorithms
MCMC algorithms
Markov chain
Metropolis-Hastings algorithm
Independent Metropolis-Hastings
Random walk Metropolis-Hastings
Gibbs sampler
Bayesian statistics
prior distribution
posterior distribution
Language croatian URN:NBN urn:nbn:hr:217:478879 Study programme Title: Mathematical Statistics Type of resource Text File origin Born digital Access conditions Open access Terms of use Created on 2017-05-22 11:19:18
