Vrednovanje i odabir modela

master's thesis

• Title (english): Evaluation and selection of models
• Author: Tin Kranželić
• Mentor: Miljenko Huzak (mentor)
• Committee member: Eduard Marušić-Paloka (predsjednik povjerenstva)
• Committee member: Pavle Goldstein (član povjerenstva)
• Committee member: Miljenko Huzak (član povjerenstva)
• Committee member: Marcela Hanzer (član povjerenstva)
• Granter: University of Zagreb; Faculty of Science; (Department of Mathematics); Zagreb
• Defense date and country: 2021-12-02, Croatia
• Scientific / art field, discipline and subdiscipline: NATURAL SCIENCES; Mathematics

Abstract

U ovom radu ukratko smo analizirali ključne ideje iza problema odabira i vrednovanja modela. Prilikom procjene statističkog modela iz opaženih podataka, izbor najprikladnijeg modela izmedu više potencijalnih modela je neizbježan problem. Također, jednom kada imamo finalni model, željeli bismo znati i koliko precizne predikcije na nevidenim podacima možemo očekivati. U tu svrhu, analizirali smo odnos varijance i pristranosti modela, te način na koji porast kompleksnosti modela utječe na njih. Potom smo uveli definicije optimizma pogreške treniranja i objasnili zašto pogreška treniranja nije nužno vjerodostojan indikator konačne generalizacije modela. Također smo razmotrili analitičke metode AIC i BIC za aproksimaciju unutar-uzoračke pogreške u svrhu odabira modela. Na kraju smo dali kratak pregled tehnike unakrsnog vrednovanja koja pruža direktan procjenitelj izvan-uzoračke pogreške, te kao takva, osim u svrhu odabira modela može poslužiti i u svrhu vrednovanja modela. Sve navedene metode i tehnike potkrijepili smo jednostavnim, ali ilustrativnim primjerima.

Abstract (english)

In this thesis, we have briefly analyzed the key ideas of the model selection and evaluation problems. When estimating a statistical model from observed data, choosing the most appropriate model among multiple potential models is an inevitable problem. Once we have the final model, we would also like to know what to expect from the final model regarding the accuracy of predictions on unseen data. For this reason, we analyzed the relationship between model variance and bias, and the way in which the increase in model complexity affects them. We then introduced definitions of training error optimism and explained why training error is not necessarily a reliable indicator of the final generalization of the model. We also considered AIC and BIC, analytical methods for in-sample error approximation for model selection purposes. Finally, we have given a brief overview of the cross-validation method which provides us with the direct estimator of extra-sample error, and as such, in addition to the purpose of model selection, it can also serve the purpose of model evaluation. We have supported all the above methods and techniques with simple but illustrative examples.

• Keywords: varijanca; pristranost modela; pogreška treniranja; analitička metoda AIC; analitička metoda BIC
• Keywords (english): variance; model bias; coaching error; AIC analytical method; BIC analytical method
• Language: croatian
• URN:NBN: urn:nbn:hr:217:315279
• Study programme: Title: Mathematical Statistics; Study programme type: university; Study level: graduate; Academic / professional title: magistar/magistra matematike (magistar/magistra matematike)
• Type of resource: Text
• File origin: Born digital
• Access conditions: Open access
• Created on: 2021-12-15 10:28:26