Varijante gradijentnog spusta u strojnom učenju s primjerima

master's thesis

• Title (english): Variants of gradient descent in machine learning with examples
• Author: Dorotea Bošnjak
• Mentor: Nikola Sandrić (mentor)
• Committee member: Nikola Sandrić (predsjednik povjerenstva)
• Committee member: Maja Resman (član povjerenstva)
• Committee member: Marko Erceg (član povjerenstva)
• Committee member: Filip Najman (član povjerenstva)
• Granter: University of Zagreb; Faculty of Science; (Department of Mathematics); Zagreb
• Defense date and country: 2023-09-26, Croatia
• Scientific / art field, discipline and subdiscipline: NATURAL SCIENCES; Mathematics

Abstract

U ovom radu stječemo dublje razumijevanje gradijentnog spusta u strojnom i dubokom učenju, istraživajući različite varijante i optimizacije. U prvom poglavlju iznosimo ključne koncepte i teoreme vezane uz konvergenciju gradijentnog spusta i metoda ubrzanja. Nadalje, analiziramo varijante gradijentnog spusta koje se razlikuju po broju točaka iz skupa za treniranje koje se uzimaju u obzir u svakoj iteraciji. Iz njihove jednostavnosti proizlaze brojni izazovi i ograničenja, što nas potiče na istraživanje optimizacijskih algoritama. Za svaku od optimizacijskih metoda proučavamo teorijske temelje i opis koraka algoritma s ciljem stjecanja intuicije i razumijevanja njihovog funkcioniranja. Nakon teorijskog objašnjenja slijedi primjena varijanti i optimizacijskih algoritama na dva različita modela. Jedan je iz područja medicine, a drugi je iz genetike. Na taj način dobivamo uvid u praktičnu primjenu ovih metoda i njihovu implementaciju u Pythonu. Istražujemo njihovu učinkovitost na konkretnim modelima.

Abstract (english)

In this thesis, we aim to better understand gradient descent in machine and deep learning by exploring its different versions and improvements. In the first chapter, we cover essential concepts and theorems related to how gradient descent works and how it can be made faster. We also look at different variations of gradient descent that involve considering different amounts of data in each step. While studying these variations, we encounter various challenges and computer limitations that lead us to optimization techniques. For each of these optimization methods, we dive into the theory behind them and explain how they work step by step. Our goal is to help readers gain an intuitive understanding of these methods. After explaining the theory, we apply these optimization techniques to two different models. One in the medical field and the other in genetics. This allows us to see how these methods are used in practice and how they can be implemented using Python. We also evaluate how well they perform on real-world models.

• Keywords: konvergencija gradijentnog spusta; metode ubrzanja; optimizacijski algoritmi; Python
• Keywords (english): gradient descent convergence; acceleration methods; optimization algorithms; Python
• Language: croatian
• URN:NBN: urn:nbn:hr:217:216150
• Study programme: Title: Mathematical Statistics; Study programme type: university; Study level: graduate; Academic / professional title: sveučilišni magistar matematike (sveučilišni magistar matematike)
• Type of resource: Text
• File origin: Born digital
• Access conditions: Open access
• Created on: 2024-01-30 11:59:55