Arhitekture dubokih neuronskih mreža u problemu obrade prirodnog jezika

master's thesis

• Title (english): Architectures of deep neural networks in the problem of natural language processing
• Author: Tin Štefanić
• Mentor: Luka Grubišić (mentor)
• Committee member: Luka Grubišić (predsjednik povjerenstva)
• Committee member: Zvonimir Bujanović (član povjerenstva)
• Committee member: Andrej Dujella (član povjerenstva)
• Committee member: Goran Muić (član povjerenstva)
• Granter: University of Zagreb; Faculty of Science; (Department of Mathematics); Zagreb
• Defense date and country: 2021-09-22, Croatia
• Scientific / art field, discipline and subdiscipline: NATURAL SCIENCES; Mathematics

Abstract

U ovom diplomskom radu obrađuju se izabrane arhitekture dubokih neuronskih mreža korištene u problemu obrade prirodnog jezika. U prvom poglavlju cilj je obraditi klasične arhitekture, rekurzivne neuronske mreže i LSTM, te njihove prednosti i mane. Zatim u idućem poglavlju se obrađuje modernija Transformer arhitektura, arhitektura koja sadrži prednosti klasičnih arhitektura, ali nema njihove mane. U trećem poglavlju su obrađene napredne pred-trenirane Transformer arhitekture, BERT i njegova nadogradnja ALBERT. Zatim u zadnjem poglavlju te arhitekture su primjenjene na problem odgovaranja na pitanja, problem na kojem su izražene mane klasičnih arhitektura.

Abstract (english)

This thesis deals with selected architectures of deep neural networks used in the problem of natural language processing. In the first chapter, the goal is to deal with classical architectures, recurrent neural networks and LSTM, and their advantages and disadvantages. Then, in the next chapter, we discuss the modern Transformer architecture, an architecture that possesses the advantages of classical architectures, but doesn’t share their disadvantages (e.g. it is easier to use parallel algorithms and it allows for transfer learning). The third chapter deals with the advanced pre-trained Transformer architecture, BERT and its upgrade ALBERT. Then, in the last chapter, these architectures are applied to a question answering problem, a problem on which the flaws of classical architectures are expressed.

• Keywords: klasične arhitekture; rekurzivne neuronske mreže; LSTM; Transformer arhitektura; ERT; nadogradnja ALBERT
• Keywords (english): classical architectures; recurrent neural networks; LSTM; Transformer architecture; BERT; ALBERT
• Language: croatian
• URN:NBN: urn:nbn:hr:217:848781
• Study programme: Title: Computer Science and Mathematics; Study programme type: university; Study level: graduate; Academic / professional title: magistar/magistra računarstva i matematike (magistar/magistra računarstva i matematike)
• Type of resource: Text
• File origin: Born digital
• Access conditions: Open access
• Created on: 2021-11-08 12:54:39