Download
PDF 24.65 MB
master's thesis
19   3
Natural Language Processing
Ena Pribisalić (2019)
Josip Juraj Strossmayer University of Osijek
Department of Mathematics
Chair of Applied Mathematics
Applied Mathematics Research Group
Cite this document

Pribisalić, E. (2019). Obrada prirodnog jezika (Master's thesis). Retrieved from https://urn.nsk.hr/urn:nbn:hr:126:997882

Pribisalić, Ena. "Obrada prirodnog jezika." Master's thesis, Josip Juraj Strossmayer University of Osijek, Department of Mathematics, 2019. https://urn.nsk.hr/urn:nbn:hr:126:997882

Pribisalić, Ena. "Obrada prirodnog jezika." Master's thesis, Josip Juraj Strossmayer University of Osijek, Department of Mathematics, 2019. https://urn.nsk.hr/urn:nbn:hr:126:997882

Pribisalić, E. (2019). 'Obrada prirodnog jezika', Master's thesis, Josip Juraj Strossmayer University of Osijek, Department of Mathematics, accessed 13 October 2019, https://urn.nsk.hr/urn:nbn:hr:126:997882

Pribisalić E. Obrada prirodnog jezika [Master's thesis]. Osijek: Josip Juraj Strossmayer University of Osijek, Department of Mathematics; 2019 [cited 2019 October 13] Available at: https://urn.nsk.hr/urn:nbn:hr:126:997882

E. Pribisalić, "Obrada prirodnog jezika", Master's thesis, Josip Juraj Strossmayer University of Osijek, Department of Mathematics, Osijek, 2019. Available at: https://urn.nsk.hr/urn:nbn:hr:126:997882

Metadata
TitleObrada prirodnog jezika
Title (english)Natural Language Processing
AuthorEna Pribisalić
MentorDarija Marković (mentor)
Committee memberDomagoj Matijević (predsjednik povjerenstva)
Committee memberDarija Marković (član povjerenstva)
Committee memberDomagoj Ševerdija (član povjerenstva)
GranterJosip Juraj Strossmayer University of Osijek
Department of Mathematics
(Chair of Applied Mathematics)
(Applied Mathematics Research Group)
Osijek
Defense date and country2019-09-17, Croatia
Scientific / art field,
discipline and subdiscipline		NATURAL SCIENCES
Mathematics
Mathematical Logic and Accounting
Abstract
Cilj ovoga rada je upoznavanje s osnovama obrade prirodnog jezika, točnije s analizom riječi i gramatičkih pravila koja koristimo u engleskom jeziku. Pri pretraživanju ili filtriranju teksta služimo se regularnim izrazima dok sličnost riječi mjerimo pomoću udaljenosti uređivanja. Ulazni tekst je potrebno normalizirati, a pri predvđanju iduće riječi u nizu koristimo modele N-grama koji se procjenjuju pomoću zbunjenosti. Kod modeliranja koriste se zaglađivanje te backoff i interpolacija. Drugi način komunikacije je govor, koji je potrebno digitalizirati i nakon toga obraditi. Danas postoje mnogi alati koji nam mogu pomoći u analizi zvučnih valova koji nastaju govorom. Sinteza govora može se provesti na 3 načina: lančanom sintezom, formantskom sintezom te artikularnom sintezom. Računalni pristup sintaksi predstavljaju kontekstno-slobodne gramatikame koje implementiraju gramatička pravila. Izvod pravila prikazujemo stablima parsiranja, a problem mapiranja niza riječi u pripadno stablo parsiranja zove se sintaktičko parsiranje pri čemu koristimo CKY (Cocke-Kasami-Younger) algoritam. Konačno, prvi korak pri procesu izvlačenju informacija je prepoznavanje imenskih entiteta. Taj dio izvlačenja informacija koristi označavanje riječi u nizu redom, uz IOB kodiranje, a pri procjeni ovakvih sustava koriste se opoziv, preciznost i F-mjera kao metrike.
Abstract (english)
The aim of this paper is getting to know the basics of natural language processing, more precisely word and grammar rules analysis which we use in english language. We use regular expressions in tasks like searching and filtering a text and we measure word similarity using edit distance. We need to normalize the input text, and during next word prediction we use N-gram models which we evaluate with perplexity. We use smoothing, backoff and interpolation during modeling. Another way people can communicate is using speech, which needs to be digitized and processed. Today there is a plethora of tools which we can use to analyse speech sound waves. As for speech synthesis, there exist 3 different paradigms: concatenative synthesis, formant synthesis and articulatory synthesis. Approaching syntax from computational point of view, we use context-free grammars. Syntax parsing is a problem of mapping a word sequence into a parse tree, where a parse tree can be obtained by applying grammar rules. Here we use CKY (Cocke-Kasami-Younger) algorithm. Finally, the first step in information extraction is named entity recognition. This part of information extraction uses sequential word tagging, along with IOB encoding, and for system evaluation, we use recall, precision and F-measure.
Keywords
Keywords (english)
Languagecroatian
URN:NBNurn:nbn:hr:126:997882
Study programmeTitle: Mathematics; specializations in: Financial and Statistical Mathematics, Mathematics and Computer Science, Industrial and Applied Mathematics
Course: Financial and Statistical Mathematics
Study programme type: university
Study level: graduate
Academic / professional title: magistar/magistra matematike (magistar/magistra matematike)
Type of resourceText
File originBorn digital
Access conditionsOpen access
Terms of usehttp://rightsstatements.org/vocab/InC/1.0/
Created on2019-09-19 06:56:09