MATNLI MA’LUMOTLARGA DASTLABKI ISHLOV BERISH ALGORITMLARI
Keywords:
Matnli ma’lumot, dastlabki ishlov, TF-IDF, so‘zlar sumkasi modeli, vektor fazosi, Word2Vec, CBOW, Skip-Gram, semantika, normallashtirish, to‘xtash so‘zlar.Abstract
The article analyzes the algorithmic foundations of the pre-processing of text
data. First, the stages of text cleaning using methods such as HTML tags, stop words, punctuation
marks, and kernelization are considered. Classical methods for numerical representation of text,
such as TF-IDF, bag-of-words model, and vector space model, are covered. Modern models based
on neural networks, such as Word2Vec, CBOW, and Skip-Gram, are also introduced. These models
preserve the semantic connections and context in the text, which allows for in-depth analysis of
the text. The article also covers the theoretical foundations of algorithms and their application in
real areas.
References
Saleh A.I., Abulwafa A.E., Al Rahmawy M.F. A web page distillation strategy for efficient focused crawling based on optimized Naive Bayes (ONB) classifier // Applied Soft Computing. – 2017. – Vol. 53. – P. 181–204.
Бабомурадов О.Ж., Қўйлиева Ф.А., Туракулов О.Х. Clustering approaches to text documents // Научный журнал транспортных средств и дорог. – 2022. – № 1. – С. 109–114.
Karystianis G., Thayer K., Wolfe M., Tsafnat G. Evaluation of a rule-based method for epidemiological document classification towards the automation of systematic reviews // Journal of Biomedical Informatics. – 2017. – Vol. 70. – P. 27–34.
Babomuradov O.J., Turakulov O.X. Matnni intellektual tahlillash yondashuvlari // Informatika va energetika muammolari. – 2016. – № 4. – B. 68–72.
Babomuradov O.J., Turakulov O.X. Matn belgilari o‘lchovini qisqartirish hamda tabiiy til matnini tahlillash yondashuvi // Innovative Developments and Research in Education: International Scientific-Online Conference. – Canada, 2023-yil 23-dekabr. – Part 24. – P. 320.
Babomuradov O.J., Mamatov N.S., Boboyev L.B., Otaxonova B.I. Text documents classification in Uzbek language // International Journal of Recent Technology and Engineering. – 2019. – Vol. 8, No. 2. – P. 39–43.