3.1 MLE model

In the MLE model, we learn segmentation in two steps (Üstün and Can Reference Üstün and Can2016). In the first step, we detect the potential morpheme boundary points within each word in the corpus. In the second step, we learn the final segmentation points using the potential morphemes obtained from the first stage.

The initial segmentation is performed based on the semantic similarity between word forms that are derived from each other. For example, the substrings recreat, recreation, and recreational are all semantically related to recreationally. We use the semantic relatedness similar to parent–child relations introduced in MorphoChain model (Narasimhan et al. Reference Narasimhan, Barzilay and Jaakkola2015). We use the parent–child relations for the initial segmentation of the corpus. Word embeddings obtained from word2vec (Mikolov et al. Reference Mikolov, Chen, Corrado and Dean2013) are used analogously to compute the semantic relatedness between the prefixes of each word. The cosine similarity between the embeddings of two prefixes in n-dimensional space is computed as follows:

(1) \begin{eqnarray}\cos (v(w_1),v(w_2)) &=& \frac{|v(w_1)\cdot v(w_2)|}{\left \| v(w_1) \right \|\cdot \left \| v(w_2) \right \|} \end{eqnarray}

where $w_1$ and $w_2$ denote the prefixes (i.e., subwords), $v(w_1)$ denotes the word embedding of $w_1$ , and $v(w_2)$ denotes the word embedding of $w_2$ .

Each word is scanned from right to left, with a letter removed at each step. If the cosine similarity between two prefixes is above a manually set threshold value,Footnote ^b then the segmentation point is saved as a potential segmentation and the segmented suffix is saved in a potential morpheme list. This process is repeated until the beginning of word is reached.

The process is given in Algorithm 1. Here, $\textit{GetSuffix}(charNo, word)$ returns a suffix from the end of the word with length charNo. $\textit{GetPrefix}$ (charNo, word) returns a prefix from the beginning of the word with length charNo, and $\textit{Cosine}(prefix, word)$ calculates the cosine similarity between the prefix and the word.

An example stripping process is given in Figure 1. The cosine similarity values for the same example are given in Table 1. It is clear that cosine similarity is noticeably higher at the real segmentation points than at non-segmentation points. The cosine similarity becomes very small if the prefix and the word are not semantically related. We assign $-1$ for the cosine similarity if either the prefix or the word does not exist in the word embedding space.

Figure 1. An illustration that shows the segmentation of the word fearlessly. The segmentation is performed from the right end of the word by removing letters using a manually set semantic similarity threshold.

Algorithm 1 Baseline segmentation algorithm that detects potential morpheme boundary points based on cosine similarity between word embeddings.

Table 1. The cosine similarity between the prefixes of the word fearlessly is given. $0.25$ is assigned for the cosine similarity threshold in this example

Once potential segmentations are obtained, the likelihood of the segmentation for a given corpus under the MLE model is defined as follows:

(2) \begin{eqnarray}\hspace*{-15pt} p(W|S) &=& \prod_{i=1}^T{p(w_i|S)} \hspace*{90pt} \end{eqnarray}

(3) \begin{eqnarray} &=& \prod_{i=1}^T{p(w_i=m_{i}^{0}+\cdots+m_{i}^{N}|S)} \end{eqnarray}

(4) \begin{eqnarray} &=& \prod_{i=1}^T{p(m_{i}^0|S)\prod_{j=1}^N{p(m_{i}^j|m_{i}^{j-1},S)}} \end{eqnarray}

Here, in Equation (2), W is the word list of size T and S is the initial segmentation suggested by the inner-word cosine similarities. Therefore, the probability of each word $w_i$ with morphemes $m_{i}^{0},\ldots,m_{i}^{N}$ is defined using the likelihood under the initial segmentation. For each word, we search for the segmentation that leads to the maximum likelihood under the given segmentation:

(5) \begin{equation}\arg\max\limits_{w_{i}=m_{i}^{0}+\cdots+m_{i}^{N}\in {S_{w_i}}} P(w_{i}=m_{i}^{0}+m_{i}^{1}+\cdots+m_{i}^{N}|S) \end{equation}

where $m_{i}^j$ denotes the ith morpheme in $w_{i}$ that consists of N morphemes. Here, $S_{w_i}$ denotes the set of all possible segmentations of $w_i$ .

We replace the root morphemes with a start symbol F to reduce the sparsity. We assume that the root is always the leftmost morpheme, which may not be always the case in English, German, or the Semitic languages. However, due to the usage of word embeddings, our model requires all tokens to be available in the dataset on their own with a proper embedding in the embedding space. However, this is not always true for the prefixes.Footnote ^c Hence, the probability of the first morpheme after the root being less is computed as follows:

(6) \begin{equation}p(\textit{less}|F)=\frac{n(\!<F, less>\!)}{n(F)}\end{equation}

where $n(\!<F, less>\!)$ is the frequency of less being the first morpheme coming just after the root. Both bigram and unigram frequencies are obtained from the initial segmentations. Here, we use the word types and not tokens. We also discard the frequency information provided by Morpho Challenge data. We do not distinguish root morphemes and bound morphemes in normalization. In other words, $p(m_i^0)$ is normalized by the total frequency of all morphemes and not just the root morphemes.

Viterbi algorithm is used to find the segmentation with the maximum likelihood according to Equation (5). We apply Laplace smoothing to eliminate the zero probabilities in the model.Footnote ^d

3.2 MAP model

In the MAP model, we adopt the Bayesian notion of the probability to place some prior belief on the segmentation of the corpus. For that purpose, we use the cosine similarity between the embeddings of the different forms of a word because we know that words that are inflected or derived from each other are semantically related to each other and this semantic relatedness is known a priori. Therefore, we use both orthographic features (such as the observed morpheme counts as likelihood) and semantic features (cosine similarity between word embeddings). The posterior probability of the segmentation for a given corpus is defined as follows:

(7) \begin{eqnarray} \hspace*{-66pt} p(S|W) &\propto& p(W|S) p(S)\hspace*{115pt}\end{eqnarray}

(8) \begin{eqnarray} \hspace*{-38pt} &\propto& \prod_{i=1}^T{p(w_i|S)p(S)} \hspace*{96pt}\end{eqnarray}

(9) \begin{eqnarray} &\propto& \prod_{i=1}^T{\Big(P(w_{i}=m_{i}^{0}+m_{i}^{1}+\cdots+m_{i}^{N}|S) \prod^N_{k=1} p(s_i^k)\Big)} \end{eqnarray}

(10) \begin{align} \hspace*{-49pt} &\propto \prod_{i=1}^T{P(m_{i}^{0}|S)} P(m_{i}^{1}|m_{i}^{0},S)p(s_i^1)\nonumber \\ &\quad \cdots P(m_{i}^{N}|m_{i}^{0},\cdots,m_{i}^{N-1}, S)p(s_i^N)\hspace*{49pt}\end{align}

where S refers to the segmentation of all the words in the corpus that is learned during inference (whereas S denotes the initial segmentation of the words in the corpus which does not change in a separate inference step). N is the number of morphemes in each word. Here, $s_i^k$ refers to the kth morpheme boundary in $w_i$ that splits $m_i^k$ from the rest of the word, that is, $m_i^0+\cdots+m_i^{k-1}$ . The probability of splitting a word at any point is proportional to the cosine similarity between the embeddings of the two prefixes. Thus, the probability of splitting the word $w_i=m_i^0+\cdots+m_i^{k-1}+m_i^{k}$ after the $k-1$ th morpheme is estimated as follows (the word is split such as $m_i^0+\cdots+m_i^{k-1}$ and $m_i^{k}$ ):

(11) \begin{align}p(s_i^k) = \frac{cos(m_i^0+\cdots+m_i^{k-1}, m_i^0+\cdots+m_i^{k})}{\sum_{z=1}^N{cos(m_i^{0}+\cdots+m_i^{z-1},m_i^{0}+\cdots+m_i^{z})}}\end{align}

Therefore, we convert the cosine similarity into a proper probability distribution by normalizing with the summation of cosine similarities through all possible binary split points. Here, m may not be a valid morpheme. It corresponds to any split point in a word during inference. However, at the end of the inference, it normally corresponds to valid morphemes.

We use the MAP model by adopting two language models: unigram and bigram language model for the likelihood estimation.

3.2.1 Unigram model

In the unigram model, each morpheme is assumed to be independent from the other morphemes within the same word. The dependency is incorporated through the prior information which adopts the cosine similarity coming from the word embeddings. The unigram language model for the likelihood is defined as follows:

(12) \begin{align}p(w_i|S) &=P(w_{i}=m_{i}^{0}+m_{i}^{1}+\cdots+m_i^{N}|S) \nonumber \\ &=\prod_{j=1}^Np(m_{i}^{j}|S)\end{align}

3.2.2 Bigram model

In the bigram model, we adopt a first-order Markov chain where each morpheme depends only on the previous morpheme:

(13) \begin{align}p(w_i|S) &= P(w_{i}=m_{i}^{0}+m_{i}^{1}+\cdots+m_i^{N}|S)\nonumber\\ &= p(m_{i}^{0})\prod_{j=1}^Np(m_{i}^{j}|m_{i}^{j-1}, S)\end{align}

3.3 Limitations

Since both MAP and MLE models utilize semantic information, they require word embedding of each word in the dataset. Otherwise, our models cannot suggest any segmentation for those words that do not have embeddings. This is one of the limitations of the models. This limitation causes prefixes not to be learned by the models since they do not exist on their own in the datasets and they do not have embeddings. One possible way to overcome this limitation is to use the n-gram or morpheme embeddings instead of using word embeddings. However, this will require defining the word similarities again based on n-gram-based word embeddings, since we postulate that inflected forms of a word will remain semantically similar to each other. We did further analysis using n-gram-based word embeddings and they are presented in Section 5.2.4.

Another limitation of our approach is the requirement of embeddings for each prefix of a word in order to compare the prefix with the entire word to assess a possible morpheme boundary. Since prefix affixes do not appear as separate words, their embeddings are not learned with standard word embedding methods, such as word2vec (Mikolov et al. Reference Mikolov, Chen, Corrado and Dean2013). There are methods that learn morpheme embeddings such as morph2vec (Üstün and Can Reference Üstün and Can2016), which could be employed in our approach. Additionally, it would be possible to assign embeddings to such prefixes as the average of the differences between the words that contain it and the same words without the prefix (e.g., v(impossible)-v(possible) where v denotes the word embedding). The prefix issue exposes a limitation for languages with extensive usage of prefixes, such as German and English. However, some languages such as Turkish do not have prefixes. We leave including prefixes in our approach as a future work.

5.1 Data

We used publicly available Morpho Challenge (Kurimo et al. Reference Kurimo, Lagus, Virpioja and Turunen2011) datasets for both training and testing. We did experiments on three languages: Turkish, English, and German. We built the training sets by choosing most frequent 10K and 50K words from the full word lists provided by Morpho Challenge. We combined the Morpho Challenge training and development sets for testing purposes. We did not use a separate development set for parameter tuning.Footnote ^f Manually collected newspaper archives were used for learning the Turkish word embeddings with a dimensionality of 200. We used word2vec (Mikolov et al. Reference Mikolov, Chen, Corrado and Dean2013) and its open-source Java implementation Deeplearning4J (Team Reference Team2016) to generate the word embeddings. English word embeddings are pretrained on Google News dataset and German word embeddings are pretrained on Wikipedia dataset. Both word embeddings have a dimensionality of 300. The size of all datasets are given in Table 2.

Table 2. Size of the datasets for Turkish, English, and German

5.2 Evaluation and results

We follow the same evaluation methodology provided by Morpho Challenge. A number of word pairs are randomly sampled from the result analyses that have at least one morpheme in common. For each sample word pair that really has a morpheme in common according to the gold standard segmentation, one point is given. Total number of points is divided by the total number of sampled word pairs. This gives the precision. For instance, given that the proposed segmentation of the English word access is acces $+$ s, two word pairs are selected randomly from the result set. Let us assume that access shares the morpheme acces with the word accesses. In this case, we obtain the word pair access – accesses. Let us also assume that access shares the morpheme $+$ s with the word books. This gives another word pair, which is access – books. Based on the gold standard segmentations, access_N, access_N + PLU, book_N + PLU, the pair access – accesses is correct (shared morpheme is access_N) and the pair access – books is incorrect (no common morpheme). Therefore, the precision for the word access becomes $1/2=50\%$ . The same is applied for recall by sampling word pairs randomly that have one morpheme in common according to the gold standard segmentations. It is checked from the result analyses whether they really have a morpheme in common. If the word pair has really a morpheme in common, one point is given and the total number of points is divided by the total number of sampled word pairs. Finally F-measure is computed as the harmonic mean of the precision and recall:

(18) \begin{equation} F-measure = \frac{2}{\frac{1}{precision}+\frac{1}{recall}}\end{equation}

We assigned $d = 0.25$ for the cosine similarity threshold to extract the potential morphemes in the initial step in the MLE model. We performed several experiments on the development sets (Kurimo et al. Reference Kurimo, Lagus, Virpioja and Turunen2011) in order to assign the threshold value. Such a threshold value is not needed in the MAP model.

We did experiments with four models we proposed in this article. The models are MLE unigram model (S-MLE unigram), MLE bigram model (S-MLE bigram), unigram MAP model (S-Unigram MAP), and bigram MAP model (S-Bigram MAP). Here S stands for semantic. We evaluated the initial segmentations obtained from the first step of the MLE model as a baseline model (S-Baseline). Additionally, we did two more iterations by updating the parameters with the new MLE estimates obtained from the Viterbi algorithm in the MLE model. The results are given as S-MLE unigram and S-MLE bigram with the number of iterations.

We compare our model with Morfessor Baseline (Creutz and Lagus Reference Creutz and Lagus2005b) (M-Baseline), Morfessor CatMAP (Creutz and Lagus Reference Creutz and Lagus2005a) (M-CatMAP), and MorphoChain system (Narasimhan et al. Reference Narasimhan, Barzilay and Jaakkola2015). Additionally, we compare our models with a baseline model where the most frequent morphemesFootnote ^g are split off from each word using the longest common subsequence from the end of the word until the word cannot be split anymore. We trained Morfessor Baseline and the Morfessor CatMAP by assigning the perplexity threshold and the iteration number as 10 (which are the default parameters). As for the MorphoChain, we assigned frequency threshold=0, vector size=200, minimum segmentation length=3, and the top affix selection number=100, which are also the default parameters of the model. For that purpose, we trained these models on the same training sets and tested on the same test sets for the evaluation. The results obtained from 10K of training set size are given in Tables 3, 4, and 5. The results obtained from 50K of training set size are given in Tables 6, 7, and 8.

Table 3. Results obtained from the 10K datasets for Turkish (Bold text indicates the highest score in the column.)

Table 4. Results obtained from the 10K datasets for English (Bold text indicates the highest score in the column.)

Table 5. Results obtained from the 10K datasets for German (Bold text indicates the highest score in the column.)

Table 6. Results obtained from the 50K datasets for Turkish (Bold text indicates the highest score in the column.)

Table 7. Results obtained from the 50K datasets for English (Bold text indicates the highest score in the column.)

Table 8. Results obtained from the 50K datasets for German (Bold text indicates the highest score in the column.)

The results obtained from 10K training set show that our MLE bigram model outperforms the other models on Turkish. Our MAP models also outperform Morfessor Baseline, Morfessor CatMAP, and MorphoChain; however, the MAP model does not perform as well as our proposed MLE models. This could be due to oversegmentation because of high cosine similarities obtained from the word embeddings. For English, again MLE bigram model outperforms our other proposed models. However, the best scores are obtained from Morfessor Baseline. Morfessor Baseline also outperforms the other models on German. Our MLE bigram model comes the second with further one iteration for the parameter estimates in German. However, our MAP models do not perform very well on German analogously to other two languages. This may be due to the heavy usage of compounds in German language. Since we postulate a semantic similarity between the derivations of the words, this may not apply to compounds. For example, hundemüde (meaning exhausted in English) is composed of two words: hund (dog) and müde (tired), which are not semantically similar. Moreover, connector sounds might be inserted while combining different words together in a compound in German language. For example, an extra “e” letter is inserted in the compound (das) schwein-e-fleisch (meaning pork meat). This hinders from obtaining the word embeddings of prefixes in a word in the initial segmentation. Our model does not perform well for compounds because of these two reasons.

If we compare the baseline models in Turkish, we can see that the baseline model that splits off the most frequent morphemes from the end of each word performs at par with the semantic baseline model (S-Baseline) (see Tables 3 and 6). However, the semantic baseline model performs slightly better than the baseline model using only the most frequent morpheme list on the 50K dataset. In English and in German, this difference is much more significant compared to Turkish. The results show that using semantic information through the word embeddings has a substantial impact on segmentation regardless of the morphological structure of the language. However, the final result is still far behind the other methods, such as Morfessor (Creutz and Lagus Reference Creutz and Lagus2002) because of the reasons mentioned above for German. In English, there is an extensive usage of irregular words and since we assume a concatenative morphology here, still the baseline model and other models fall behind the Morfessor (Creutz and Lagus Reference Creutz and Lagus2002). In Turkish, the semantic baseline model still performs very well compared to the other models; however, the baseline model using the most frequent morphemes performs as well as the semantic baseline model due to the concatenative nature of the baseline algorithm that allows to find many of the valid morphemes.

The results obtained from 10K and 50K training sets show how the size of the training set affects the accuracy of the models. Our MLE bigram model outperforms other models with a F-measure of 49.78% on Turkish with two further iterations in parameter estimation. Our unigram and bigram MAP models also perform better than other models such as Morfessor Baseline and Morfessor CatMAP on Turkish. Morfessor Baseline outperforms other models on English with a F-measure of 65.10%. However, our S-MLE unigram model comes the second and outperforms our other models. Bigram models improve the results significantly for morphologically complex languages, whereas it is not the case in morphologically poor languages such as English. Morfessor Baseline outperforms all models again with a F-measure of 47.58% on German. Differently from the experiments on smaller training sets, our bigram MAP model comes the second and both the semantic baseline model and the MLE models outperform Morfessor CatMAP and MorphoChain for German. This shows that our model performs better when it is trained larger training sets for three of the languages.

S-MLE method works well for concatenative morphology since it begins from the end of the word and gradually strips off the suffixes using the semantic information. English does not have a heavy morphology compared to Turkish so it cannot benefit from the semantic segmentation as much as Turkish. The same also applies for German but for a different reason. Since German relies heavily on compounds, it also cannot benefit from iterative segmentation from the end of the word as suggested in the baseline model.

To sum up, it shows that our model outperforms other models on Turkish in both training settings. This shows that using only semantic information gives promising results. Our MLE model performs the best among our models on English although the highest obtained scores fall behind that of Morfessor Baseline, which outperforms all models for English. However, our MLE scores are the highest among all models after Morfessor Baseline. For German, Morfessor Baseline is again the best, but this time our bigram MAP-based model performs better than other models.

5.2.1 Analysis of the results

Regarding the oversegmentation versus undersegmentation in different models, S-MLE model seems to be more robust to oversegmentation compared to the S-MAP model in Turkish and English. The precision and recall scores of S-MLE model are balanced, although the precision scores are slightly higher than recall scores for both English and Turkish. Morfessor CatMAP (Creutz and Lagus Reference Creutz and Lagus2005a) and MorphoChain (Narasimhan et al. Reference Narasimhan, Barzilay and Jaakkola2015) suffer from undersegmentation in all of the three languages, and therefore the recall scores obtained from Morfessor CatMAP and MorphoChain are very low and precision scores are comparably higher although the F-measure is fair average among all scores. In contrast, S-MLE model shows a different picture for German, where the recall scores are much higher than precision scores and thereby the model suffers from oversegmentation in German. This is actually a similar outcome for German language for all models except Morfessor Baseline (Creutz and Lagus Reference Creutz and Lagus2002) and MorphoChain (Narasimhan et al. Reference Narasimhan, Barzilay and Jaakkola2015) which give a higher precision unlike other models. This is again due to the heavy usage of compounds in German and therefore due to very long words that are prone to be oversegmented. As for S-MAP model, the precision and recall are less balanced compare to S-MLE model. There is a slight oversegmentation issue in both English and German, where the recall scores are substantially higher than precision scores. However, although the recall scores in Turkish are higher than precision scores for the S-MAP model, they are more balanced compared to English and German; therefore, oversegmentation is not an issue in Turkish for the S-MAP model.

Table 9. Some segmentation examples for the different models on English 10K results. The bold segmentation refers to the correct segmentation

Table 10. Some segmentation examples for the different models on Turkish 10K results. The bold segmentation refers to the correct segmentation

Table 11. Some segmentation examples for the different models on German 10K results. The bold segmentation refers to the correct segmentation

Some correct and incorrect examples in English, Turkish, and German results are given in Tables 9, 10, and 11 respectively. MAP model performs better than MLE model, where MLE model is prone to undersegmentation in German. However, MLE model performs better in English and Turkish compared to MAP model where the MAP model is prone to oversegmentation.

If we compare three languages from the morphological complexity point of view, we can easily claim that Turkish is the most morphological complex language out of these languages. This morphological complexity is due to the agglutination where each word is composed of many morphemes and the semantics is still preserved while deriving new word forms from a root. In English, each word usually takes 2.2 morphemes in average at the end; however, this number in Turkish is 3.3 and it is 3.1 in German in average (as we analyzed from the test sets). In German, this number includes also the number of the additional roots in each word. Therefore, we can argue that Turkish is comparably morphologically richer than both English and German. Table 9 shows that our models can still learn the correct segmentations in English if the roots did not change after the inflection or derivation, such as switch+ing, help+less+ness. However, we cannot expect from our models to work for irregular word forms since we aim to extract the morpheme boundary points in each word. English being a language with a heavy usage of irregular word forms is also not in the focus of our proposed models.

In German, the structure is completely different compared to English and Turkish. Words are usually composed of several different roots. Since we are aiming to take advantage of using the semantic similarity between derived word forms, this is not an advantage for German where semantically unrelated words can come together to produce a completely semantically unrelated word. The low German results also validate this situation.

We performed also boundary evaluation for the highest scores obtained for English and Turkish as applied in MorphoChain (Narasimhan et al. Reference Narasimhan, Barzilay and Jaakkola2015). The results are given in Table 12. All models are trained on 50K datasets and tested on Morpho Challenge gold segmentations. The boundary evaluation requires the surface forms to be evaluated. Since we could not find such a gold standard segmentation data for German that has only the surface forms, we did not include the German scores for this evaluation. MorphoChain evaluation gives higher scores compared to Morpho Challenge evaluation. There are several reasons for this (as explained in Can and Manandhar Reference Can and Manandhar2018). First, morpheme labels are used in Morpho Challenge evaluation, which increases the total number of points that is computed over all word pairs, and it lowers the scores. Second, only the gold segmentation that has the maximum match with the result segmentation is considered in MorphoChain evaluation and not all the given segmentations as in Morpho Challenge. Therefore, generally all scores are higher in MorphoChain evaluation.

Table 12. MorphoChain (Narasimhan et al. Reference Narasimhan, Barzilay and Jaakkola2015) evaluation for Turkish and English (Bold text indicates the highest score in the column.)

In order to compute the upper bound of the methods, we investigated the performance based on the existence of the words in the word embedding space. To this end, we performed segmentation if any prefix of the word has a word embedding without using any semantic similarity between the prefixes of the word, which will give an upper bound for the methods. The upper bound for the F-measure in Turkish is 40.53% if no semantic information is incorporated but only the existence of the prefixes of a word is exploited during segmentation, which is lower than 41.28% when the semantic similarity threshold is incorporated in the model. This shows that the incorporation of the semantic similarity improves the upper bound. We also investigated the MLE method’s upper bound analogously. We obtained a F-measure of 39.69% from the bigram MLE model using 50K training set in Turkish, whereas we obtained 48.36% from the bigram MLE model using the semantic similarity in the initial segmentations. Results are coherent with the baseline model and again it shows that the incorporation of the semantic similarity improves the scores significantly.

We analyzed how many segments in the gold standard are actually represented in their word embeddings to see if our method is applicable to other languages. To this end, we created a word list from the gold standard set by combining the prefixes of each given word. For example, we obtained book, booking, and bookings from the gold segmentation book+ing+s. As a result, in Turkish, 86% of words in the gold standard are represented in the word embedding space. This coverage seems to be reasonable to apply our method. In English, this coverage is at around 72% which is lower than Turkish and this coverage also might be hindering the performance of our method in English. Since the gold segmentations of the German words is not provided by Morpho Challenge, we could not compute the coverage of the words in the German word embedding space.

5.2.2 The effect of the word embedding dimensionality

In order to evaluate the effect of the word embedding dimensionality size on the performance of our models, we trained word2vec (Mikolov et al. Reference Mikolov, Chen, Corrado and Dean2013) for various dimensions on Turkish. We again used the manually collected newspaper archives. We performed experiments on the unigram and bigram MAP models for dimensions ranging from 50 to 300. Figure 3 shows the F-measure scores for different vector dimensions. The scores show that the optimal results are obtained from 100 dimensions. Smaller dimensions give poor results and larger dimensions also begin to give poorer results when the dimension size is increased gradually.

Figure 3. Results obtained from S-Unigram MAP and S-Bigram MAP models on Turkish 10K training dataset for different word embedding sizes.

5.2.3 The effect of the semantic similarity

Additionally, we performed another set of experiments to discover the effect of the cosine similarity threshold used in segmentation. Table 13 gives the F-measure scores obtained from S-Baseline and S-MLE models. The results show that a cosine threshold of $0.25$ gives the highest score for the MLE model, whereas $0.15$ gives the highest score for the baseline model.

Table 13. F-measures obtained from S-Baseline and S-MLE models on Turkish for different cosine similarity threshold values (Bold text indicates the assigned threshold value in the experiments.)

It should be noted that there is not any manually set threshold value in the MAP models. In order to measure the effect of the semantic features, we added a non-negative exponent to all the cosine terms in Equation (11). The results for different values of the exponent are given in Table 14 for Turkish. The results show that more the effect of the semantic similarity is incorporated, the higher the scores are in general. However, the scores are the highest when the exponent is reduced to $0.75$ . This also shows that incorporating semantic information helps in morphological segmentation.

Table 14. The F-scores on Turkish that show the effect of semantic similarity by changing the effect of the cosine terms in Equation (11) with various exponent values: $0.25$ , $0.5$ , $0.75$ , 1

5.2.4 The effect of using n-gram-level word embeddings

In order to investigate the effect of the word embeddings, we also used fastText (Bojanowski et al. Reference Bojanowski, Grave, Joulin and Mikolov2017) on S-Baseline model using a semantic similarity threshold of $0.25$ . FastText (Bojanowski et al. Reference Bojanowski, Grave, Joulin and Mikolov2017) is another method to learn word embeddings using the n-grams of each word rather than taking each word as a distinct unit during learning the embedding space as in word2vec (Mikolov et al. Reference Mikolov, Chen, Corrado and Dean2013). The results obtained for Turkish are given in Table 15. The results obtained from fastText are significantly higher than that of the ones obtained from S-Baseline that uses word2vec (Mikolov et al. Reference Mikolov, Chen, Corrado and Dean2013) embeddings in Turkish. We also analyzed the upper bounds of using both word2vec and fastText when the prefixes are segmented if their embeddings exist in the embedding space without incorporating any semantic information. The results obtained from fastText is still higher than that of word2vec. This shows that using n-gram-level word embeddings improves the scores significantly, since it also learns the word embeddings out of their n-grams and therefore it already incorporates the morpheme-level information within the embeddings. Moreover, using fastText embeddings shows the semantic similarity effect better than the word2vec embeddings.

Table 15. The F-scores on Turkish that show the effect of using n-gram-level word embeddings, that is fastText Bojanowski et al. (Reference Bojanowski, Grave, Joulin and Mikolov2017) (Bold text indicates the highest score in the column.)

Table 16. The computation times and memory requirements for the models (S-MLE Unigram, S-MLE Bigram, S-Unigram MAP, S-Bigram MAP) trained on Turkish 10K dataset