You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hi,
First of all SymSpell is damn fast and kind of does my job for spell correction but the issue I am facing is when my application user intentionally type any completely wrong word or sentence Symspell would would come up with a right word for it which can be avoided
Example
User types: avedoamlkejuike...
Syspell: a video am like juice keen...
Something like this which is totally irrelvant for my usecase
So how can I solve this just by using Symspell??
Thanks in advance
The text was updated successfully, but these errors were encountered:
This is a common problem. It is not only that the user intentionally types something wrong, but there are always unknown words in the input text that are not in the dictionary.
So how we can distinguish a non-existing or unknown word from a misspelled word?
Solution1: restrict the maximum edit distance (the number of splits + the numer of spelling corrections) within a sliding text window. If a string needs to be split into too many small words, and almost all of the sub-words need an additional spelling correction we can assume that this is an unknown/non-existing word.
Solution2: Use n-gram probabilities or Markov-chains. The co-occurrence of words is not random. Some words are more likely to occur together in a sentence than others, some words are frequently follow each others, others never. if the n-gram probabilities of the split and corrected words are below a certain threshold, we can assume that this is not a genuine correction, but an unknown/non-existing word.
Both solutions are currently not part of SymSpell and need to be implemented as an extension or by modifying the SymSpell code.
Hi,
First of all SymSpell is damn fast and kind of does my job for spell correction but the issue I am facing is when my application user intentionally type any completely wrong word or sentence Symspell would would come up with a right word for it which can be avoided
Example
User types: avedoamlkejuike...
Syspell: a video am like juice keen...
Something like this which is totally irrelvant for my usecase
So how can I solve this just by using Symspell??
Thanks in advance
The text was updated successfully, but these errors were encountered: