Thread: Syntax Highlighting for English?

You're discussing part-of-speech tagging of natural language here, which is considered "solved" (almost solved) and for which there exist very robust programs already. I linked to some part of speech tagging methods in post #31, namely Brill and HMM. They basically have a forest and maximize the probability of several trees producing the given phrase.

This already works very well (depending on the vocabulary, slang, out-of-dictionary forms etc). What the discussion should focus on is, if in natural language, this a) is useful and b) takes away the subtlety of our language, like jokes, literature or even "sexual" ambiguous remarks.

Originally Posted by Decrypt

The idea is that, if this highlighter is to work, I think that it'd have to use general grammatical rules instead of a strict set of uses for each word as mentioned above. However, to start, you'd have to have some set of words whose use is iron-clad, and, in the end, you'd probably have to use both a set of grammatical rules and a database of words and their uses to implement it properly.

But now we're way out of the realm of what could be coded up "in a few hours." We're talking full part-of-speech tagging. People spend years writing PhD theses on just a little tiny corner of the theory.

The original argument was whether highlighted text would be easier to read. Part of speech tagging is possible and useful, but making sentences easier to read is not one of its uses.