Training NER model using Stanford Core NLP CRF Classifier

Named-Entity Recognition (NER) is one of the most popular NLP tasks. It’s popular because it produces annotation result that can be used directly (eg. extracting people name from text) or indirectly (eg. extracting feature for classification task).

One of the easiest way to do it is by downloading and using latest Stanford Core NLP suite from to train a NER model using our own dataset. Here are some steps to do it.

First, prepare necessary files:

  1. Training dataset. List of word tokens annotated with their Named-Entity class (eg. PERS for Person entity). Example: jane-austen-emma-ch1
  2. Training properties. Training cofiguration properties for a CRF classifier (eg. input file(s), output file, features .etc). Example: jane-austen
  3. Test dataset. Similar to training dataset but with different list of tokens. Example: jane-austen-emma-ch2

If you have datasets in ENAMEX or Open NLP format, you can use these simple python scripts or to convert them

Enter stanfordnlp unzipped directory and run this command to train model:

java -cp "*" -prop jane-austen.prop

The result will show the output model file name ner-model.ser.gz:

[main] INFO - CRFClassifier training ... done [2.1 sec]. 
[main] INFO - Serializing classifier to ner-model.ser.gz... 
[main] INFO - done.

To test the model against the test file run this command:

java -cp "*" -loadClassifier ner-model.ser.gz -testFile jane-austen-emma-ch2.tsv

The result shows the performance of the model. In this case it achieves 82% precision, 72% recall and 77% F-measure:

[main] INFO - CRFClassifier tagged 1999 words in 1 documents at 6305.99 words per second. 
[main] INFO - Entity	P	R	F1	TP	FP	FN 
[main] INFO - PERS	0.8205	0.7273 0.7711	32	7	12 
[main] INFO - Totals	0.8205	0.7273	0.7711	32	7	12

You can also train the classifier using for language other than English as long as you provide proper training/testing dataset.

Cheers! 🍻


Leave a Reply

1 Comment threads
1 Thread replies
Most reacted comment
Hottest comment thread
2 Comment authors
yohanes.gultom@gmail.comHima Recent comment authors
newest oldest most voted
Notify of

Thanks for post. Can’t find Input files.