我正在尝试从文本blob(NER / IE)中提取位置,并尝试了许多解决方案,这些解决方案都太过于无用了spacy,斯坦福等等.

我的数据集确实只有80-90％准确(spacy就像70％),我遇到的另一个问题是没有概率对这些实体意味着什么,所以我不知道自信而且不能继续因此.

我尝试了一种超级天真的方法,将我的blob分成单个单词,然后将周围的上下文作为特征提取,也使用位置地名查找(30 / 40k位置地名)作为特征.然后我只使用了一个分类器(XGDBoost),结果在我用大约3k手动标记数据点(100k总共只有3k位置)训练分类器时效果更好.各州/国家的准确率为95％,城市为85％.

这种方法显然很糟糕,但为什么它比我尝试的一切都要好？我认为NER的黑盒方法对我的数据问题不起作用,我尝试了spacy自定义训练,它真的只是看起来不会起作用.对实体没有信心也是一种杀手,因为他们给你的概率几乎毫无意义.

是否有一些我可以更好地解决这个问题,以进一步提高我的结果？浅nlp为2/3 / 4克？我的方法的另一个问题是分类器的输出不是一些顺序实体,它实际上只是分类的单词blob,它们不知何故需要聚集成一个实体,即： – >旧金山,加利福尼亚州只是’城市’,’城市’,’0′,’州’,没有它们是同一个实体的概念

spacy示例：

示例blob：

About Us - Employment Opportunities Donate Donate Now The Power of Mushrooms Enhancing Response Where We Work Map Australia Africa Asia Pacific Our Work Agriculture Anti - Trafficking and Gender - based Violence Education Emergency Response Health and Nutrition Rural and Economic Development About Us Who We Are Annual Report Newsletters Employment Opportunities Video Library Contact Us Login My Profile Donate Join Our Email List Employment Opportunities Annual Report Newsletters Policies Video Library Contact Us Employment Opportunities Current Career Opportunity Internships Volunteer Who We Are Our History Employment Opportunities with World Hope International Working in Service to the Poor Are you a professional that wants a sense of satisfaction out of your job that goes beyond words of affirmation or a pat on the back ? You could be a part of a global community serving the poor in the name of Jesus Christ . You could use your talents and resources to make a significant difference to millions . Help World Hope International give a hand up rather than a hand out . Career opportunities . Internship opportunities . Volunteer Why We Work Here World Hope International envisions a world free of poverty . Where young girls aren ’ t sold into sexual slavery . Where every child has enough to eat . Where men and women can earn a fair and honest wage , and their children aren ’ t kept from an education . Where every community in Africa has clean water . As an employee of World Hope International , these are the people you will work for . Regardless of their religious beliefs , gender , race or ethnic background , you will help shine the light of hope into the darkness of poverty , injustice and oppression . Find out more by learning about the of World Hope International and reviewing a summary of our work in the most recent history annual report . Equal Opportunity Employer World Hope International is both an equal opportunity employer and a faith - based religious organization . We hire US employees without regard to race , color , ancestry , national origin , citizenship , age , sex , marital status , parental status , membership in any labor organization , political ideology or disability of an otherwise qualified individual . We hire national employees in our countries of operation pursuant to the law of the country where we hire the employees . The status of World Hope International as an equal opportunity employer does not prevent the organization from hiring US staff based on their religious beliefs so that all US staff share the same religious commitment . Pursuant to the United States Civil Rights Act of 1964 , Section 702 ( 42 U . S . C . 2000e 1 ( a ) ) , World Hope International has the right to , and does , hire only candidates whose beliefs align with the Apostle ’ s Creed . Apostle ’ s Creed : I believe in Jesus Christ , Gods only Son , our Lord , who was conceived by the Holy Spirit , born of the Virgin Mary , suffered under Pontius Pilate , was crucified , died , and was buried ; he descended to the dead . On the third day he rose again ; he ascended into heaven , he is seated at the right hand of the Father , and he will come again to judge the living and the dead . I believe in the Holy Spirit , the holy catholic church , the communion of saints , the forgiveness of sins , the resurrection of the body , and the life everlasting . AMEN . Christian Commitment All applicants will be screened for their Christian commitment . This process will include a discussion of : The applicant ’ s spiritual journey and relationship with Jesus Christ as indicated in their statement of faith The applicant ’ s understanding and acceptance of the Apostle ’ s Creed . Statement of Faith A statement of faith describes your faith and how you see it as relevant to your involvement with World Hope International . It must include , at a minimum , a description of your spiritual disciplines ( prayer , Bible study , etc . ) and your current fellowship or place of worship . Applicants can either incorporate their statement of faith into their cover letter content or submit it as a separate document . 519 Mt Petrie Road Mackenzie , Qld 4156 1 - 800 - 967 - 534 ( World Hope ) + 61 7 3624 9977 CHEQUE Donations World Hope International ATTN : Gift Processing 519 Mt Petrie Road Mackenzie , Qld 4156 Spread the Word Stay Informed Join Email List Focused on the Mission In fiscal year 2015 , 88 % of all expenditures went to program services . Find out more . Privacy Policy | Terms of Service World Hope Australia Overseas Aid Fund is registered with the ACNC and all donations over $2 are tax deductible . ABN : 64 983 196 241 © 2017 WORLD HOPE INTERNATIONAL . All rights reserved .'

结果：

('US', 'GPE')
('US', 'GPE')
('US', 'GPE')
('the', 'GPE')
('United', 'GPE')
('States', 'GPE')
('Jesus', 'GPE')
('Christ', 'GPE')
('Pontius', 'GPE')
('Pilate', 'GPE')
('Faith', 'GPE')
('A', 'GPE')

解决方法:

即使是最好的基于深度学习的NER系统,这些天也只能达到92.0的F1.基于深度学习的系统(CNN-BiLSTM-CRF)应该优于Stanford CoreNLP的普通CRF序列标记器.最近,在整合语言模型方面取得了更多进步.您可能想看看AllenNLP.

但如果你想要99.0％的超高精度,那么你将暂时整合基于规则的方法.

我认为基于规则的处理可能会有所帮助.例如,您可以编写一个模式,说“城市城市O,州”应该合并为一个实体.此外,您可能需要考虑丢弃未出现在您的位置/地点字典中的实体.或者丢弃不在位置字典中但属于其他类型的实体.但我发现很难相信很多未知的字符串序列是你关心提取的位置地名.我认为人名最有可能不在词典之外.

如果您下载软件,UIUC的NLP工具中会包含一些字典.

当运行StanfordCoreNLP时,使用ner,regexner,entitymentions注释器将允许将连续的NE标签自动分组到实体中.有关管道的完整信息：https://stanfordnlp.github.io/CoreNLP/cmdline.html

此外,请记住,这些系统的开箱即用版本通常都是针对过去15年来的新闻文章进行培训的.重新训练靠近您的设置的数据至关重要.最终你可能最好只编写一些基于字典的提取规则.

您可以查看Stanford CoreNLP的TokensRegex和RegexNER功能,了解如何将Stanford CoreNLP用于此目的.

TokensRegex：https://nlp.stanford.edu/software/tokensregex.html
RegexNER：https://nlp.stanford.edu/software/regexner.html