报错

Traceback (most recent call last):
    self.bertForToken = BertForTokenClassification.from_pretrained(self.opt.bert_model_dir, num_labels=self.opt.tag_nums)
  File "/home/xxx/anaconda3/lib/python3.7/site-packages/pytorch_pretrained_bert/modeling.py", line 600, in from_pretrained
    model = cls(config, *inputs, **kwargs)
  File "/home/xxx/anaconda3/lib/python3.7/site-packages/pytorch_pretrained_bert/modeling.py", line 1118, in __init__
    self.bert = BertModel(config)
  File "/home/xxx/anaconda3/lib/python3.7/site-packages/pytorch_pretrained_bert/modeling.py", line 704, in __init__
    self.embeddings = BertEmbeddings(config)
  File "/home/xxx/anaconda3/lib/python3.7/site-packages/pytorch_pretrained_bert/modeling.py", line 257, in __init__
    self.LayerNorm = BertLayerNorm(config.hidden_size, eps=1e-12)
  File "/home/xxx/anaconda3/lib/python3.7/site-packages/apex-0.1-py3.7.egg/apex/normalization/fused_layer_norm.py", line 133, in __init__
    fused_layer_norm_cuda = importlib.import_module("fused_layer_norm_cuda")
  File "/home/xxx/anaconda3/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 965, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'fused_layer_norm_cuda'

在使用pytorch_pretrained_bert报错No module named ‘fused_layer_norm_cuda’，其实是apex的问题

git clone https://github.com/NVIDIA/apex
cd apex
pip install -v --no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" ./

参考链接

https://github.com/NVIDIA/apex/issues/116
https://www.it610.com/article/1295293602620055552.htm
https://blog.csdn.net/Flying_sfeng/article/details/103314536