报错信息
RuntimeError: "LayerNormKernelImpl" not implemented for 'Half'
更加完整的报错信息:
/home/anaconda3/envs/py385/lib/python3.8/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
1188 if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
1189 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1190 return forward_call(*input, **kwargs)
1191 # Do not call functions when jit is used
1192 full_backward_hooks, non_full_backward_hooks = [], []
/home/anaconda3/envs/py385/lib/python3.8/site-packages/torch/nn/modules/normalization.py in forward(self, input)
188
189 def forward(self, input: Tensor) -> Tensor:
--> 190 return F.layer_norm(
191 input, self.normalized_shape, self.weight, self.bias, self.eps)
192
/home/anaconda3/envs/py385/lib/python3.8/site-packages/torch/nn/functional.py in layer_norm(input, normalized_shape, weight, bias, eps)
2513 layer_norm, (input, weight, bias), input, normalized_shape, weight=weight, bias=bias, eps=eps
2514 )
-> 2515 return torch.layer_norm(input, normalized_shape, weight, bias, eps, torch.backends.cudnn.enabled)
2516
2517
RuntimeError: "LayerNormKernelImpl" not implemented for 'Half'
解决方法
最近在研究大模型,遇到这个问题,其实报错的原因只是因为我在推理的时候,忘记将模型放在GPU上,所以解决也很简单:
model.to('cuda:0')
把模型放在GPU上即可解决。
参考:https://github.com/huggingface/transformers/issues/21989
报错原因
更详细一点解释,是因为模型做了半精度,即fp16,也就是说在前面的代码中,你应该是执行过这一句:
model = model.half()
而fp16,在CPU上是不能生效的,如果用CPU推理,只能老老实实的用fp32。
补充
如果你是在部署stable-diffusion的时候遇到这个错误,可以参考hf的这个issue:
https://huggingface.co/CompVis/stable-diffusion-v1-4/discussions/64