g4dn.xlarge T4 GPU - kernel dead when loading a 7b LLM

Question

I am loading the `mistral-7b` model from HF Transformers, however, I keep getting kernel dead error on jupyter notebook.

```
from transformers import AutoModelForSequenceClassification, AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained("mistralai/Mistral-7B-v0.1")
model
```

I am trying to inspect the `jupyter.log` stream in cloudwatch, however, I do not see any errors specific to the kernels. Where should I be looking for the `dead kernel` error?

Tomonori Shimomura · Answer

When you see "The kernel appears to have died" message and "Dead kernel" label, please try to run the same code in a Terminal session instead of in a Notebook. Then you could see the error message of the root cause. (Most likely the python process died due to native module crash). You would have to copy & paste Python code cells one by one.

g4dn.xlarge T4 GPU - kernel dead when loading a 7b LLM

Add your answer

Relevant content