neuron compiling a bert model

0

Hi, I want to neuron compile a bert large model(patentbert from google) which has sequence length 512. How do I do this? Also I want to call the model as before or need to know what I should change while calling it. I compiled the anferico/bert-for-patents using the instructions here https://awsdocs-neuron.readthedocs-hosted.com/en/latest/src/examples/tensorflow/huggingface_bert/huggingface_bert.html. Also is it the same to use patentbert from google and huggingface anferico/bert-for-patents ??

The saved model cli output for

  1. Google's patentbert MetaGraphDef with tag-set: 'serve' contains the following SignatureDefs:

signature_def['serving_default']: The given SavedModel SignatureDef contains the following input(s): inputs['input_ids'] tensor_info: dtype: DT_INT64 shape: (-1, 512) name: input_ids:0 inputs['input_mask'] tensor_info: dtype: DT_INT64 shape: (-1, 512) name: input_mask:0 inputs['mlm_positions'] tensor_info: dtype: DT_INT64 shape: (-1, 45) name: mlm_positions:0 inputs['segment_ids'] tensor_info: dtype: DT_INT64 shape: (-1, 512) name: segment_ids:0 The given SavedModel SignatureDef contains the following output(s): outputs['cls_token'] tensor_info: dtype: DT_FLOAT shape: (-1, 1024) name: Squeeze:0 outputs['encoder_layer'] tensor_info: dtype: DT_FLOAT shape: (-1, 512, 1024) name: bert/encoder/layer_23/output/LayerNorm/batchnorm/add_1:0 outputs['mlm_logits'] tensor_info: dtype: DT_FLOAT shape: (-1, 39859) name: cls/predictions/BiasAdd:0 outputs['next_sentence_logits'] tensor_info: dtype: DT_FLOAT shape: (-1, 2) name: cls/seq_relationship/BiasAdd:0 Method name is: tensorflow/serving/predict

MetaGraphDef with tag-set: 'serve, tpu' contains the following SignatureDefs:

signature_def['serving_default']: The given SavedModel SignatureDef contains the following input(s): inputs['input_ids'] tensor_info: dtype: DT_INT64 shape: (-1, 512) name: input_ids:0 inputs['input_mask'] tensor_info: dtype: DT_INT64 shape: (-1, 512) name: input_mask:0 inputs['mlm_positions'] tensor_info: dtype: DT_INT64 shape: (-1, 45) name: mlm_positions:0 inputs['segment_ids'] tensor_info: dtype: DT_INT64 shape: (-1, 512) name: segment_ids:0 The given SavedModel SignatureDef contains the following output(s): outputs['cls_token'] tensor_info: dtype: DT_FLOAT shape: unknown_rank name: TPUPartitionedCall:5 outputs['encoder_layer'] tensor_info: dtype: DT_FLOAT shape: unknown_rank name: TPUPartitionedCall:6 outputs['mlm_logits'] tensor_info: dtype: DT_FLOAT shape: unknown_rank name: TPUPartitionedCall:4 outputs['next_sentence_logits'] tensor_info: dtype: DT_FLOAT shape: unknown_rank name: TPUPartitionedCall:7 Method name is: tensorflow/serving/predict

  1. Neuron compiled anferico/bert-for-patents MetaGraphDef with tag-set: 'serve' contains the following SignatureDefs:

signature_def['__saved_model_init_op']: The given SavedModel SignatureDef contains the following input(s): The given SavedModel SignatureDef contains the following output(s): outputs['__saved_model_init_op'] tensor_info: dtype: DT_INVALID shape: unknown_rank name: NoOp Method name is:

signature_def['serving_default']: The given SavedModel SignatureDef contains the following input(s): inputs['input_1'] tensor_info: dtype: DT_INT32 shape: (-1, 512) name: serving_default_input_1:0 inputs['input_2'] tensor_info: dtype: DT_INT32 shape: (-1, 512) name: serving_default_input_2:0 The given SavedModel SignatureDef contains the following output(s): outputs['output_1'] tensor_info: dtype: DT_FLOAT shape: (-1, 2) name: StatefulPartitionedCall:0 Method name is: tensorflow/serving/predict

Concrete Functions: Function Name: 'call' Option #1 Callable with: Argument #1 DType: list Value: [TensorSpec(shape=(None, 512), dtype=tf.int32, name='input_1'), TensorSpec(shape=(None, 512), dtype=tf.int32, name='input_2')]

Function Name: '_default_save_signature' Option #1 Callable with: Argument #1 DType: list Value: [TensorSpec(shape=(None, 512), dtype=tf.int32, name='input_1'), TensorSpec(shape=(None, 512), dtype=tf.int32, name='input_2')]

Function Name: 'aws_neuron_function' Option #1 Callable with: Argument #1 args_0 Argument #2 args_0_1

Function Name: 'call_and_return_all_conditional_losses' Option #1 Callable with: Argument #1 DType: list Value: [TensorSpec(shape=(None, 512), dtype=tf.int32, name='input_1'), TensorSpec(shape=(None, 512), dtype=tf.int32, name='input_2')]

This is the stack trace when I replace the original bert model with neuron compiled model. Traceback (most recent call last): File "/home/ubuntu/venv/lib/python3.7/site-packages/tensorflow/python/eager/function.py", line 1645, in _call_with_flat_signature args.append(kwargs.pop(compat.as_str(keyword))) KeyError: 'input_1'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/ubuntu/venv/lib/python3.7/site-packages/tensorflow/python/eager/function.py", line 1615, in _call_impl cancellation_manager) File "/home/ubuntu/venv/lib/python3.7/site-packages/tensorflow/python/eager/function.py", line 1651, in _call_with_flat_signature raise TypeError(f"{self._flat_signature_summary()} missing required " TypeError: signature_wrapper(input_1, input_2) missing required arguments: input_1, input_2.

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "main.py", line 319, in <module> verbose) File "main.py", line 223, in process_each_project verbose=verbose) File "/home/ubuntu/ranking_pipeline/rank_utils.py", line 437, in rank response, inputs, _ = self.model.predict(search_sentences) File "/home/ubuntu/ranking_pipeline/bert_utils.py", line 292, in predict inputs['mlm_ids'], dtype=tf.int64), File "/home/ubuntu/venv/lib/python3.7/site-packages/tensorflow/python/eager/function.py", line 1601, in call return self._call_impl(args, kwargs) File "/home/ubuntu/venv/lib/python3.7/site-packages/tensorflow/python/eager/function.py", line 1617, in _call_impl raise structured_err File "/home/ubuntu/venv/lib/python3.7/site-packages/tensorflow/python/eager/function.py", line 1611, in _call_impl cancellation_manager) File "/home/ubuntu/venv/lib/python3.7/site-packages/tensorflow/python/eager/function.py", line 1688, in _call_with_structured_signature self._structured_signature_check_missing_args(args, kwargs) File "/home/ubuntu/venv/lib/python3.7/site-packages/tensorflow/python/eager/function.py", line 1707, in _structured_signature_check_missing_args raise TypeError(f"{self._structured_signature_summary()} missing " TypeError: signature_wrapper(*, input_2, input_1) missing required arguments: input_1, input_2.

Thanks in advance Warm regards Ajay

질문됨 일 년 전370회 조회
1개 답변
0

Due to the 512 length limit, you have basically two options:

  1. You cut the longer texts off and only use the first 512 Tokens. The original BERT implementation (and probably the others as well) truncates longer sequences automatically. For most cases, this option is sufficient.
  2. You can split your text in multiple subtexts, classify each of them and combine the results back together ( choose the class which was predicted for most of the subtexts for example). This option is obviously more expensive.

You can read about these options here.

is it the same to use patentbert from google and huggingface anferico/bert-for-patents ?

Yes, both versions will work with the given use case. You can look at a simplified comparison between the services here.

profile pictureAWS
답변함 일 년 전

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인

관련 콘텐츠