T4 inference performance

Question

Hi, What is the code used to obtain the performance numbers of T4 gpus for inferencing. Thanks

Dell-DylanJ · Accepted Answer

I heard back sooner than expected. This is the information provided:

User has to download TensorRT and compile the code for trtexec executable which will be located in the (TensorRT-5.0.2.6/targets/x86_64-linux-gnu/samples/trtexec).
cudNN should be downloaded and included in the path.
Datasets were downloaded from the following locations,
- alexnet: https://github.com/BVLC/caffe/tree/master/models/bvlc_alexnet
- googlenet: https://github.com/BVLC/caffe/tree/master/models/bvlc_googlenet
- vgg_16 and vgg_19: http://www.robots.ox.ac.uk/~vgg/research/very_deep/
- resnet_50, resnet_101 and resnet_152: https://github.com/KaimingHe/deep-residual-networks

Dell-DylanJ · Answer

Hello,

Are you looking for CLI commands to view load on a GPU? I'm not seeing anything of the sort, if so. The developer and user guides are linked below, they may be of some use.

https://docs.nvidia.com/deeplearning/sdk/tensorrt-developer-guide/index.html#python_topics

https://docs.nvidia.com/deeplearning/sdk/tensorrt-archived/tensorrt_210/tensorrt-user-guide/index.html

antojohn · Answer

Hi, I see DELL published some T4 performance results at https://www.dell.com/support/article/in/en/indhs1/sln316556/inference-using-the-nvidia-t4?lang=en I would like to know what was the benchmark used to generate these numbers ? So that I can try to recreate it.

Dell-DylanJ · Answer

If I'm understanding you properly, it sounds like the answer is in the article. That they ran pretrained models on a select number of images and plotted for time. I'll see if I can reach one of the authors of the article for more information on what they did, though. It may take some time to get a response one of them, though.

antojohn · Answer

Thanks for the reply.. this helps !!

Rack Servers

T4 inference performance

Was this post helpful?