Make a benchmarks

In this week, I got the inference latency time for each model with a small dataset. (under 40 samples)

As a result of checking roughly, the value of latency changed the most according to input shape, and the filter size also affected it a lot.

And now I’m currently making larger dataset with below’s combinations.

kernel_list = [2*i+1 for i in range(3)]                     # 1, 3, 5
filter_list = [2**i for i in range(4, 8)]                   # 16, 32, 64, 128
input_hw = [8, 16, 32, 64]
input_channels = [16, 32, 48]

To Check List

• Start github repo
• Simple regression
  • How to make simple regression? Below is the idea…. Is that correct?
  • \[latency = ax_1 + bx_2 + cx_3 + .... + k_n x_n + bias\]
  • $x_n$ : one layer’s latency time
  • $k_n$ : weight (?) -> value that need to train
• CSV to SQLite?
  • Still working on csv….
  • I’m just thinking of saving all the results in csv first and then making a database later.