Routes towards greater AI efficiency

  1. Reducing total number of paramters
  2. Reducing total number of required training examples.

Research routes

  1. There must be a correlation between the number of model paramters and the complexity of the dataset you wish to generalize over. In this reguard, a fixed number of parameters at the start of training doesnt make sense. What are the chances you struckl the Oscams Razor parameter count first try? What if the number of paramters was dynamic, and the model decided how complex it should be?.