⇩ Markdown

the bitter lesson suggests you should try to reduce inductive bias as much as possible

the bitter lesson suggests you should try to reduce inductive bias as much as possible.

architectural choices provide inductive bias (like using a residual connection, etc). Given infinite compute you'd do a universal relaxed architecture search with as few assumptions as possible.