residual connection

Backlinks

the bitter lesson suggests you should try to reduce inductive bias as much as possible

architectural choices provide inductive bias (like using a residual connection, etc). Given infinite compute you'd do a universal relaxed architecture search with as few assumptions as possible.

see in context