(Spoilers, extended) Why do people think Samwell is an author self-insert?

I've often seen it said that George R R Martin wrote himself into the story with Samwell Tarly.

Can anyone explain this to me?

As far as I can tell, the only similarities between author and character are a love of books and being overweight.

Sam self identifies as a coward and has been downtrodden and picked on his entire life. Do we really think George feels this way about himself?