You are of course right. For doing just punctuation restoration, it would be much faster and more accurate to use some kind of sequene tagger (eg: BiLSTM CRF) and predict for each char ot each word (woild need ensemble) the punct that needs to be added.
But, my goal while writing this was to also do small grammatical and spelling corrections. This is why i went with seq2seq.