It would be great to have Readers/Writers instead of []byte when (de-)serializing data.
For bigger files, like you mentioned BitTorrent (https://medium.com/perlin-network/noise-an-opinionated-p2p-networking-stack-for-decentralized-protocols-in-go-bfc6fecf157d), it would be good to not blow up the heap, but stream the data directly (e.g. using io.Copy).
Maybe we could end up with something like

type SerializableStream interface {
	Marshal() io.ReadWriter
}