My earliest childhood memory was being babysat when I was around 4 or 5 and making a fort out of sticks with my babysitter and my brother. I remember having a good time and being upset when it fell down. There's nothing really extraordinary about this memory nor does it seem to really show the identity of who I am now.
I also have strange, seemingly useless early memories. It definitely seems that there are a great deal of "useless" memories retained from childhood. This could possibly be a result of the mind's developmental cycle, for example, just warming up, or becoming efficient at retaining long-term memories. Another possibility is that these seemingly "useless" memories were actually of some sort of significance in a subtle way that has now been forgotten. In other words, maybe you changed or were altered in some specific way at this point in your life, but you only remember the visual details!