My earliest memory is when I was about three years old. My family went to Vermont for a wedding. I don't remember the wedding, but I remember the hotel we were staying at. I was on the edge of the pool with my swimmies on and I was playing with toy cars. I also remember swimming in the pool with the help of my mom and floating on a large inflatable raft. This memory is particularly memorable because I am pretty sure it was my first time every swimming in a large pool. I think I am still the same person because I remember being very excited about swimming and I still get excited about going swimming today.
It sounds like this struck you as particularly memorable because of how much enjoyment you associated with swimming. Swimming often since then would increase the amount of neurons in your brain devoted to swimming; association would lead to a clearer memory of your first experience swimming.