I think an important factor that could influence attachments would be either a genetic or psychological disorder. For a family that has a child with mental retardation, sometimes it doesn't matter how caring and attentive the parents are; the child still won't be able to form a secure attachment. If the same parents had a child who didn't have mental retardation, and raised him in the same environment, it's more likely that child would have a secure attachment. So, sometimes it really doesn't even matter the upbringing of a child, they are genetically predisposed to act a certain way.
I agree with you that genetic or psychological disorders could affect attachment. In these cases, it cannot be helped and it is very unfortunate. In cases like autism, social interaction is already affected from the beginning because of a disorder in neural development.