Love of self is not vanity. It indicates you respect your beliefs, thoughts, and actions. It is a sign of self-respect. Those who respect themselves automatically know how to respect others. You must respect yourself, feel good about yourself. Your internal feeling will then automatically get translated externally to others.
If you truly loved yourself, you could never hurt another.