My husband and I are building a house so moved in with my parents for a few months. They live in a new house, are on a private well and have a water softener. The problem is that the water seems to be TOO soft. When we wash our hands, we end up having to wipe some soap off on the towel. I can never get my hair or face fully rinsed in the shower. No matter how little soap is in the dishwasher, it never fully rinses the dishes.
Both my husband and I have lived in other places with soft water, but have never had this problem.
The goal of a softener is to reduce or eliminate hardness, that is dissolved minerals. Check yours and make sure that hardness has not been reduced to zero. It should be about 100 parts per million.
Aside from what you describe, zero will also cause the water heater to wear out in record time.