This assumption couldn't be more false.
All three of my children were raised by me as a sole single parent. They visited their father together about once per month as was mandated by the court. Their father lived out of state, so he didn't assist with the children's daily needs. He would take time off work when they came to visit, and on very rare occasions he would come to visit them. He did pay child support after his wages were garnished, and occasionally he would even help pay 50% of an unexpected medical bill. That was quite honestly the FULL extent of his parenting responsibilities as he saw them.
When the children were old enough to make their own choices about visiting their dad, each of them began going for visits less and less often. At the same time, they also began pulling away from me. At times they would express deep seated anger about a variety of things. Sometimes the anger was described as disappointment that I didn't protect them from the abuse, and other times it was about one of my many failures as a parent. None of these discussions were easy. IN fact they were brutal, emotional and raw. In hindsight I can see that they were also critically informative and provided deep insight about the depth of the conflict they felt growing up with parents who didn't agree on most things.
Eventually the children began talking about the difficulties they experienced when their dad or I didn't prove to be trustworthy. These talks began with harsh accusations, but when they discovered I was strong enough to listen, the talks shifted to difficult discussions. The children started sharing their pain and the fear of trust that was created by feeling unable to trust anyone other than themselves. Slowly they shared about the experience of having new step siblings and girlfriends come in and out of their lives and the difficulties of spending holidays with various sets of family. Independently they even began talking about things they hadn't expressed before about how difficult some of their visits were and how terrifying it was to feel unable to tell me about those experiences due to fear of what might happen if someone found out. They eventually stopped talking about how I'd failed as a parent and started thanking me for teaching them the skills they needed to function independently and for letting them define the relationship with their dad on their own terms.