Be more strict about the no contact. My ex is a lot like yours (but worse, believe it or not), and I've survived almost 15 years of shared custody with him via strict no contact. We were in a strictly email/text only about the child set-up until just last week, when I had to limit it even further to email only because he got nasty in a text, blaming me about things he is completely guilty of, very similar to what you just went through. You can't win an argument with crazy. You can simply disengage.
Usually things are great for me re-him because there really is no contact, and that feels wonderful. No more gaslighting, no more verbal abuse, no more things too crazy to go into here, no more being blamed for what a POS partner and father he is.
Drop-offs/pick-ups are not in person, but instead at a neutral public parking lot where the child walked between the cars while we stayed in our cars. I recommend that for you as soon as the kids are old enough to walk between two cars that are parked next to each other. Until then, I recommend still doing trading off kids in a neutral public place (leads to no emotional or power or other people issues re being at his house or yours), even if you have to walk them up to him until they're old enough to do it alone.
That someone else is going to be raising my babies.
You are 100% raising your children. Daycare is not a substitute parent, any more than their teacher at school will be a substitute parent. Having to work and use daycare isn't as pleasant as being a SAHM, I'm sure (like most mothers, I never even had that opportunity), but it doesn't make you any less of a parent.