Man, your story is hard to read, as the AP did sneak into your family like a snake and actually did achieve his goal at the end. He and your wife are horrible people. Sorry.
I always wonder about those kinds of people, there morals, upbringing and their inner monologue. It's like a different language to me.
They have been playing you hard. No sex since he came into your life, sneaking into your family, your kids probably know him pretty well and are used to him (which probably was your wife's plan from the get go, to make the kids get used to him).
But how is it that you let a man look after your kids you hasn't kids on his own, how did you wife make you do that, allow him get so close?
I do think your wife took advantage of you being on the spectrum. Happened to me as well.
Aren't you worried that he is going to be a bigger part in your kid’s life?
For the peace of your mind, you should have tried to find some evidence of the affair (her phone for instance or a pi). There is always evidence.
Contrary to what you say I don't think he is a predator, whatever that means.
She is not the "victim" of a predator. She is a grown human being with agency. She made her choice.
But good for you that you put your foot down at the end! That’s a hard thing to do, proud of you brother!
I wouldn't tell the kids; you have no proof and they are too young! But I would the family and friends know; problem is you have no evidence. But you could describe the past 1,5 years to them and tell them: "you just wait, she is going to end up with him".