

This is often a point of confusion, but human sex is binary. There’s edge cases that require clarification as to how they fit into the binary, but don’t disprove it.
Human sexuality overall is complex and that’s why we differentiate gender from sex. The sex binary and gender spectrum complement each other though, and don’t clash.
If you’re interested in learning more, here’s some background reading:
https://en.wikipedia.org/wiki/Gonochorism
We fall into that category, where we have two body plans, each organized around producing either sperm or ova. Other species have more body plans, such as recognizably distinct males, females, and hermaphrodites:
https://en.wikipedia.org/wiki/Trioecy
Those species are a good contrast. Humans don’t have that variation, and so sex is binary in humans.
There’s literature that explains this specifically in detail, though most of it doesn’t really explicitly talk about it, much like math papers don’t generally explain that integers can be added together.
Do you have a particular edge case in mind? One that’s commonly brought up is https://en.wikipedia.org/wiki/Ovotesticular_syndrome, but that doesn’t fall outside the sex binary. Having a bit of nonfunctional tissue doesn’t affect one’s sex.
Colors aren’t a great analogy either, because in anisogamous species, gametes are strictly binary. There’s sperm and ova, with 0 overlap and 0 other options. “Purple gametes” just don’t exist.
This also isn’t my opinion, this is the accepted definition in the field of biology.