Let's try the following to help clarify things. Do you think that it would be an Constitutionally unacceptable violation of privacy if there were a state law prohibiting oral contraceptives, vasectomies, or facelifts? This is the original context in which the right of privacy was brought up to the Supreme Court, the Griswold case, linked to here.
What's missing from the Griswold case, of course, is the presence of a fetus whose rights, arguably, merit protection. My view is that if you take the fetus out of the equation, Roe was correct, just as if you take the right of the Negro slave to liberty out of the equation in Dred Scott, then Dred turns out to be right.