I agree with ya, I'm not counting it out... just not counting on it, ya know? I don't think it that likely, but I could very easily be wrong (I see the possibility)
If Dumbledore and Harry are related, I think it's very distantly.
With the Weasleys and Harry being related, I think it's a somewhat far-fetched possibility, but still a possibility. In the "Molly's Second Cousin" thread, Krio mentions the possibility that second-cousin-the-accountant is Lily's father.
It would make sense that Harry is safe at Hogwarts and the Weasley's as well as the Dursley's as per the "mother's blood" prophesy, if the Weasley's and the Evans' are related. However, it's when Dumbledore is absent from Hogwarts that Harry has his run ins with Voldemort.
Keep in mind, though, that Dumbledore doesn't allow Harry to go directly to the Weasleys, or to stay with him, over the summer holidays. I don't really think that the "protection" he receives while at the Weasleys is anything more than Arthur and Molly's very observant eyes. They generally don't encourage the kids to go outside of the Burrow, and, lucky for Harry, he has no desire to. When he's with the Weasleys at Grimmauld Place, he's surrounded by very talented wizards (Mr and Mrs. Weasley, Bill, George and Fred, Sirius, Lupin) all the time, with other powerful wizards dropping in (Moody, Shacklebolt, Snape, even though he doesn't like him).
I don't think that it's ONLY the blood relative thing that keeps him safe, but I think that's a great protection he can have that no one else can, so it's important, and that's why he'd have to go to Privet Drive every summer for a little while. I still don't think, though, that he would have to be inside the house at Privet Drive, for no other reason than I think Dumbledore would have told him to stay inside, whether he did it or not is another story.
So, when he's away from "home" (Privet Drive) in the Muggle world, he's still supposedly safe (the dementor thing was a fluke, since they were most likely sent by the ministry, not Voldemort... the blood-relative thing protects him from Voldy). When he's away from Privet Drive at the Burrow or Grimmauld Place, he's safe because the Weasleys are keeping an eye on him and are very talented and powerful. When he's at Hogwarts, as long as Dumbledore is around, he's safe. When Dumbledore leaves, that's when we know to start worrying.