Well, becised the stress and lots of other things in his life - having Smart Parents does not garantee a smart kid! Also, a lot of times smart parents start teaching their kids early on, and Harry's parents are not there to make him learn. And, transfering power into something does not equal trasnferring smartness.
But the Main Reason is - u can see that Harry is good in the subjects that he likes. So he just doesnt bother paying extra attention in other classes. If you are comparing him to Hermoine, well she spends a lot of free time learning things and reading, she is motivated by just about any subject. DADA motivates Harry - so he excells in it (even better then Hermoine).