There is a very real misconception about how the Pope is Infallible. Indeed, it is not only held by non-catholics, but many Catholics as well. But the truth is that the Pope can only be Infallible when certain conditions are met. And while the Church does not list how many times those conditions have been met, most theologians only agree on 2 instances.
The first was in 1854 When Pope Pius IX came out with the Immaculate Conception of Mary (another misconception for a later day). The Second was in 1950 when Pope Pius XII spoke on the Assumption of Mary.
Some theologians believe there are more instances, and indeed there maybe. But it was the Vatican Council I that came up with the official doctrine so any Infallible statements prior to that are argued upon, even though the doctrine has its roots in the Gospels and Paul's letters. So while the Pope may have been speaking infallibly in the past, some dont recognize it until it became an official Teaching of the Church.
So it is evident that not everything the Pope says is infallible, and indeed from the sparsity of times it is even thought to have been done, most of the time the Pope is not infallible.
For the Pope to make an infallible statement, some conditions have to be met. First he must be speaking ex cathedra, or for the whole Church. Second it cannot contradict any previous teachings of the Church. Third, he must basically state that he is resolving some point of Doctrine with absolute authority, and finally, he must be addressing the entire Church (the entire Church being all members of faith).
So while some Priests last year were railing against John Kerry and his pro-life position on abortion, no Pope has ever spoken Ex Cathedra on the subject, and therefore it is a teaching of the Church, not a doctrine of the Church. And that in itself is a critical distinction. For any Catholic can doubt a Teaching of the Church, and still remain a member in good standing. But you cannot doubt a Doctrine of the Church and still remain a Catholic (Doubting in and Doubting out).
The Doctrine of Infallibility is a powerful one, and while the Church has been abusive in many of its practices in the past, that is one it has not abused. And for Good reason. For the whole concept behind Infallibility is that you are speaking with the Voice of God. And even corrupt Popes did not want to cross Him!