Prisoner of Azkaban takes the top spot on lots HP lists for both the film and the book. However, I have one major issue with it.
The time travel is wrong. I am a stickler for time travel rules. There's a way in which to do it, and a way in which human's have no choice and paradoxical world ending implications are prevalent. You cant have the things that happen after the characters go back in time happen the first time 'round. That's just wrong. What if they didn't do it the second time? What would happen then? It has to be that when you go back in time, you rewrite the subsequent future. Not just live it from a different perspective.
Whatever. This is a point I've made many times to many people and, for the most part, they don't care. But it's my biggest problem with Prisoner. That, and I think they upped Hermione's snob quotient so high, she's completely unlikeable and nothing like Hermione. "RONALD!"
Sirius is fantastic though - and Gary Oldman is fantastic at making him fantastic. Lupin too. And David Thewlis too. Neither of them get enough play in the later films.
It also has the best ending. And it's the last one that isn't terribly depressing. It's one of the better Harry Potter "films" but it's not one of my personal faves.