I think the only really weak titles in the X series were X6, X7 and Command Mission. X6 was just so badly translated and had a bunch of cheap deaths, and X7 had some of the worst controls ever in a Mega Man game. Command Mission wasn' that bad, but the game play got boring after a while. X1, X2 and X3 were lots of fun to play, and were some of the best Mega Man games out there. X4 was good, but not as good. X5 suffered from the same problem. X8 is lots of fun to play, and has great reply value, but some of the stages pissed me off.
Keep in mind, I like all the X games, except maybe X7, but after X4, the decline becomes pretty noticeable.