There is no more tragic figure in Talmudic literature than that of Acher, the Other. Destined for greatness, the teacher, colleague and friend of Rav Meir left the path of Torah, becoming perhaps the most famous apostate in Jewish history. Yet despite, or perhaps because of, his rejection of Judaism he was figure of great interest to his colleagues. They implored him to return but he was unwilling or perhaps unable, to return. "Return return my people - all except for Acher." (Chagigah 15a)
Despite his apostasy he and Rav Meir retained their great friendship with Rav Meir ignoring the criticism of his colleagues as he "ate from the fruit while removing the pit." (ibid)
Our rabbis speculated as to what it was that caused Acher to go astray and the variety of explanations given serve to indicate the futile nature of such an endeavour. It is hard enough to know what motivates one's own self in their ups and downs of religious life - to do so for others borders on the foolhardy. Yet whatever may have motivated Acher - and it is rarely one event that causes a life changing direction - the reasons suggested by the Sages are lessons for us.
"Rav Yosef said: If only Acher had interpreted the verse as had Rav Yaakov the son of his daughter he would not have sinned." (Kiddushin 39b)
The Torah is full of promised rewards both individual and even more so collective, for following the precepts of the Torah. Peace and prosperity, wealth and good weather, plentiful harvest and beautiful fruit and long life. The list of rewards has troubled many as it is so "mundane", dealing with the physical aspects of life and totally ignoring all spiritual rewards. The Torah does not even directly mention a messianic age or a world to come. The focus of the Torah is on this world and this world only.
Thus we can only be shocked to read the view of Rav Yaakov that "sechar mitzva bhai alma leicka, there is no reward for mitzvot in this world." (Kiddushin 39b)
It surely goes against the plain meaning of the text and would seemingly violate the rule that "a verse does not leave its plain meaning." (Shabbat 63a) While the derasa, the additional layers of meaning that can be gleaned from the text, may be even more important than the pshat, the plain meaning of the text, it is meant as a supplemental meaning and not one to replace the pshat. And as the Gemara notes a few lines earlier and has been incorporated into our davening there are mitzvot in which man eats the fruit [of reward] in this world and the principal remains for the world to come, beginning with the mitzva of kivud av veim. Is it any wonder Acher could not accept this view?
On the other hand it is most difficult to claim that we are rewarded for our mitzva observance in this world. There is too much injustice in the world to uphold that view. To use the Talmudic example: How would one explain the case of one who following the request of his parents to bring some young birds, climbs up the nest and sends away the mother bird falls and is killed on his way down the ladder. It is specifically the mitzvoth of shiluach hakan and kibud av veim where the Torah promises a long life. "Where is the good days of his life and where are the long days of his life" the Talmud wonders?
What I find fascinating is what follows. The Gemara responds by stating that such would be a good question if it actually happened but maybe such a case never actually occurred[1]. To this they answer that Rav Yaakov was actually witness to such a case - and as a response developed the view that the reward of a good long life is for "the world [to come] that is all good and to the world [to come] that is never ending."
Yet the Gemara is not convinced. G-d can promise a reward - all things being equal. But it is most dangerous to view any issue from one angle only and there are always extenuating circumstances that can nullify what would otherwise be. Perhaps there was faulty ladder and whenever damage is near one cannot rely on miracles; perhaps he had thoughts of idolatry on his mind and while our sinful thoughts are not generally held against us, idolatrous thoughts are different. Worship of G-d or gods is to large extent expressed most accurately in our inner thoughts.
In any event the Gemara uses this story as one possible explanation as to why Acher left the path. While it might be because he did not agree with Rav Yaakov it is worth considering that perhaps in theory Acher agreed the rewards of the Torah are [primarily?] meant for the world to come. However no matter what he may have believed, upon witnessing the death of child doing two mitzvot for which one is promised a long life it was too much to handle and he lost his faith. Seeing is believing in more way than one.
The Gemara presents a second possibility along the same lines but with different details. Acher witnessed the tongue of Chutzpit HaMetugaman being dragged on the ground by a pig[2]. Chutzpit Hameturgeman was the one tasked with translating and expounding the shiurim of Rabban Gamliel to the students of the Beit Midrash. He was one of the ten martyrs executed by the Romans. Upon seeing what had happened Acher exclaimed "the mouth that brought forth so many pearls shall be dirtied in the dust. He went out and sinned[3][4]."
It is heartening to know that Acher's legacy is not all lost. (see here for one aspect of his legacy) It is his grandson whose teaching may have saved Acher. But even if they did not, or perhaps could not, Acher's grandchildren and hence Acher is part of the chain of Torah stretching through the generations.
---------------------
[1] As to general injustice in the world; the suffering of the righteous and reward of the wicked, such is an aspect of G-d's system we cannot understand. But as unjust this issue of theodicy may be, such is needed so that man not effectively have free choice taken away from him, which would be the case if every good deed was rewarded and every bad one punished. Yet the contradiction to the direct promise of the Torah is most directly seen in the case at hand.
[2] In another beautiful play on words the Gemara refers to the pig as "davar acher", that other thing.
[3] It is worth noting that the Gemara says he sinned, not that he lost his faith. Talmudic literature viewed people through their actions not their beliefs.
[4] Other Talmudic sources saw Acher's apostasy a result of philosophical, not experiential issues.