Imagine if instead of twenty-four books the Tanach consisted of only six. Strange as it sounds that was the original plan. "Rav Adda the son of Rav Hanina said: Had Israel not sinned, only the five books of the Torah and the Book of Joshua would have been given to them" (Nedarim 22b). At Sinai G-d taught Moshe all the Torah that the Jewish people should have needed. If not for the sin of the spies the Jewish people would have entered Israel under the leadership of Moshe and there would have been no need for the book of Joshua. Because of our sins it would take another generation to conquer the land and hence the need for Sefer Yehoshua. And that should have been it. After Divine revelation what could be left for other prophets?
Had we not continued to sin there would have been no need for the constant rebuke of the prophets. We would not have to hear about our oppression of the widow and the poor, about our rote observance of rituals, our hypocrisy and lack of character. But alas we continued to sin so more books were needed.
Yet could it truly be that if not for the sinning of the Jewish people we would not have needed the rest of Tanach? Is there not so much more to the other 18 books of Tanach than rebuke? What about thebeauty, pathos and passion of Sefer Tehillim, the wisdom of Mishlei, the melancholy of Kohelet, the inspiration and comfort of Yishayahu, the beauty of Ruth and so much more. While I am not satisfied that what I write below properly explains this most enigmatic teaching (I would love to hear your suggestions) I share what I hope are worthy insights.
We recognize that the best way to keep someone out of trouble is to keep them busy. The less time with nothing to do the less time people will be up to nothing good. Theoretically Biblical law should be enough to ensure we live a life of holiness. But theory and practice do not always align.
And this is apparently how Rashi understood this teaching. "Because they kicked and sinned it was added to them the great wisdom of the other books lehatrichan yoter"(Rashi 22b s.v. mipfnei). The word lehatreach means to burden but is generally used in rabbinic literature to refer to the burden of wasted time. Had the Jewish people not sinned we would have intuited much of the great wisdom found in other books of Tanach. Much of it would have not even been necessary as one (hopefully) gains much wisdom from one's mistakes. The less mistakes we make the less we need to learn.
Much of rabbinic law is based on verses from these "other" books of Tanach. And here too much of rabbinic law is a reaction to or a precautionary measure to prevent sin. We have the laws of muktza to prevent Shabbat violation. We forbid chicken and milk lest we mistakenly mix meat and milk - the list of rabbinic ordinances to prevent sin is endless. The less we sin the less we need preventive measures. Our many fast days from Tisha B'Av to BeHaB were a response to sin. The institution of mechitzah - the source of so much controversy between and even within the" denominations" - was an attempt to prevent frivolity in the Temple (see Sukkah 51b).
It is in many ways a sad fact of halachic history that as Jews became less religious often rabbinic authorities felt the need to become stricter in their application of law. And this is even (especially?) when it is "others" who are sinning. As the majority of the Jewish world began - and continues - to practice and even define their Jewishness in non-halachicterms, Orthodoxy has been forced to become much stricter. To cite one poignant example, when Jews started clandestinely going to the former Soviet Union to strengthen the Jewish community they were often forced to stay at hotels not within walking distance of the shul. Rabbi Soloveitchik was asked if they could have a non-Jewish taxi drive them to the shul - something this is not necessarily difficult to justify halachically. Yet the Rav responded that as much as he would like to, because the Conservative movement permitted driving on Shabbat he could not allow such. Similarly the Rav felt he could not allow a "sah bus"with a non-Jewish driver to pick up people who would then agree to leave their cars at home. Instead of noting the difference between driving and being picked up many would likely say even the Orthodox permit driving on Shabbat.
One can bemoan this state of affairs as much as we like - and the Rav was not happy about the need to rule this way. However halacha is a system observed in real life not some theoretical construct. As we all know theory and practice do not always align.
I look forward to the day when we will sin less allowing our halachic authorities to be more "flexible". In the interim let us study the great wisdom contained within all twenty-four books of Tanach.
If so, then, if I may say so, it almost makes me glad that we sinned.
 The generally excepted opinion is that the commentary ascribed to Rashi in Nedarim was not actually written by him - something one who has a basic familiarity with Rashi's commentary on the Talmud immediately intuits. Nonetheless for simplicity's sake and in keeping with the printed Talmud before us we will refer to this commentary as Rashi.
 The custom to fast the Monday, Thursday and Monday after Pesach and Sukkot (one I have personally never seen observed) was instituted to address the sinful levity that often accompanies "vacation time".
 I heard both these stories personally, the first from Rabbi Aharon Rakeffet and second from Rabbi Shlomo Riskin.
 The Brisker derech of learning with its emphasis on theoretical constructs focuses on Talmud generally leaving the study of practical halacha for others.