While there are 613 categories of mitzvoth they are not all created equal. Saving a life is more important than observing Shabbat, Shabbat is more important than keeping kosher and kashrut in turn is more important than shaking a lulav. Generally it is more important not to violate a negative command than to observe a positive one[1]and the observance of biblical commands takes precedence over those of a rabbinic nature, which are more important than customs.
The above holds true in theory but not always in practice where factors beyond the technical observance of mitzvoth come into play. While on a technical level violating the laws of family purity is as serious a violation as is eating on Yom Kippur, from a communal perspective I think its fair to say that one who does not even observe Yom Kippur is at greater risk of assimilation than one who does not observe the laws of family purity.
On occasion, when circumstances warranted, our Rabbis gave greater import to rabbinic law than biblical law. To cite one example: Our calendar is set up in such a way that Hoshana rabba can never fall on Shabbat. The Sages feared that if Hoshana rabba fell on Shabbat - in which case the rituals would be cancelled that year - then in subsequent years the day would not be widely observed. With no specific mention of Hoshanna rabba in the Torah and with various sects denying that such a special day existed this was a very real concern.
However in order to do so the calendar would have to allow Rosh Hashanah to fall on Shabbat thereby causing us to neglect the biblical mitzva of shofar, a mitzva that is clearly more important than any associated with Hoshana rabba. Yet our sages understood that one could miss a day of shofar with little impact[2] on future years something that might not be the case with hoshanot. These factors outweighed the technical hierarchy of these mitzvoth.
The fact that shofar itself is not blown on Shabbat is itself not easily understood from a technical halachic point of view. Depriving all from hearing the shofar because the one who blows it may accidentally carry on Shabbat may goes against the Torah hierarchy of mitzvoth - but falls within the realm of rabbinic legislation "imposed" on the Torah.
At times it is social or historical considerations that are paramount. The Mesech Chochmah (Shemot 12:22) explains that sometimes it is the least important of mitzvoth - if we can even call them such - that have greater significance than mitzvoth fundamental to our way of life. Sometimes having a Jewish name, or speaking Hebrew or even wearing a magen david around one's neck may be what keeps the person in the fold - something Shabbat or kashrut could not do for this person.
Many of the difficult questions of Jewish law that arise require determining how much weight to give to technical halachic principles and how much to meta-halachic concerns. More often than not it is not the technical halacha that is the primary issue but rather balancing the underlying values that might be in conflict. Should we give more weight to the individual or to communal norms? Where do we draw the line between chidush, innovation and mesorah, tradition? When is compromise ideal, appropriate or misguided? These are just some of the issues with which poskim must grapple[3].
One area where there is little dispute is that personal religious growth can never come at the expense of one's obligation to others. The Talmud requires a worker to shorten his prayers so as not to take time away from work. One must have kavanah on one's own time.
Which brings us to Chanukah and Bava Kamma. "A camel that was laden with flax passed by in the public domain and its load of flax entered into a shop and caught fire from the storeowner's candle and lit a large house on fire, the owner of the camel is liable. But if the shopkeeper left his candle outside, the shopkeeper is liable. Rabbi Yehudah says: If it was a Chanukah light, he is exempt." (Bava Kamma 62b)
It is clear that according to the Sages the fact that one is lighting Chanukah candles does not exempt one from damages the candles may cause down the road. Though Chanukah candles are meant to be lit outdoors one should not place them in the public domain. Rather it is a "mitzva to light outside the door of his [private] home" or when such is not possible "at the window adjacent to the public domain." (Shabbat 21b)
Yet one could argue that even Rav Yehuda would agree that doing a mitzva is no license to cause potential damage. The reason one is exempt is not because one did a mitzva. Rather it because at Chanukah time the streets are full of lights and it thus becomes the responsibility of the passerby to be on the lookout. This is similar to what we discussed earlier here that damage caused by the walking of an ox in the public square is not recoverable. As oxen - at least in Talmudic times - roamed the streets it becomes the responsibility of those walking the streets to be on the lookout and not to leave stuff strewn about.
Rav Yehuda argues that Chanukah candles should be treated the same way. While an individual generally would not be exempt from damages caused by performing a particular mitzva Chanukah candles are different. This is a mitzva incumbent upon all. Furthermore ner Chanukah is one of those rare unique mitzvoth that is done for the purposes of pirsumei nissa - of publicizing the miracle.
Normally we are forbidden to publicize our piety in regards to mitzvoth between man and G-d as such would be a violation of yuhara, religious arrogance. But Chanukah candles (and the reading the Megillah) is different as it reflects a national celebration of G-d's deliverance and must be publicized. And if such requires a storeowner to light outside so be it. When it comes to Chanukah the burden of responsibility shifts to those walking the streets to ensure they not stumble over Chanukah lights. When one adds the fact that Chanukah candles had a most limited time to be lit - unlike today, once stores closed some 30 minutes after dark the time to do the mitzva had ended - the exemption of Rabbi Yehuda makes eminent sense. As evening arrives on Chanukah the Jewish street is alight and passerby beware.
And yet the view of Rav Yehuda is rejected by the Sages. Even on Chanukah one must be careful to ensure we light away from traffic. One has to figure out a different way to publicize the miracle.
[1] This has more to do with the fact that generally speaking an act of commission is worse than an (non) act of omission and less to do with the actual importance of the mitzvoth themselves. In fact the Ramban (Shemot 20:8) explains (and as we elaborated on here) a positive mitzva reflects love of G-d whereas a negative mitzva reflects fear of G-d. And since love is greater than fear, aseh doche lo taaseh, a positive mitzvah "pushes aside" a negative mitzvah, and is to be observed despite the fact that it involves a "violation" of a negative command.
[2] It may also be that with Rosh Hashanah being two days even if the first day falls on Shabbat shofar would not be neglected that year. Such an approach however only makes sense if one assumes the practice of two days of Rosh Hasahanah preceded the decree not to blow shofar on Shabbat. Such an assumption is most questionable in general and surely not true in regard to the land of Israel where it took until the middle ages until two days of Rosh Hashanah was universally observed.
[3] It is because so many of the questions that one must seek guidance on deal with these meta-halachic issues it is imperative that one's posek be of similar haskafic outlook. It makes little sense for someone pursuing a Ph.D in philosophy to ask a posek who does give his own kids a general education to ask if one should take a year off of his studies to learn in Yeshiva.