Our Gemara on Amud aleph refers to Adultery as an Issur Kal, a relatively light prohibition, which Rashi (“makom”) explains is because the prohibition is not permanent, i.e. it lifts after divorce or death.  Presumably, the Gemara does not equate the severity of the transgression and the punishment (death), with the technical aspects of the mitzvah that make it considered less severe than other forms of forbidden relations that remain permanent.  In other words, it may be a terrible and severe sin, but in terms of applying kal v’chomer and other principles of derash it is less severe (kal) because it is not a permanent prohibition.

Relating to the idea of evaluating the severity of a mitzvah, of course we have the famous teaching in Avos (2:1)

וֶהֱוֵי זָהִיר בְּמִצְוָה קַלָּה כְבַחֲמוּרָה, שֶׁאֵין אַתָּה יוֹדֵעַ מַתַּן שְׂכָרָן שֶׁל מִצְוֹת

be careful with a light commandment as with a grave one, for you did know not the reward for the fulfillment of the commandments.

The Maharal Tiferes Yisrael 61 explains this idea as follows: 

In reality, we can tell if certain mitzvos are more severe than others or more important. This might be based on the punishment or other aspects that show its severity. However, there is also the individual merit that is based on the subjective effort and personal test it involves. Thus, the Mishna means, even a small mitzvah needs to be worked on, as you do not know the reward you may receive THIS time, based on the sacrifice you make.

The Maharal adds: since mitzvos are there to correct and repair aspects of physicality and bring them back to Hashem, it is unknown what a particular mitzvah will achieve in a particular circumstance. It can have far reaching implications even if it is a small mitzvah.

There is a Chassidish Vort on this phrase, which is found in the beginning of the ethical will of the Rivash.  The word for careful “zahir” in Aramaic means to shine.  Thus, it is a directive to make a small mitzvah light up and shine like an important mitzvah because through that dedication, it will elevate it to a higher status.