The subject has recently been raised as to what constitutes leavening (chametz) חמץ or seh-ore' (leaven) שׂאר according to the Biblical (tanak) definition.
We all agree that yeast is a leaven. But why is yeast a leaven? Well, if you look at the base definition of chametz, you will find that it has to do with fermentation, not with being a rising agent. This can also be seen with the word chometz חמץ, which is the Hebrew word for vinegar.
Now during the Passover, they were to be ready to move out at a moments notice, and the rising of bread with the fermentation process of yeast, was not a fast process, and it could also be argued that the fermentation represents sin, and so for these two reasons, the Israelites were told to only make unleavened bread (matzah).
So then, what about Baking Soda (and Baking Powder)? Well, Baking Soda, works by and completely different process (there is no fermentation involved) and it does so almost instantly, so by both of those attributes, it does not fall under chamatz or seh-ore'.
As an aside, to those of you who would point out that Baking Soda has aluminum in it, I would point out that you can get baking soda without aluminum, but I digress.
In any case, here is a link to a good article on the subject.