The issue is that the mortgage is a debt that needs to be paid. Think of it this way; if you live in your home for the rest of your life, you need to keep paying your mortgage.
That's why I keep it in there.
Where as home equity can become "paper gains" a mortgage is very real.
It's true that paying it down involves trading cash (the most liquid asset) for a smaller mortgage. That's why I am happy paying it down as slow as possible and diverting all of my surplus to cash and investments.