Mortgage expert says tariffs and job losses may hurt buyer confidence just as interest rate cuts fuel optimism