Recent advances in machine learning inspire the development of deep neural network-based smart sensing applications for the Artificial Intelligence of Things (AIoT). However, due to the nature of the AIoT sensing data, the machine learning models are in general subject to poor generalizability due to the scarcity of labeled training data and run-time domain shifts. The existing solutions rely on data-driven approaches and do not consider the physical laws that govern data generation or domain shifts. This paper discusses the potential of utilizing the known physical laws to improve the machine learning model generalizability for AIoT applications. Through three case studies, we demonstrate that physics-informed machine learning can (1) effectively assist the generalization of deep neural networks and (2) achieve better performance compared with conventional approaches. Our objective is to encourage more exploration into combining physical principles and machine learning algorithms in physics-rich AIoT.