Abstract: Based on multi-sensor data from star sensors and inertial sensors, a high-performance Kalman filtering algorithm with linearized quaternion measurements was proposed in this paper. According to the ultra-stable and ultra-static platform characteristics of the task, new pseudo-linear measurements were constructed by an approximate transformation for quaternion measurements under small angle change of the spacecraft so that the linear assumption of Kalman filtering was satisfied. Combined with the discrete-time state space model of spacecraft system and the multi-sensor measurements, a Kalman filtering algorithm with linearized quaternion measurements was designed, to achieve high-precision in-orbit state estimation of spacecraft system. The simulation experiments are provided to demonstrate the effectiveness of the proposed Kalman filtering algorithm, which meets the precision requirement of spacecraft attitude estimation for space-based gravitational waves detection and provides the high-precision observation for spacecraft attitude control.