The particle crushing effect of coarse-grained soil has been widely studied. This study conducted lateral compression tests under high-pressure conditions to study the compression characteristics and particle crushing laws of waste particles from tunnel excavation of phyllite. Samples with continuous and intermittent gradation were set up to analyze the physical parameters such as compression deformation law, gradation curve, crushing index, and plastic work before and after loading. The results show that the strain of each group of samples increases rapidly with stress during the loading process and then gradually stabilizes. The continuous group gradation shows good compressive bearing capacity. As the content of phyllite particles decreases, the overall deformation of the samples can be effectively reduced. Under high stress, the grading of each intermittent grading sample gradually converges to a continuous grading, with similar physical and mechanical properties. The fragmentation indicators applicable to the particles of waste rock in phyllite are Br and Bg. Establishing a hyperbolic model to fit the plastic work and relative fragmentation rate Br during the compression process, it was found that fitting the plastic work model can better characterize the fragmentation law of waste particles in phyllite. The research results can provide certain reference and guidance for further understanding the compression characteristics and particle fragmentation laws of soft rock waste materials such as phyllite, as well as for the filling of geotechnical engineering structures.