A new modulation scheme was proposed to deal with the error propagation（EP） existing during the successive cancellation（SUC） detection in the VBLAST（vertical Bell labs layered space-time） system.In the scheme

the former demodulated signal was designed to use the modulation with better noise suppression performance（e.g.binary phase shift keying modulation）

and the latter demodulated signal was suggested to employ the modulation with higher frequency ef-ficiency（e.g.multi-decimal modulations）.Analysis and simulation results show that system reliability for SUC detection is improved without penalty to the system availability.