I think you are over tightening your belt to close the sides. If your armour looks perfect when hanging from a coat hanger then the problem is how you are wearing it.
To me, it looks as though you wear the abdomen too high and too tight. If you try lengthening the lower half of your torso straps so you wear it lower, and don't tighten your belt so much, I think you could solve your problem with the back plate.