Well, just for the sake of anyone reading this that MAY run into the same problem, I got it all situated!
I pulled apart as much rubber as i could from the latex handguard(DO NOT FORCE IT BECAUSE IT COULD RIP YOUR HANDGUARD), and then I just simply trimmed around the edges of the handguard, got another set of gloves, and reglued.
When i glued, i took the advice of DarkCMF(Tim) and put the rubber glove on, prepped the area of ONE corner, got the handguard into position, applied the glue, and pressed down.
Then i took the glove off after the glue dried and the handguard was secured, and commenced with gluing the rest of the handguard to the glove. This way, the handguard is in perfect positioning and you can avoid runs of super glue.
I used Loctites Plastic Bonding System Glue, a suggestion from Gazmosis(Steve). Works AMAZINGLY! Its a two part glue, come with a prep marker and a regular sized bottle of super glue(CA glue). And yes this prep pen DOES work with any super glue, so if you run out, hopefully you have a stock pile of various super glues like myself