Mouse and bug problem?

As long as you clean your guinea pig cage out regularly and don't leave around crumbs and such, there shouldn't be any type of bug problem. It could be a neighbors infestation. I'd go ahead and contact your landlord through a certified letter explaining the situation. If failure to fix your issue with bugs then report them.