"We stay in touch, just not so often," Brett Oppenheim told PEOPLE of his current friendship with Christine Quinn after she left ... and she's a big loss because she's made for reality TV ...