Eat the Meat - Throw Out the Bones

So very often in Christian settings I see people following teachers that are, at the very least, questionable in their doctrine.  Some of these popular teachers are actually beyond questionable and could truly be labeled as false teachers.   Sometimes people will comment that they know this or that person might say some things that are questionable, but they also have a lot of good things to say.  They tell me they will eat the meat and throw out the bones.  They will take the good and throw out the bad. 

Is that what God would have us to do?