The brothers could not speak peacefully with Joseph because they allowed their anger and resentment to control them, rather than asserting their control over their anger.
By Rabbi Neal J. Loevinger
Parashat Vayeshev begins the concluding drama of the book of Genesis, the story of Joseph and his 11 brothers, their estrangement and eventual reunion. Joseph is the favored son, and acts like it, so his brothers conspire to throw him in a pit, then sell him into slavery, then tell Jacob that Joseph was attacked by an animal. He ends up in Egypt, as the servant of a powerful man, Potiphar.
Meanwhile, his brother Judah is having problems of his own; his sons die childless, and he refuses to give his daughter-in-law Tamar to his youngest son so he may have children. She dresses like a prostitute, entices Judah to sleep with her, and she is vindicated as having acted correctly in the end, and bears children.
angry fist punching a wall Potiphar's wife desires Joseph, and when he refuses, he is thrown into prison, where he ends up interpreting the dreams of Pharaoh's servants, which will eventually bring him to the attention of Pharaoh himself.
"His brothers saw that it was he [Joseph] whom their father loved more than all the brothers, and they hated him, and they could not speak with him peaceably." (Genesis 37:4)
A familiar pattern in the Book of Genesis repeats itself in the story of Joseph and his brothers: one son is favored over the others, and there is tension, jealousy, and eventual estrangement within the family. In this case, Joseph brings "bad reports" about the brothers to their father, and they see Jacob giving Joseph special treatment, such as his ketonet passim, [a striped or more likely an ornamented/embroidered] coloured cloak. The brothers are angry, jealous, and resentful, and thus alienated from each other.
What struck me about this verse is the Torah's description of the emotional state of the resentful brothers: "they could not [lo yachlu] speak with him peaceably." It's not quite clear what that last phrase means.
Rashi says they didn't speak with him at all, whereas other commentators say that they spoke to Joseph resentfully, or spoke amongst themselves in non-peaceful ways against Joseph. Rashi at least tries to give the other brothers a little bit of credit by pointing out that at least they weren't hypocrites: they didn't pretend to love him while hating him in their hearts, but rather avoided him altogether.