There was some antagonism towards Joseph from his half brothers. The rivalry between their mothers was almost certainly one cause: how would you like to be one of the sons of the less-loved wife? Joseph’s habit of telling his father whenever his brothers did something wrong didn’t help.
Jacob’s clear favouritism of Joseph was another reason:
Genesis 37:3-4 Jacob loved Joseph more than any of his other children because Joseph had been born to him in his old age. So one day Jacob had a special gift made for Joseph—a beautiful robe. But his brothers hated Joseph because their father loved him more than the rest of them. They couldn’t say a kind word to him.
Joseph was born to Jacob when Jacob was 91. On a plain reading of the text, Joseph was only six or seven years younger than his eldest brother (one commentator suggests 26 years’ age difference), but Joseph was the first son of the wife Jacob loved so much, and the wait for that son, through 10 other sons and at least one daughter, would have been long.
On top of the obvious favouritism came the dreams, or 11 sheaves of wheat, and in a later dream, the sun, moon and stars, all bowing down to him. Joseph’s family knew what these dreams meant:
Genesis 37:8 His brothers responded, ‘So you think you will be our king, do you? Do you actually think you will reign over us?’ And they hated him all the more because of his dreams and the way he talked about them.
Even Jacob started to wonder how these dreams could mean what they appeared to mean, and scolded Jacob, possibly because he knew the high cost of sibling rivalry as he’d seen between Ishmael and Isaac, himself and Esau, and now Joseph and his brothers:
Genesis 37:10-11 This time he told the dream to his father as well as to his brothers, but his father scolded him. ‘What kind of dream is that?” he asked. ‘Will your mother and I and your brothers actually come and bow to the ground before you?’ But while his brothers were jealous of Joseph, his father wondered what the dreams meant.