My pitbull is afraid of people and growls at them! Help!

"Buddy came from a shelter.  I've had him for about 4 years and he is probably 5 or 6 years old.  I don't have a backstory for him.  He was left at the shelter and lived there for about two months.  He was about to be put down when I adopted him.

Buddy is great with me and Chris, my partner.  With strangers it is a different story. He exhibits fear regularly. If anyone has a hat or sunglasses on, or anything in their hand, or makes sudden movements he cowers and may growl. 

He also is extremely protective of me and growls when most strangers come near me. Can we assist him? I am not sure how to change this behavior."

Join Suzy, Chris, Patty and Buddy in this conversation on what might be done to get Buddy over his fear and protectiveness!