Replying up top so people actually read this.  I just did this quest and when you are at the camp, the humans will aggro THROUGH the gigantic wooden fence, so if you have a high aggro radius, try to make sure there is noone on the other side of the fence you would aggro.