Insiders say Hamilton has lost trust