Get the homeless away from that big castle full of hundreds of spare rooms, demands Tory council leader