Even if the supervisor is out of the preview of ID act even then the principles of natural justice are applicable to him. It is advisable that you issue him a show cause notice and call for his explanation and thereafter take the action of dismissal on the grounds of loss of confidence stating that the explanation is not found to be satisfactory.

You are free to show him the door.