Custom dataset for TLT

If network is already trained with coco2017, and I would like to refine 1 particular class using transfer learning e.g. person.

For this I have to prepare a custom dataset and have a question for particular scenario. Suppose an image in custom dataset have multiple classes in it:

1: do i need to annotate all classes? e.g. person and chair
or
2: is it ok to just annotate person class? and leave out chair

Overall objective is to keep the learned waits of all classes but refine just one class with new dataset using transfer learn training for all 80 coco classes.

It is ok to just annotate person class.

Thank you.