There is a machlokes in Yoma whether the אבנט of a regular Kohen was כלאים or not. The Gemara in Erchin 3b assumes that it was. The Gemara there has a ה"א that a Kohen should be פטור from ציצית because maybe you are only chayav in ציצית if you can't wear כלאים, however, Kohanim who wear כלאים (e.g. the אבנט) would be פטורים. The Gemara answers that since they can only wear כלאים while doing avoda they are chayav in ציצית.
The din is that a Kohen is allowed to wear the בגדי כהונה in the Beis Hamikdash while not doing the avoda. However, the Rambam writes that the אבנט, the Kohen must take off as soon as he finishes the avoda since it is כלאים. The only heter that the Kohen has to wear it is עשה דוחה לא תעשה, as soon as he finishes the avoda this no longer applies. The Raavad disagrees and holds that the אבנט is no different then the other בגדים and can be worn even while not doing avoda.