Place coconut milk in a large saucepan over medium heat. Heat until simmering, stirring frequently.
Place sugar and egg yolks in a small bowl and whisk until blended. Measure 1/2 cup hot coconut milk. While continuing to whisk yolks, gradually add hot milk, 1 tbsp. at a time until milk has been incorporated into egg yolks. (This is done to prevent eggs from curdling.)
Pour egg mixture into saucepan with remaining milk and cook, stirring constantly with a wooden spoon, until custard thickens slightly, enough to coat the back of the spoon (about 4 minutes). Immediately pour mixture through a fine mesh strainer into a bowl. Stir in yogurt and zests, and cool custard completely.
In a blender or food processor, purée avocado with lemon and lime juices and about 1 cup cooled custard until smooth. Fold avocado mixture into custard. The mixture should be thick, creamy and pale green. Freeze custard in an ice cream maker according to manufacturer's instructions. Transfer frozen yogurt to an airtight container and freeze until solid, at least 4 hours.
Note from Kim Kelly: I actually prefer this frozen yogurt soft, like the soft serve at the local shop.
Makes 1 quart