In a small saucepan, add all the ingredients into it and place it over low heat. Once it warms up, stir it regularly so the sweetener dissolves. Once it has dissolved, remove the pan from the heat.
Cover the pan and let it sit for two hours, for the vanilla flavor to infuse into the coconut milk and for it to cool completely.
Whisk the mixture well then transfer it into 8 popsicle molds. Place in the freezer for at least 4 hours or overnight, to firm up.
If you'd like to dip them in chocolate, melt the chocolate in a large bowl. Line a large baking sheet with parchment paper. Remove the popsicles from the molds and moving quickly, dip them in the melted chocolate and place them on the lined sheet. Place them back in the freezer for the chocolate to firm up.
Notes
TO STORE: Popsicles should always be stored in the freezer, preferably in their popsicle molds. However, if you dip them in chocolate, they will no longer fit in the molds. To rectify this, place the popsicles onto a lined plate and cover them completely. They will keep fresh for up to 6 months.