Unfortunately this is an artifact based on how the outline is created. It tests your texture at 8 locations which will cause these patterns at sharp-angled corners < 90 degrees. The solutions below are not perfect, but will improve the situation:

Solution 1 - Using mip maps:
You can enable Generate Mip Maps at the texture in the Texture Import settings in the Inspector.
Then at the outline material you can adjust Advance - Outline Mip Level until you are happy with the outline shape.

Solution 2 - Painting a very transparent border:
Paint under any sharp corners with minimum opacity (e.g. 1 / 255) which is barely noticable, so that the resulting corner is >= 90 degrees. Alternativaly, you can also add a very transparent rounded border around your image whole image via e.g. a Stroke effect in Photoshop, where you want the outline to be painted.
Then at the outline material, reduce the outline width to a small value, and set the Advanced - Outline Threshold value below the painted value, e.g. to 0.001.

Harald wrote

Unfortunately this is an artifact based on how the outline is created. It tests your texture at 8 locations which will cause these patterns at sharp-angled corners < 90 degrees. The solutions below are not perfect, but will improve the situation:

Solution 1 - Using mip maps:
You can enable Generate Mip Maps at the texture in the Texture Import settings in the Inspector.
Then at the outline material you can adjust Advance - Outline Mip Level until you are happy with the outline shape.

Solution 2 - Painting a very transparent border:
Paint under any sharp corners with minimum opacity (e.g. 1 / 255) which is barely noticable, so that the resulting corner is >= 90 degrees. Alternativaly, you can also add a very transparent rounded border around your image whole image via e.g. a Stroke effect in Photoshop, where you want the outline to be painted.
Then at the outline material, reduce the outline width to a small value, and set the Advanced - Outline Threshold value below the painted value, e.g. to 0.001.

do you have any advice on doing this after export without breaking the atlas? I tried to do this to the atlas png in the unity asset after it was imported from spine and it just broke everything and stopped rendering.

beejam wrote

do you have any advice on doing this after export without breaking the atlas? I tried to do this to the atlas png in the unity asset after it was imported from spine and it just broke everything and stopped rendering.

Please describe in more detail what exactly you did (which steps did you take, what did you do in your image editor, etc) and how the result looks like in Unity, how "it just broke everything and stopped renderering" actually looks?

A common problem is to modify a premultiplied-alpha (PMA) atlas .png image in Photoshop, paint over it and then Photoshop does not apply PMA again in transparent areas (since it exports as straight alpha) and thus creates rendering artifacts. You can read about it in this recent forum thread here:
Spine to unity edit atlas png

This will however not stop anything from rendering, so we might need further info and some screenshots showing your problem.