I'm primarily writing "general-purpose" code (especially parsers and formatters) rather than code that does the same math operation on a big array, so it's usually not reasonable to even use the same approach to the problem with different vector extensions :(
Even in the latter case, different approaches are often required. For an 8x8 byte block difference, SSE2 prefers horizontal accumulation (PSADBW) while ARM64 prefers vertical (UABAL). It's noticeably suboptimal if you try abstracting across these with generic primitives.