Repository navigation
Fix rounding in SSE byte stores - #357
Nathan Chandrasekar (vera-rubin) wants to merge 2 commits into
Conversation
|
I'll need to look closely at these proposed changes. There are a number of issues where I need to ensure that ARM matches, and that it doesn't negatively impact DirectXTex, etc. which is the primary consumer of these functions. |
| // Mask off any fraction | ||
| vResulti = _mm_and_si128(vResulti, MaskUByte4); | ||
| // Do a horizontal or of 4 entries | ||
| __m128i vResulti2 = _mm_shuffle_epi32(vResulti, _MM_SHUFFLE(3, 2, 3, 2)); |
There was a problem hiding this comment.
Can we use the same _mm_packs_epi32 / _mm_packs_epi16 pattern with XMStoreUByteN4 and XMStoreByteN4?
|
Yes: |
On SSE, storing
XMVectorReplicate(0.75f)withXMStoreByte4orXMStoreUByte4produces0x00000001instead of0x01010101.XMStoreByteN4similarly packs0.25fas0x1f1f1f1finstead of0x20202020. The documented Byte4, UByte4, and ByteN4 operations round each component before packing.We convert each clamped component to an integer before packing with SSE2 instructions, following the short-store implementation.
We checked the reproducer on SSE2, SSE4.1, and AVX2 and added a math3 regression that fails before and passes after, while the full suite retains its five existing math3 failures.
— ChipLab