I tried the following combination:
1Way, 1KB/Way, 32B/Line - PASS
1Way, 32KB/Way, 32B/Line - PASS
4Way, 32KB/Way, 32B/Line - PASS
4Way, 128KB/Way, 32B/Line - PASS
4Way, 32KB/Way, 64B/Line - FAIL
4Way, 128KB/Way, 64B/Line - FAIL
32B linesize can boot kernel successfully, but 64B always fails.
I tried a simple simulation and did not find any clear RTL issues. I am not sure if the problem lies in Vivado (Vivado 2022.2, XCVU19P device).