Bluetooth high latency even though latency should be very low

I connected everything right and when I playback using my phone speaker I notice no latency at all but when I connect a Bluetooth device to my phone (WH1000XM3 using LDAC) there is latency again even though all other apps like Youtube etc are pretty much latency free. Is there a way to fix that?

Since YouTube compensates the Bluetooth latency by synchronizing the images and the audio, it can’t be used to establish if there’s latency or not.

In order to know if the latency you’re noticing is solely due to the headset, could you please do some tests on apps that play real-time audio?
e.g: Piano apps, soundboards or this one

With this app, I’m seeing 150 ms latency on one of my Bluetooth speakers for example.