Allah describes the Jews' disbelief, stubbornness and defiance of what they know of the truth of Allah's Messenger ﷺ , that if the Prophet brought forward every proof to the truth of what he was sent with, they will never obey him or abandon following their desires. In another instance, Allah said:

إِنَّ الَّذِينَ حَقَّتْ عَلَيْهِمْ كَلِمَةُ رَبِّكَ لاَ يُؤْمِنُونَ - وَلَوْ جَآءَتْهُمْ كُلُّ ءايَةٍ حَتَّى يَرَوُاْ الْعَذَابَ الاٌّلِيمَ

(Truly, those, against whom the Word (wrath) of your Lord has been justified, will not believe. Even if every sign should come to them, until they see the painful torment.) (10:96, 97)

This is why Allah said here:

وَلَئِنْ أَتَيْتَ الَّذِينَ أُوتُواْ الْكِتَـبَ بِكُلِّ ءَايَةٍ مَّا تَبِعُواْ قِبْلَتَكَ

(And even if you were to bring to the People of the Scripture (Jews and Christians) all the Ayat (proofs, evidences, verses, lessons, signs, revelations, etc.), they would not follow your Qiblah (prayer direction)).

Allah's statement:

وَمَآ أَنتَ بِتَابِعٍ قِبْلَتَهُمْ

(...nor are you going to follow their Qiblah ), indicates the vigor with which Allah's Messenger ﷺ implements what Allah commanded him. Allah's statement also indicates that as much as the Jews adhere to their opinions and desires, the Prophet adheres by Allah's commands, obeying Him and following what pleases Him, and that he would never adhere to their desires in any case. Hence, praying towards Bayt Al-Maqdis was not because it was the Qiblah of the Jews, but because Allah had commanded it. Allah then warns those who knowingly defy the truth, because the proof against those who know is stronger than against other people. This is why Allah said to His Messenger and his Ummah:

وَلَئِنِ اتَّبَعْتَ أَهْوَآءَهُم مِّن بَعْدِ مَا جَآءَكَ مِنَ الْعِلْمِ إِنَّكَ إِذَا لَّمِنَ الظَّـلِمِينَ

(Verily, if you follow their desires after that which you have received of knowledge (from Allah), then indeed you will be one of the wrongdoers.)