I moved mountains today to make sorting work correctly, and it just doesn’t. I am really frustrated by this and I don’t understand if the problem is the setup or there are some limitations or a bug.
I have special characters in my alphabet (Slovenian, ČŠŽ).
At first I was using simple SQLite setup and I had numerous problems with sorting.
Then I moved to PostgreSQL docker setup, which didn’t help either as the default/only locale is en_US, which didn’t help me at all.
Then I moved to host PostgreSQL installation, added locales, even tried with ICU.
Now the situation is slightly better, but still a nightmare.
I have Departments and I have Students.
Student list is ordered correctly by surname.
But when i open department and there’s a list of students, it just isn’t ordered correctly for some reason.
It is worth mentioning again, that psql sort on column works fine, it is also correctly sorted on the main collection - it is only the related records that do not get sorted correctly. If the surname starts with a special charcater it is just placed as the last/first in order - this tells me the related sort is being done differently under the hood compared to main collection sort. Why?
Why? Why is this so hard? Not everything revolves around English.
There are other languages in the world.
In the last 14 days setting up my project I have hit so many obstacles with Directus, that seem totally unnecessary, yet they are there. I am slowly loosing hope in using it at all and just moving the other direction, because I am fighting with it every single day. So close - yet so far.
I make sorting on main collection, go to postgres logs and find the select statement being made
I try and query with same select and it works perfectly, sort correct
then I go to collection that has associated records (Department) - sorting is wrong
I again find this query in the logs
try exactly the same query in psql, works correctly, sort being done perfectly
then I find the API call being made in developers console
the returned array of records is sorted correctly
but the display isn’t
So there is no problem with the Database, there is nothing wrong with the setup as far as I understand.
Can anyone from Directus please answer why this is happening and if there is a workaround.
Maybe it is a bug, which is fine… at least we know it is there.
Hey @Svashtar, I’m sorry to hear that you are having trouble with getting set up, that’s definitely not what we aim to have as an experience.
I’ve forwarded on your problem internally (thank you very much for providing the extra details) and we recommend opening up an issue so we can look into it as it does seem strange. Sorry I haven’t got a clear answer for you at this point.
Of course. As I mentioned earlier, the response returned is ordered correctly.
Also, it does not make any difference in setting this default order. I can set it to whatever I want, it is just a default order. When I order the column however (default or manual) it is not ordered correctly.
Since also the column widths are being lost/reset, I am pretty sure this table is done client side.
And client side means, whatever the JS library is used, it orders it again (I guess) by its own rules and has no idea/information on the collation being used, it just orders it by ASCII I guess.
It should just respect the JSON order returned and display it as is, otherwise there is no advantage in sorting and returning the sorted response. It might as well just return unsorted and leave it to JS to sort it - but that’s wrong.
Examples:
Students collection, sorted and filter for department “1. B” (main table layout for students collection).
Sorted correctly in response, also sorted correctly in the view/table:
Natural order is ABCČDĐ … RTSŠ … ZŽ
In the table, the “specials” are just placed as last, as whatever is doing the ordering doesn’t know where to place them.
Sorry for a bunch of screens, but I don’t know how to explain it better.
Screens are very helpful! Ok just to be sure, you are using the standard interface for a O2M field and you’re in the “table” view from the screenshots.
Do yo see the same behaviour when you open the Students collection directly and manually filter by your department?
Maybe it’s that the decomposed caron value might be then just be omitted in the sorting process.
“C” → U+0043
“Č” → U+0043 + U+030C
I might be off here tho — collation (and time zones) rank top on my “developer nightmare” list
Anyway, I think those two functions should/could use collation aware sorting, especially since the database usually sorts as expected “natural language” given the right collation is set on a table, while nested or relational data then might be (re)sorted differently by Directus.
As I said, it doesn’t make a lot of sense to send a sort request, sort it on server, return sorted response - only for JavaScript to sort it again.
I would understand, if the sorting on the table would be done client side (when clicking column sort), but it does trigger the request, so why sort it again
IDK, don’t want to question implementation, but this is what is happening I guess
