Nextcloud Sorts : a Nextcloud application prototype to navigate your files more easily

What if Framasoft took a look at the free/libre collaborative software Nextcloud ?

 

In our newsletter #28 (Autumn 2021), we talked about Romain, an intern at Framasoft, whose internship topic was related to Nextcloud.

A few weeks later, we invited  » socially committed  » structures using Nextcloud to answer a survey (now closed), created with the designer Marie-Cécile Godwin, and La Fabrique à Liens.

Then, once the results were analyzed (you will find an anonymized raw version and a synthetic analysis at the end of the article), we were able to identify a set of unmet user needs, on which Romain could work.

Nextcloud provides loads of features, but the one that remains the most central is probably the storage and sharing of files. However, browsing files in the web interface is quite tedious : a click each time you change folder, and thus a more or less fast reloading of the tree structure. And sometimes many clicks to go from one branch of the tree to another.

There was probably a way to make it more accessible and more intuitive.

Romain, who knew neither the PHP computer language nor the Nextcloud software solution when beginning his internship, indulged himself with the development of a prototype of a third-party application, which would not only allow to « open the file tree » within the same interface, but also to be able to make advanced searches.

Two months later, the « Sorts » plugin was born, and we would like to share its story below.

---

Hello Romain, can you introduce yourself ?

My name is Romain and I am 24 years old. I grew up in Guadeloupe (French West Indies) before coming to study in Villeurbanne, at the Institut National des Sciences Appliquées in Lyon, where I will graduate in 2022 as a Telecom Engineer.

When I’m not busy with my studies, I spend a lot of time tinkering and repairing old machines, and getting involved in community projects ! I am particularly involved in Karnaval Humanitaire, a student association that organizes a music festival for which I was the site manager in 2021 and 2022.

Picture of Romain, INSA Lyon intern at Framasoft from September 2021 to February 2022

 

Why did you choose Framasoft for your internship ?

I wanted my internship to be in line with my values of free sharing of knowledge, and away from capitalism and big companies !

I already knew a bit about Framasoft’s projects, especially the online services and software development, and I knew I would find interesting projects in a great environment… and I was not disappointed !

Let’s come to the subject of your internship. What was the main goal ?

My internship took place at the beginning of a larger project, codenamed « Framacloud » [Framasoft’s note: we’ll be telling you more about this ambitious project in the upcoming months], whose goal is to allow structures fighting for social progress and social justice to gain ownership, master and control over digital collaboration processes.

This Framasoft project is focused on an online collaboration and file sharing software solution : Nextcloud.

However, although it is one of the most successful and complete open source solutions in this field, it is primarily designed to meet the needs of customers of « Nextcloud GmbH« , the German company that publishes the software. These customers are large structures, public, university or private. As a result, there is a risk of differences between the expectations of small associations and the development priorities of Nextcloud GmbH.

The goal of my internship was therefore to find out how to improve this software in order to make it more useful and more accessible for alternative organisations.

OK, that’s a big topic ! How did you manage about it ?

Well, first of all, I had to learn about the development of Nextcloud, and we had to learn more about this software : how it works, its flaws and especially what was missing for the users we were targeting.

After several tests of the software and several hypotheses on how to improve it, we decided to get closer to our target users. So we set up a survey targeting people who were part of organisations committed to social progress and social justice and who were already using Nextcloud. This survey questioned their uses of collaborative computing within the collective, their uses of Nextcloud, their frustrations and expectations.

Thanks to the almost 200 answers of this survey we decided on the software developments to be achieved during this internship and that led to the creation of a Nextcloud plugin called « Sorts » !

 

Let’s focus a moment on this survey work. What were the results ?

The main concerns that came out of this survey were rather general concerns about the Nextcloud tool. But it allowed us to see what is important for the public we wanted to address.

Of the 20 or so topics I was able to identify in the responses, the first two were topics we couldn’t do much about : collaborative document editing and the general « slowness » of the tool.
On the other hand, among the following subjects we had more perspective to help change things in a few months of training : handling and ergonomics of the software, synchronization problems or help to users to find their way among the files of the collective.

It is this last concern : « finding one’s way in the tree of files more easily » that interested me the most, and that led me to the development of « Sorts ».

For those who want to have a look at the details of the survey, we have published the anonymized results, as well as a synthesis of the different subjects I have discussed in order to refine the subject of the workshop (it is probably easier to digest than the spreadsheet of raw results).

[Framasoft’s note: You can find the results of the survey at the end of the article]

So, you decided to create the Nextcloud application « Sorts ». But… what does this plugin actually do ?

The idea behind Sorts is to help people find the files they are looking for and to understand how the folders and files have been organized by the collective they are part of.

To solve the first problem we took advantage of all the information about each file that Nextcloud was already storing (modification date, weight, « tagging » of the file by the user, …) and I coded an interface that allows users to perform a search mixing these different attributes. For example « Find me all the files in the folder « Grant » and its subfolders that are marked as important, and that are in PDF format ».

 

 

To solve the second problem we decided to present the folders, subfolders and files in a way that was not yet present in Nextcloud : in a tree list. That is to say that when you click on a folder, instead of « entering » this folder and seeing only its direct content, the folder is « expanded » and you see its content as well as the folders and files which are « next to » it. This tree-like list takes up more space than a simple list but it allows you to understand where you are in the folders, which helps you to understand how they are arranged.

 

 

Technically, did you encounter any issues ?

Yes, like in any development process. I think one of the big challenges was to find out which part of the Nextcloud APIs to use, what were its limitations and how to deal with it. The Nextcloud current search feature is designed around a « grouped » search (« unified search ») where the user searches for a string of characters, and Nextcloud returns everything that matches this string among the various resources (files, todos, events, emails, conversations, …). This is not at all what we wanted to do : search among files only according to several conditions, some of which are not strings (dates, numbers, …). But, fortunately for us, there was another search API specific to files. This other API looked very promising because it was already thought to allow combining search conditions on file-specific attributes. However, this API was quite old and not widely used, which sometimes gave me a bit of trouble.

Moreover, I realized quite lately that the API did not take into account two of the different file attributes : « tags » and file sharing information. This information is managed in totally separate APIs. So I faced this dilemma : either I rewrite an API that does the database queries itself with all the attributes, or I complete the existing API, or I cobble together something where I do 3 database queries per search and combine the results. The first solution would have taken too much time and the second solution would have been rejected by Nextcloud GmbH (you don’t change Nextcloud APIs so easily), so I cobbled something together, and so much for the performance of the application.

Did you have contacts with the Nextcloud community or its editor (Nextcloud Gmbh), and if so, how did it go ?

Yes of course, when we started to have a clear idea of what we wanted to do with Sorts I wrote a note of intent with a link to a prototype on the Nextcloud plugin development forum. This led to some exchanges with Nextcloud’s employed developers who were interested in the project and sent me some constructive feedback. We even had a video meeting with them to discuss the plugin but also the wider project, but with everyone’s schedules this meeting took place quite late in the development of Sorts and it didn’t impact the plugin much.

And now, the nagging question : is Sorts really working ?

Well YES ! 🎉 Sorts offers an « unfolding » file tree view and already allows combined searches on a good variety of characteristics that files can have !

However, this is a Beta version and it has some limitations… I had to make some accommodations with the technical issues mentioned in the previous question, and while the current version works on small Nextcloud instances, it will likely have trouble to scale when working with real instances counting hundreds of users and thousands of files.

Nextcloud Sorts 0.1.0-beta feature demo video (source)

Features of Nextcloud Sorts 0.1.0-beta (source)

 

What’s next ? What do you think is next, and when ?

Sorts is available on https://packages.framasoft.org and on the Nextcloud app store in beta version !

We want to continue to maintain this application and to process and accept all possible contributions, but neither I nor Framasoft have plans to develop it full time at the moment. Sorts is now entering the world of community/voluntary development, which means that the developments carried by Framasoft and myself will be done according to our desires and availabilities, without any particular agenda (which also means that we won’t announce any « release date »).

We are coming to the end of this interview. Would you like to share with us a feeling about the work done during this workshop ?

Summarize an internship in one emotion ? That’s a tough one ! Developing a program goes from frustration when it doesn’t work, to the excitement of investigating why it doesn’t work, to the satisfaction of seeing the feature work when you’ve found it.

No, more seriously, there were some frustrations like not having much time to develop or not finding as much time and motivation as I would have liked to complete the project after the internship, but I am satisfied. Satisfied to have made something that works but especially to have been able to design this plugin from almost the beginning to almost the end, taking the time to identify what could be useful, to think about what it should look like, and then to think about how to realize it technically.

Last question, recurrent in our interviews : what is the question you would have liked to be asked, and what would be your answer ?

Why didn’t you publish Sorts earlier ?

That’s my big regret ! And I think that the people who were interested in the application during the note of intent also asked themselves that question. But my associative and personal life was quite busy after the internship and didn’t leave me much time to take care of Sorts, that’s also what volunteer development is about.

Thank you Romain ! And to all the people who made this work possible, especially by answering the questionnaire !

Reminder of the different links mentioned in the article :

• Results of the anonymized survey (raw format .csv or LibreOffice Calc format .ods)
• Quick analysis of the survey, in french, by Romain (.pdf format)
• Romain’s internship report (in english. Also available in french)
• Source code on Framagit
• Download and install Sorts application on the Nextcloud App store (Warning : beta version, use at your own risk. However, the risks are limited because the Sorts application does not make any modification on your files)