First draft of PinJePunt analysis

Docs/Reasonings/PinJePunt_One_Year_Later.md

@@ -1,13 +1,88 @@
-# 'Pin je Punt' - one year later
+)# 'Pin je Punt' - one year later
 
 About a year ago, we launched a mapping campaign at the request from [_Visit Flanders_ (Toerisme Vlaanderen)](toerismevlaanderen.be/pin-je-punt). (The project is explained below)
 
 A part of the campaign involved a guided import. The agencies had many datasets lying around (e.g. about benches or picnic tables) which they wanted to have imported in OSM. As doing a data import is hard - and the data was sometimes outdated, we opted for another approach: for every possible feature, a map note was created containing a friendly explanation and instructions to open MapComplete - which would close the map note on behalf of the contributor, marking them "imported", "not found" or "not valid" depending on what the contributor chose.
 
-Most map notes are closed by now, but the central question in this analysis today is: _should remaining map notes be closed in batch, or do we leave them open for longer_? Note that input of the local community will be gathered as well.
+Most map notes are closed by now, but the central question in this analysis today is: _should remaining map notes be closed in batch, or do we leave them open for longer_? Note that input of the local community will be gathered as well - this article will mostly serve as a point to start the discussion.
 
-## 
+## The datasets
 
+Various datasets were provided to upload - which were converted into notes. In the table below, you'll find a breakdown by topic, the date when they were uploaded, the number of notes created and how much of those notes were already closed and the top contributors for the category.
+
+In this table, I'm not including if the feature has been added to OpenStreetMap, has been marked as not existing anymore or marked as being a duplicate.
+
+Most of those notes have been opened by a [dedicated account](https://openstreetmap.org/user/Toerisme%20Vlaanderen%20-%20Pin%20je%20punt), except for two imports which accidentally did not use this account (noted in the table below).
+
+
+| Feature type (source)                                                | date        | Number of features | Handled | Handled percentage | Contributor (closed notes)                                                                                                                                |
+|----------------------------------------------------------------------|-------------|--------------------|---------|--------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------| 
+| Toilets (Toerisme Limburg) __Notes created by Pieter Vander Vennet__ | January 22 | 177 | 84 | 47% | Eebie (26), Pieter Vander Vennet (11), phillipec (9)                                                                                                      |   
+| Benches or maybe a picnic table (Toerisme Limburg)                   | february 22 | 730       | 722                | 99%                | Eebie (378),A127 (90), joost schouppe (36), dentonny (34), jozin-belgium (32), mjans (23), Pieter Vander Vennet (13), kersentaart (13), proudtobeevi (11) |
+| Playgrounds (Toerisme Vlaams-Brabant)                                | march 22    | 51          | 46                 | 90%                | Pieter Vander Vennet (23), Joost Schouppe (6)                                                                                                             |
+| Picnic tables (Toerisme Vlaams-Brabant)                              | march 22    | 222         | 172                | 77% | A127 (44), Eebie (15), Joost (15)                                                                                                                         |
+| Ebike Charging station (Toerisme Vlaams-Brabant)                     | march 22    | 20          | 13                 | 65%  |                                                                                                                                                           | 
+| Picnic Tables (Toerisme Oost-Vlaanderen)                             | march '22   | 33          | 28                 | 85% | Joost Schouppe (10), A127 (5), L'Imaginaire (5)                                                                                                           |
+| Cycle rental (Toerisme West-Vlaanderen)                              | april '22   | 5           | 5                  | 100% |                                                                                                                                                           |
+| Benches (Toerisme Antwerpen)                                         | april '22   | 54          | 44                 | 81% | pi11 (11), A127 (7)                                                                                                                                       |
+| Picnic table (Toerisme Antwerpen)                                    | april '22   | 91          | 73                 | 80% | Jakka (21), wegspotter (13), pi11 (12)                                                                                                                    |
+| picnictable (Westtoer) __Made by L'imaginaire__                      | april '22 | 340 | 296 | 87% | L'imaginaire (250), Jakka (10)                                                                                                                            |
+| Blue bike cycle rental (website scrape by Pieter Colpaert)           | may 22      | 60          | 44                 | 73% | Pieter Vander Vennet (12), Joost Schouppe (4)                                                                                                             | 
+| Benches (Toerisme Vlaanderen Kempen&Maasland)                        | may 22      | 94          | 88                 | 94% | Eebie (49), A127 (13), Joost Schouppe (8)                                                                                                                 |
+| Benches (Open Data Oostende)                                         | june '22    | 1044 (!) | 686                | 66% | A127 (520), Joost Schouppe (41), L'imaginaire (36), Jakka (23)                                                                                            |
+
+
+## Over time
+
+The following shows a graph of open notes for this campaign over time. The blueish line shows the total amount of Open Notes, which sharply jumps upwards when a new dataset was added.
+
+Other lines represent the amount of notes closed by an individual contributor. As is visible, A127 and Eebie have done a tremendous amount of work, whereas around 20 other contributors have contributed a modest amount of points.
+
+![](./AllStatistics.png)
+
+The amount of work represents a clear power curve, with most of the work done by a few contributors and many contributors with a few imports.
+
+![](./ContributorsPie.png)
+
+Another interesting graph is how much of the features got imported and how much got refused. As it turns out, 53% of all features got imported. Note that, if the point gets added by using mapcomplete, the note will be closed with the message 'imported'. As many contributors used other editors, I'm checking for other keywords as well to mark them as 'imported', 'duplicate' or 'not_found'. Of course, humans are messy and the keyword-based approach is incomplete and inexact. 13% of closed notes could not be matched automatically.
+
+
+About 6% of the points were marked as 'could not be found'. Some of the manually closed notes indicate that the area has changed and the feature (often a bench) is removed. As such, this is a good indicator or the staleness of the source dataset.
+
+The fact that 6% of the features to import turned out not to exist anymore, this is a good argument for not blindly importing data into OSM! But this also poses that we should _maintain_ the map and that features such as benches should be checked regularly. Ideally, the municipality administration would integrate updating OSM into their flow...
+
+At last, even though no notes were created if a similar feature was already in OSM, about 10% of the notes was rejected as being a duplicate. This is partly because one dataset of benches turned out to also contain picnic tables - good for 168 'duplicate' entries, yet duplicates are quite common in other datasets too.
+
+## Some other numbers
+
+In total, **2921** notes have been created, of which **78%** has been handled - that are 2301 that have been reviewed and imported (or closed with an indication that they cannot/should not be imported).
+
+That is a huge effort, of which I would like to thank all involved. Especially **A127** who closed **684** notes and **Eebie** who handled **474** notes - your work is amazing!
+
+The work of A127 is amazing by the sheer volume of the work, but I want to give Eebie an extra thanks as he took it upon himself to search for the _toilets_. These were notoriously hard to find and to survey, as het often tried to use the actual toilet in social facilities, group nursing homes or administrative centers. These were often closed or unaware that they were listed as having a public toilet; at other times, those toilets were marked >100m away from the actual location.
+
+
+
+## Differences between the datasets
+
+The response on the datasets and the imports varied heavily by the type of the feature.
+
+The **benches and picnic tables** are relatively straightforward. Visiting the place - physically or virtually with aerial imagery or Mapillary - suffices to decide if the feature still exists. As such, those tasks got handled relatively quickly. Only where no Mapillary and no aerial imagery are available, the map notes remain.
+
+The other datasets proved to be harder. The **playgrounds** often needed some local knowledge - e.g. a playground might only be accessible to the members of the local youth organisation; or the adminstration eagerly labeled a patch of grass where kids could play some football as a proper playground. Some of them are hard to do remotely.
+
+The hardest dataset to handle are the **toilets**, especially toilets in municipality buildings and social facilities. They cannot be seen on aerial imagery by definition, neither is Mapillary available. Furthermore, these facilities are often subject to opening hours and the rules about use by the public might change. In other words, a survey is necessary for pretty much every feature to import.
+
+![](./StatePie.png)
+
+
+## Conclusions
+
+In hindsight, the guided import was a success. By creating a note, the process was very discoverable and many people helped out, including two 'hero importers' (but there are social limits to the amount of notes that can be created like this). For small datasets (<100 points), I would be tempted to automatically create this kind of notes again. For bigger datasets (especially if >500 points), I'd probably opt to use MapRoulette to store the data and to mark it as 'done', as not to pollute the notes with them.
+
+Especially small datasets which can be armchair-mapped have good levels of completion.
+
+As there still is some activity on the notes, there is no reason to close them. On the other hand, there isn't _much_ activity anymore. The datasets that are mostly done have a few very hard cases left where a survey is needed; but as only a few notes are still resting, it doesn't bother a lot of people...
 
 ## The project in a nutshell
 

UI/ImportFlow/ImportViewerGui.ts

@@ -209,6 +209,64 @@ class MassAction extends Combine {
 }
 
 class Statistics extends Combine {
+    private static r() {
+        return Math.floor(Math.random() * 256)
+    }
+
+    private static randomColour(): string {
+        return "rgba(" + Statistics.r() + "," + Statistics.r() + "," + Statistics.r() + ")"
+    }
+    private static CreatePieByAuthor(closed_by: Record<string, number[]>): ChartJs {
+        const importers = Object.keys(closed_by)
+        importers.sort((a, b) => closed_by[b].at(-1) - closed_by[a].at(-1))
+        return new ChartJs(<any>{
+            type: "doughnut",
+            data: {
+                labels: importers,
+                datasets: [
+                    {
+                        label: "Closed by",
+                        data: importers.map((k) => closed_by[k].at(-1)),
+                        backgroundColor: importers.map((_) => Statistics.randomColour()),
+                    },
+                ],
+            },
+        })
+    }
+
+    private static CreateStatePie(noteStates: NoteState[]) {
+        const colors = {
+            imported: "#0aa323",
+            already_mapped: "#00bbff",
+            invalid: "#ff0000",
+            closed: "#000000",
+            not_found: "#ff6d00",
+            open: "#626262",
+            has_comments: "#a8a8a8",
+        }
+        const knownStates = Object.keys(colors)
+        const byState = knownStates.map(
+            (targetState) => noteStates.filter((ns) => ns.status === targetState).length
+        )
+
+        return new ChartJs(<any>{
+            type: "doughnut",
+            data: {
+                labels: knownStates.map(
+                    (state, i) =>
+                        state + " " + Math.floor((100 * byState[i]) / noteStates.length) + "%"
+                ),
+                datasets: [
+                    {
+                        label: "Status by",
+                        data: byState,
+                        backgroundColor: knownStates.map((state) => colors[state]),
+                    },
+                ],
+            },
+        })
+    }
+
     constructor(noteStates: NoteState[]) {
         if (noteStates.length === 0) {
             super([])
@@ -285,11 +343,6 @@ class Statistics extends Combine {
                 },
             ],
         }
-
-        function r() {
-            return Math.floor(Math.random() * 256)
-        }
-
         for (const closing_user in closed_by) {
             if (closed_by[closing_user].at(-1) <= 10) {
                 continue
@@ -298,18 +351,17 @@ class Statistics extends Combine {
                 label: "Closed by " + closing_user,
                 data: closed_by[closing_user],
                 fill: false,
-                borderColor: "rgba(" + r() + "," + r() + "," + r() + ")",
+                borderColor: Statistics.randomColour(),
                 tension: 0.1,
             })
         }
-        const importers = Object.keys(closed_by)
+
-        importers.sort((a, b) => closed_by[b].at(-1) - closed_by[a].at(-1))
         super([
             new ChartJs({
                 type: "line",
                 data,
                 options: {
-                    scales: {
+                    scales: <any>{
                         yAxes: [
                             {
                                 ticks: {
@@ -320,21 +372,12 @@ class Statistics extends Combine {
                     },
                 },
             }),
-            new ChartJs({
+            new Combine([
-                type: "doughnut",
+                Statistics.CreatePieByAuthor(closed_by),
-                data: {
+                Statistics.CreateStatePie(noteStates),
-                    labels: importers,
+            ])
-                    datasets: [
+                .SetClass("flex w-full h-32")
-                        {
+                .SetStyle("width: 40rem"),
-                            label: "Closed by",
-                            data: importers.map((k) => closed_by[k].at(-1)),
-                            backgroundColor: importers.map(
-                                (_) => "rgba(" + r() + "," + r() + "," + r() + ")"
-                            ),
-                        },
-                    ],
-                },
-            }).SetClass("h-16"),
         ])
         this.SetClass("block w-full h-64 border border-red")
     }
@@ -653,25 +696,54 @@ class ImportInspector extends VariableUiElement {
                 | "already_mapped"
                 | "not_found"
                 | "has_comments" = "open"
+
+            function has(keywords: string[], comment: string): boolean {
+                return keywords.some((keyword) => comment.toLowerCase().indexOf(keyword) >= 0)
+            }
+
             if (prop.closed_at !== undefined) {
                 const lastComment = prop.comments[prop.comments.length - 1].text.toLowerCase()
-                if (lastComment.indexOf("does not exist") >= 0) {
+                if (has(["does not exist", "bestaat niet", "geen"], lastComment)) {
                     status = "not_found"
-                } else if (lastComment.indexOf("already mapped") >= 0) {
+                } else if (
+                    has(
+                        [
+                            "already mapped",
+                            "reeds",
+                            "dubbele note",
+                            "stond er al",
+                            "stonden er al",
+                            "staat er al",
+                            "staan er al",
+                            "stond al",
+                            "stonden al",
+                            "staat al",
+                            "staan al",
+                        ],
+                        lastComment
+                    )
+                ) {
                     status = "already_mapped"
                 } else if (
                     lastComment.indexOf("invalid") >= 0 ||
-                    lastComment.indexOf("incorrecto") >= 0
+                    lastComment.indexOf("incorrect") >= 0
                 ) {
                     status = "invalid"
                 } else if (
-                    [
+                    has(
-                        "imported",
+                        [
-                        "erbij",
+                            "imported",
-                        "toegevoegd",
+                            "erbij",
-                        "added",
+                            "toegevoegd",
-                        "openstreetmap.org/changeset",
+                            "added",
-                    ].some((keyword) => lastComment.toLowerCase().indexOf(keyword) >= 0)
+                            "gemapped",
+                            "gemapt",
+                            "mapped",
+                            "done",
+                            "openstreetmap.org/changeset",
+                        ],
+                        lastComment
+                    )
                 ) {
                     status = "imported"
                 } else {

css/index-tailwind-output.css

@@ -1141,6 +1141,10 @@ video {
   width: min-content;
 }
 
+.w-64 {
+  width: 16rem;
+}
+
 .w-auto {
   width: auto;
 }

langs/layers/nl.json

@@ -687,7 +687,7 @@
                         "then": "Wielerfietsen (sportfietsen) kunnen hier gehuurd worden"
                     },
                     "7": {
-                        "then": "Fietshelmpen kunnen hier gehuurd worden"
+                        "then": "Fietshelmen kunnen hier gehuurd worden"
                     }
                 },
                 "question": "Wat voor soort fietsen en fietstoebehoren worden hier verhuurd?",
@@ -8586,4 +8586,4 @@
             }
         }
     }
-}
+}