2021-09-09 00:05:51 +02:00
Development and deployment
==========================
2023-06-10 23:02:03 +02:00
There are various scripts to help set up MapComplete for developing and for deployment.
2021-09-09 00:05:51 +02:00
This documents attempts to shed some light on these scripts.
Note: these scripts change every now and then - if the documentation here is incorrect or you run into troubles, do
2025-04-07 19:10:11 +02:00
leave a message in [the issue tracker ](https://source.mapcomplete.org/MapComplete/MapComplete/issues )
2021-09-09 00:05:51 +02:00
Architecture overview
---------------------
At its core, MapComplete is a static (!) website. There are no servers to host.
The data is fetched from Overpass/OSM/Wikidata/Wikipedia/Mapillary/... and written there directly. This means that any
static file server will do to create a self-hosted version of MapComplete.
2023-06-10 23:02:03 +02:00
Dependencies
------------
2021-09-09 00:05:51 +02:00
2023-06-10 23:02:03 +02:00
`make` , `python3` , `g++`
(Nix users may run `nix-env -iA nixos.gnumake nixos.gdc nixos.python3` )
2021-09-09 00:05:51 +02:00
2023-06-10 23:02:03 +02:00
Development using *NIX
----------------------
You need at least 3GB RAM available to run MapComplete, but you'll preferably have 8GB of free RAM available.
2023-01-03 12:54:23 +01:00
2021-09-09 00:05:51 +02:00
To develop and build MapComplete, you
2021-05-07 02:18:52 +02:00
2022-05-01 18:46:05 +02:00
0. Make a fork and clone the repository. (We recommend a shallow clone with `git clone --filter=blob:none <repo>` )
2025-07-18 22:00:37 +02:00
1. Install `python3` if you do not have it already - On Linux: `sudo apt install python3` or `sudo dnf install python3`
2023-03-09 17:11:44 +01:00
2. Install `nvm` to easily install node:
- `wget -qO- https://raw.githubusercontent.com/nvm-sh/nvm/v0.37.2/install.sh | bash`
- Restart your terminal
- Run `nvm install` and `nvm use` to install and use the correct version of node. (_Note: nvm might complain that the relevant version is not yet installed. It'll have it installed only for the current user account but not system-wide - which is fine)
2025-07-18 22:00:37 +02:00
3. Run `npm run init` (including **run** , not ~~`npm init`~~ )which …
2023-03-09 17:11:44 +01:00
- runs `npm ci` for you
- generates some additional dependencies and files
2023-06-10 23:02:03 +02:00
- does various housekeeping and setup. This can take a few minutes the first time as some PNGs need to be created
2025-07-18 22:00:37 +02:00
4. Run `npm run start` to host a local testversion at http://127.0.0.1:1234/
5. By default, a landing page with available themes is served. In order to load a single theme, use `layout=themename`
2023-06-10 23:02:03 +02:00
or `userlayout=true#<layout configuration>` as [Query parameter ](URL_Parameters.md ). Note that the shorter URLs
(e.g. `bookcases.html` , `aed.html` , ...) _don't_ exist on the development version.
2021-07-13 13:57:31 +02:00
2023-06-10 23:02:03 +02:00
The previous instructions were tested on 2023-03-09 on a Ubuntu 22.04 machine.
2023-03-09 17:11:44 +01:00
2021-09-09 00:05:51 +02:00
Development using Windows
2023-06-10 23:02:03 +02:00
-------------------------
You need at least 3GB RAM available to run MapComplete, but you'll preferably have 8GB of free RAM available.
2021-07-13 13:57:31 +02:00
2025-07-18 22:00:37 +02:00
For Windows you can use the WSL subsystem.
2021-07-13 13:57:31 +02:00
2021-09-09 00:05:51 +02:00
To use the WSL in Visual Studio Code:
2021-07-13 13:57:31 +02:00
2025-07-18 22:00:37 +02:00
1. Make sure you have installed the [Remote - WSL ](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-wsl ) extension and it's dependencies.
2. Open a remote WSL window using the button in the bottom left.
3. Make a fork and clone the repository.
4. Install `npm` using `sudo apt install npm` .
5. Run `npm run init` and generate some additional dependencies and generated files. Note that it'll install the
2021-09-09 00:05:51 +02:00
dependencies too
2025-07-18 22:00:37 +02:00
6. Run `npm run start` to host a local testversion at http://127.0.0.1:1234/index.html
7. By default, a landing page with available themes is served. In order to load a single theme, use `layout=themename`
2021-09-09 00:05:51 +02:00
or `userlayout=true#<layout configuration>` as [Query parameter ](URL_Parameters.md ). Note that the shorter URLs (
e.g. `bookcases.html` , `aed.html` , ...) _don't_ exist on the development version.
2025-02-05 14:58:40 +01:00
Developing a local theme
------------------------
To test theme changes you locally make, the easiest way is:
- `npm run strt` which will run the server
- After making changes to some layers and/or themes, run `npm run generate:layeroverview` . This will recompile the relevant themes.
- You might need to refresh (F5) your browser window
2025-02-05 16:59:36 +01:00
Note that `npm run start` is equivalent to `npm run generate:layeroverview && npm run strt`
2025-02-05 14:58:40 +01:00
2021-09-09 00:05:51 +02:00
Automatic deployment
--------------------
2025-07-18 22:00:37 +02:00
MapComplete is automatically deployed to various locations, depending on the branch you are working on.
2021-05-07 02:18:52 +02:00
2025-07-18 22:00:37 +02:00
The master branch is automatically deployed to https://mapcomplete.org/, and the development branch is available at
https://dev.mapcomplete.org/. Both are automatically deployed by a Forgejo action.
2021-05-07 02:18:52 +02:00
2025-07-18 22:00:37 +02:00
Other branches are automatically deployed to https://builds.mapcomplete.org/< branchname > .
2021-06-04 17:00:36 +02:00
2021-09-09 00:05:51 +02:00
Deploying a fork
----------------
2021-06-04 17:00:36 +02:00
2021-09-09 00:05:51 +02:00
A script creates a webpage for every theme automatically, with some customizations in order to:
2023-06-10 23:02:03 +02:00
- to have shorter URLs
2021-09-09 00:05:51 +02:00
- have individual social images
- have individual web manifests
This script can be invoked with `npm run prepare-deploy`
2021-05-07 02:18:52 +02:00
If you want to deploy your fork:
2025-07-18 22:00:37 +02:00
1. `npm run prepare-deploy`
2. `npm run build`
3. Copy the entire `dist` folder to where you host your website. Visiting `index.html` gives you the landing page,
2021-09-09 00:05:51 +02:00
visiting `yourwebsite/<theme>` should bring you to the appropriate theme.
2021-05-07 13:59:40 +02:00
2024-11-25 00:58:00 +01:00
### Getting your own API-keys
Some services are bound to `https://mapcomplete.org` . In `package.json/config` , search for "#fork " and read the instructions.
2021-05-07 13:59:40 +02:00
Weird errors
------------
2023-06-10 23:02:03 +02:00
Try removing `node_modules` , `package-lock.json` and `.cache` .
2021-05-07 13:59:40 +02:00
2021-10-01 04:59:22 +02:00
Misc setup
----------
2022-01-26 21:40:38 +01:00
~~The json-git-merger is used to quickly merge translation
files, [documentation here ](https://github.com/jonatanpedersen/git-json-merge#single-project--directory ).~~
2021-11-08 03:22:00 +01:00
This merge driver is broken and would sometimes drop new questions or duplicate them... Not a good idea!
2021-10-01 04:59:22 +02:00
2021-09-09 00:05:51 +02:00
Overview of package.json-scripts
--------------------------------
2023-06-10 23:02:03 +02:00
- `increase-memory` : give Node.js some more RAM since this is a big (and memory intensive) project to build and run
- `start` : start a development server
2021-09-09 00:05:51 +02:00
- `test` : run the unit tests
2023-06-10 23:02:03 +02:00
- `init` : generate and download various assets which are needed to compile
- `generate:editor-layer-index` : download the editor-layer-index-json from osmlab.github.io
- `generate:images` : compile the SVGs into an asset
- `generate:translations` : compile the translation file into a JavaScript file
- `generate:layouts` : use `index.html` as template to create all the theme index pages. You'll want to run `clean` when
2021-09-09 00:05:51 +02:00
done
2023-06-10 23:02:03 +02:00
- `generate:docs` : generate various documents, such as information about available metatags, information to put on
the [OSM wiki ](https://wiki.openstreetmap.org/wiki/MapComplete ), ...
- `generate:report` : download statistics from OsmCha, compile neat graphs
- `generate:cache:speelplekken` : create an offline copy of all the data required for one specific (paid for) theme
- `generate:layeroverview` : read all the theme and layer configurations, compiles them into a single JSON.
- `reset:layeroverview` : if something is wrong with the layer overview, create an empty one
- `generate:licenses` : compile all the license info of images into a single JSON
- `optimize:images` : attempt to make smaller PNGs - optional to run before a deployment
- `generate` : run all the necessary generate-scripts
- `build` : actually bundle all the files into a single `dist/` folder
2021-09-09 00:05:51 +02:00
- `prepare-deploy` : create the layouts
- `deploy:staging` ,`deploy:pietervdvn` , `deploy:production` : deploy the latest code on various locations
- `lint` : get depressed by the amount of warnings
- `clean` : remove some generated files which are annoying in the repo
