2021-09-09 00:05:51 +02:00
Development and deployment
==========================
2023-06-10 23:02:03 +02:00
There are various scripts to help set up MapComplete for developing and for deployment.
2021-09-09 00:05:51 +02:00
This documents attempts to shed some light on these scripts.
Note: these scripts change every now and then - if the documentation here is incorrect or you run into troubles, do
leave a message in [the issue tracker ](https://github.com/pietervdvn/MapComplete/issues )
Architecture overview
---------------------
At its core, MapComplete is a static (!) website. There are no servers to host.
The data is fetched from Overpass/OSM/Wikidata/Wikipedia/Mapillary/... and written there directly. This means that any
static file server will do to create a self-hosted version of MapComplete.
2023-06-10 23:02:03 +02:00
Dependencies
------------
2021-09-09 00:05:51 +02:00
2023-06-10 23:02:03 +02:00
`make` , `python3` , `g++`
(Nix users may run `nix-env -iA nixos.gnumake nixos.gdc nixos.python3` )
2021-09-09 00:05:51 +02:00
2023-06-10 23:02:03 +02:00
Development using *NIX
----------------------
You need at least 3GB RAM available to run MapComplete, but you'll preferably have 8GB of free RAM available.
2023-01-03 12:54:23 +01:00
2021-09-09 00:05:51 +02:00
To develop and build MapComplete, you
2021-05-07 02:18:52 +02:00
2022-05-01 18:46:05 +02:00
0. Make a fork and clone the repository. (We recommend a shallow clone with `git clone --filter=blob:none <repo>` )
2023-06-10 23:02:03 +02:00
1. Install `python3` if you do not have it already - On Linux: `sudo apt install python3`
2023-03-09 17:11:44 +01:00
2. Install `nvm` to easily install node:
- `wget -qO- https://raw.githubusercontent.com/nvm-sh/nvm/v0.37.2/install.sh | bash`
- Restart your terminal
- Run `nvm install` and `nvm use` to install and use the correct version of node. (_Note: nvm might complain that the relevant version is not yet installed. It'll have it installed only for the current user account but not system-wide - which is fine)
4. Run `npm run init` (including **run** , not ~~`npm init`~~ )which …
- runs `npm ci` for you
- generates some additional dependencies and files
2023-06-10 23:02:03 +02:00
- does various housekeeping and setup. This can take a few minutes the first time as some PNGs need to be created
2023-03-09 17:11:44 +01:00
5. Run `npm run start` to host a local testversion at http://localhost:1234/
6. By default, a landing page with available themes is served. In order to load a single theme, use `layout=themename`
2023-06-10 23:02:03 +02:00
or `userlayout=true#<layout configuration>` as [Query parameter ](URL_Parameters.md ). Note that the shorter URLs
(e.g. `bookcases.html` , `aed.html` , ...) _don't_ exist on the development version.
2021-07-13 13:57:31 +02:00
2023-06-10 23:02:03 +02:00
The previous instructions were tested on 2023-03-09 on a Ubuntu 22.04 machine.
2023-03-09 17:11:44 +01:00
2021-09-09 00:05:51 +02:00
Development using Windows
2023-06-10 23:02:03 +02:00
-------------------------
You need at least 3GB RAM available to run MapComplete, but you'll preferably have 8GB of free RAM available.
2021-07-13 13:57:31 +02:00
2021-09-09 00:05:51 +02:00
For Windows you can use the devcontainer, or the WSL subsystem.
2021-07-13 13:57:31 +02:00
2021-09-09 00:05:51 +02:00
To use the devcontainer in Visual Studio Code:
2021-07-13 13:57:31 +02:00
2021-09-09 00:05:51 +02:00
0. Make sure you have installed
the [Remote - Containers ](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers )
extension and it's dependencies.
2021-07-13 13:57:31 +02:00
1. Make a fork and clone the repository.
2. After cloning, Visual Studio Code will ask you if you want to use the devcontainer.
3. Then you can either clone it again in a volume (for better performance), or open the current folder in a container.
4. By now, you should be able to run `npm run start` to host a local testversion at http://localhost:1234/index.html
2021-09-09 00:05:51 +02:00
5. By default, a landing page with available themes is served. In order to load a single theme, use `layout=themename`
or `userlayout=true#<layout configuration>` as [Query parameter ](URL_Parameters.md ). Note that the shorter URLs (
e.g. `bookcases.html` , `aed.html` , ...) _don't_ exist on the development version.
2021-07-13 13:57:31 +02:00
2021-09-09 00:05:51 +02:00
To use the WSL in Visual Studio Code:
2021-07-13 13:57:31 +02:00
0. Make sure you have installed the [Remote - WSL]() extension and it's dependencies.
1. Open a remote WSL window using the button in the bottom left.
2. Make a fork and clone the repository.
3. Install `npm` using `sudo apt install npm` .
2021-09-09 00:05:51 +02:00
4. Run `npm run init` and generate some additional dependencies and generated files. Note that it'll install the
dependencies too
2021-07-13 13:57:31 +02:00
5. Run `npm run start` to host a local testversion at http://localhost:1234/index.html
2021-09-09 00:05:51 +02:00
6. By default, a landing page with available themes is served. In order to load a single theme, use `layout=themename`
or `userlayout=true#<layout configuration>` as [Query parameter ](URL_Parameters.md ). Note that the shorter URLs (
e.g. `bookcases.html` , `aed.html` , ...) _don't_ exist on the development version.
Automatic deployment
--------------------
2023-08-23 18:33:30 +02:00
Currently, the master branch is automatically deployed to https://mapcomplete.org/ by a GitHub action.
2021-05-07 02:18:52 +02:00
2023-06-10 23:02:03 +02:00
Every branch is automatically built (upon push) to `https://pietervdvn.github.io/mc/<branchname>` by a GitHub action.
2021-05-07 02:18:52 +02:00
2021-06-04 17:00:36 +02:00
2021-09-09 00:05:51 +02:00
Deploying a fork
----------------
2021-06-04 17:00:36 +02:00
2021-09-09 00:05:51 +02:00
A script creates a webpage for every theme automatically, with some customizations in order to:
2023-06-10 23:02:03 +02:00
- to have shorter URLs
2021-09-09 00:05:51 +02:00
- have individual social images
- have individual web manifests
This script can be invoked with `npm run prepare-deploy`
2021-05-07 02:18:52 +02:00
If you want to deploy your fork:
0. `npm run prepare-deploy`
1. `npm run build`
2021-09-09 00:05:51 +02:00
2. Copy the entire `dist` folder to where you host your website. Visiting `index.html` gives you the landing page,
visiting `yourwebsite/<theme>` should bring you to the appropriate theme.
2021-05-07 13:59:40 +02:00
Weird errors
------------
2023-06-10 23:02:03 +02:00
Try removing `node_modules` , `package-lock.json` and `.cache` .
2021-05-07 13:59:40 +02:00
2021-10-01 04:59:22 +02:00
Misc setup
----------
2022-01-26 21:40:38 +01:00
~~The json-git-merger is used to quickly merge translation
files, [documentation here ](https://github.com/jonatanpedersen/git-json-merge#single-project--directory ).~~
2021-11-08 03:22:00 +01:00
This merge driver is broken and would sometimes drop new questions or duplicate them... Not a good idea!
2021-10-01 04:59:22 +02:00
2021-09-09 00:05:51 +02:00
Overview of package.json-scripts
--------------------------------
2023-06-10 23:02:03 +02:00
- `increase-memory` : give Node.js some more RAM since this is a big (and memory intensive) project to build and run
- `start` : start a development server
2021-09-09 00:05:51 +02:00
- `test` : run the unit tests
2023-06-10 23:02:03 +02:00
- `init` : generate and download various assets which are needed to compile
- `generate:editor-layer-index` : download the editor-layer-index-json from osmlab.github.io
- `generate:images` : compile the SVGs into an asset
- `generate:translations` : compile the translation file into a JavaScript file
- `generate:layouts` : use `index.html` as template to create all the theme index pages. You'll want to run `clean` when
2021-09-09 00:05:51 +02:00
done
2023-06-10 23:02:03 +02:00
- `generate:docs` : generate various documents, such as information about available metatags, information to put on
the [OSM wiki ](https://wiki.openstreetmap.org/wiki/MapComplete ), ...
- `generate:report` : download statistics from OsmCha, compile neat graphs
- `generate:cache:speelplekken` : create an offline copy of all the data required for one specific (paid for) theme
- `generate:layeroverview` : read all the theme and layer configurations, compiles them into a single JSON.
- `reset:layeroverview` : if something is wrong with the layer overview, create an empty one
- `generate:licenses` : compile all the license info of images into a single JSON
- `optimize:images` : attempt to make smaller PNGs - optional to run before a deployment
- `generate` : run all the necessary generate-scripts
- `build` : actually bundle all the files into a single `dist/` folder
2021-09-09 00:05:51 +02:00
- `prepare-deploy` : create the layouts
- `deploy:staging` ,`deploy:pietervdvn` , `deploy:production` : deploy the latest code on various locations
- `lint` : get depressed by the amount of warnings
- `clean` : remove some generated files which are annoying in the repo
