forked from MapComplete/MapComplete
153 lines
7.2 KiB
Markdown
153 lines
7.2 KiB
Markdown
Development and deployment
|
|
==========================
|
|
|
|
There are various scripts to help set up MapComplete for developing and for deployment.
|
|
|
|
This documents attempts to shed some light on these scripts.
|
|
|
|
Note: these scripts change every now and then - if the documentation here is incorrect or you run into troubles, do
|
|
leave a message in [the issue tracker](https://source.mapcomplete.org/MapComplete/MapComplete/issues)
|
|
|
|
Architecture overview
|
|
---------------------
|
|
|
|
At its core, MapComplete is a static (!) website. There are no servers to host.
|
|
|
|
The data is fetched from Overpass/OSM/Wikidata/Wikipedia/Mapillary/... and written there directly. This means that any
|
|
static file server will do to create a self-hosted version of MapComplete.
|
|
|
|
Dependencies
|
|
------------
|
|
|
|
`make` , `python3`, `g++`
|
|
|
|
(Nix users may run `nix-env -iA nixos.gnumake nixos.gdc nixos.python3`)
|
|
|
|
Development using *NIX
|
|
----------------------
|
|
|
|
You need at least 3GB RAM available to run MapComplete, but you'll preferably have 8GB of free RAM available.
|
|
|
|
To develop and build MapComplete, you
|
|
|
|
0. Make a fork and clone the repository. (We recommend a shallow clone with `git clone --filter=blob:none <repo>`)
|
|
1. Install `python3` if you do not have it already - On Linux: `sudo apt install python3` or `sudo dnf install python3`
|
|
2. Install `nvm` to easily install node:
|
|
- `wget -qO- https://raw.githubusercontent.com/nvm-sh/nvm/v0.37.2/install.sh | bash`
|
|
- Restart your terminal
|
|
- Run `nvm install` and `nvm use` to install and use the correct version of node. (_Note: nvm might complain that the relevant version is not yet installed. It'll have it installed only for the current user account but not system-wide - which is fine)
|
|
3. Run `npm run init` (including **run**, not ~~`npm init`~~)which …
|
|
- runs `npm ci` for you
|
|
- generates some additional dependencies and files
|
|
- does various housekeeping and setup. This can take a few minutes the first time as some PNGs need to be created
|
|
4. Run `npm run start` to host a local testversion at http://127.0.0.1:1234/
|
|
5. By default, a landing page with available themes is served. In order to load a single theme, use `layout=themename`
|
|
or `userlayout=true#<layout configuration>` as [Query parameter](URL_Parameters.md). Note that the shorter URLs
|
|
(e.g. `bookcases.html`, `aed.html`, ...) _don't_ exist on the development version.
|
|
|
|
The previous instructions were tested on 2023-03-09 on a Ubuntu 22.04 machine.
|
|
|
|
Development using Windows
|
|
-------------------------
|
|
|
|
You need at least 3GB RAM available to run MapComplete, but you'll preferably have 8GB of free RAM available.
|
|
|
|
For Windows you can use the WSL subsystem.
|
|
|
|
To use the WSL in Visual Studio Code:
|
|
|
|
1. Make sure you have installed the [Remote - WSL](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-wsl) extension and it's dependencies.
|
|
2. Open a remote WSL window using the button in the bottom left.
|
|
3. Make a fork and clone the repository.
|
|
4. Install `npm` using `sudo apt install npm`.
|
|
5. Run `npm run init` and generate some additional dependencies and generated files. Note that it'll install the
|
|
dependencies too
|
|
6. Run `npm run start` to host a local testversion at http://127.0.0.1:1234/index.html
|
|
7. By default, a landing page with available themes is served. In order to load a single theme, use `layout=themename`
|
|
or `userlayout=true#<layout configuration>` as [Query parameter](URL_Parameters.md). Note that the shorter URLs (
|
|
e.g. `bookcases.html`, `aed.html`, ...) _don't_ exist on the development version.
|
|
|
|
|
|
Developing a local theme
|
|
------------------------
|
|
|
|
To test theme changes you locally make, the easiest way is:
|
|
|
|
- `npm run strt` which will run the server
|
|
- After making changes to some layers and/or themes, run `npm run generate:layeroverview`. This will recompile the relevant themes.
|
|
- You might need to refresh (F5) your browser window
|
|
|
|
Note that `npm run start` is equivalent to `npm run generate:layeroverview && npm run strt`
|
|
|
|
Automatic deployment
|
|
--------------------
|
|
|
|
MapComplete is automatically deployed to various locations, depending on the branch you are working on.
|
|
|
|
The master branch is automatically deployed to https://mapcomplete.org/, and the development branch is available at
|
|
https://dev.mapcomplete.org/. Both are automatically deployed by a Forgejo action.
|
|
|
|
Other branches are automatically deployed to https://builds.mapcomplete.org/<branchname>.
|
|
|
|
Deploying a fork
|
|
----------------
|
|
|
|
A script creates a webpage for every theme automatically, with some customizations in order to:
|
|
|
|
- to have shorter URLs
|
|
- have individual social images
|
|
- have individual web manifests
|
|
|
|
This script can be invoked with `npm run prepare-deploy`
|
|
|
|
If you want to deploy your fork:
|
|
|
|
1. `npm run prepare-deploy`
|
|
2. `npm run build`
|
|
3. Copy the entire `dist` folder to where you host your website. Visiting `index.html` gives you the landing page,
|
|
visiting `yourwebsite/<theme>` should bring you to the appropriate theme.
|
|
|
|
### Getting your own API-keys
|
|
|
|
Some services are bound to `https://mapcomplete.org`. In `package.json/config`, search for "#fork" and read the instructions.
|
|
|
|
|
|
|
|
Weird errors
|
|
------------
|
|
|
|
Try removing `node_modules`, `package-lock.json` and `.cache`.
|
|
|
|
Misc setup
|
|
----------
|
|
|
|
~~The json-git-merger is used to quickly merge translation
|
|
files, [documentation here](https://github.com/jonatanpedersen/git-json-merge#single-project--directory).~~
|
|
This merge driver is broken and would sometimes drop new questions or duplicate them... Not a good idea!
|
|
|
|
Overview of package.json-scripts
|
|
--------------------------------
|
|
|
|
- `increase-memory`: give Node.js some more RAM since this is a big (and memory intensive) project to build and run
|
|
- `start`: start a development server
|
|
- `test`: run the unit tests
|
|
- `init`: generate and download various assets which are needed to compile
|
|
- `generate:editor-layer-index`: download the editor-layer-index-json from osmlab.github.io
|
|
- `generate:images`: compile the SVGs into an asset
|
|
- `generate:translations`: compile the translation file into a JavaScript file
|
|
- `generate:layouts`: use `index.html` as template to create all the theme index pages. You'll want to run `clean` when
|
|
done
|
|
- `generate:docs`: generate various documents, such as information about available metatags, information to put on
|
|
the [OSM wiki](https://wiki.openstreetmap.org/wiki/MapComplete), ...
|
|
- `generate:report`: download statistics from OsmCha, compile neat graphs
|
|
- `generate:cache:speelplekken`: create an offline copy of all the data required for one specific (paid for) theme
|
|
- `generate:layeroverview`: read all the theme and layer configurations, compiles them into a single JSON.
|
|
- `reset:layeroverview`: if something is wrong with the layer overview, create an empty one
|
|
- `generate:licenses`: compile all the license info of images into a single JSON
|
|
- `optimize:images`: attempt to make smaller PNGs - optional to run before a deployment
|
|
- `generate`: run all the necessary generate-scripts
|
|
- `build`: actually bundle all the files into a single `dist/` folder
|
|
- `prepare-deploy`: create the layouts
|
|
- `deploy:staging`,`deploy:pietervdvn`, `deploy:production`: deploy the latest code on various locations
|
|
- `lint`: get depressed by the amount of warnings
|
|
- `clean`: remove some generated files which are annoying in the repo
|