Set Up OSM Nominatim Geocoding Server on Ubuntu 20.04
In a previous tutorial, I explained the process of building your own OSM tile server on Ubuntu 20.04. This tutorial is going to show you how to set up Nominatim Geocoding server on Ubuntu 20.04. Nominatim provides search functionality for OpenStreetMap, so if a visitor enters an address in a search box, the latitude/longitude location for that address will be returned.
Note: If you are going to set up Nominatim for the entire planet, then you should spin up another server for Nominatim, because it will also require 64GB RAM and 1TB SSD.
Step 1: Build Nominatim From Source
Install dependency packages to build Nominatim.
sudo apt update sudo apt install build-essential cmake g++ libboost-dev libboost-system-dev libboost-filesystem-dev libexpat1-dev zlib1g-dev libbz2-dev libpq-dev libproj-dev apache2 php php-pgsql libapache2-mod-php php-intl php-cgi phpunit php-codesniffer python3-setuptools python3-dev python3-pip python3-psycopg2 python3-tidylib python3-behave python-pytest pylint git clang-tidy postgresql-server-dev-12
Create the nominatim user. (No need to create a password for this user.)
sudo useradd -d /srv/nominatim -s /bin/bash -m nominatim
Change to the /srv/nominatim/ directory.
cd /srv/nominatim/
Grant permissions to your own user account.
sudo apt install acl sudo setfacl -R -m u:username:rwx /srv/nominatim/
Download Nominatim from the official website.
wget https://nominatim.org/release/Nominatim-3.5.1.tar.bz2
Extract the tarball.
tar xvf Nominatim-3.5.1.tar.bz2
Create the build directory.
mkdir build
Change to this directory and configure the build environment.
cd build cmake /srv/nominatim/Nominatim-3.5.1
Compile the source code.
make
Step 2: Configure Nominatim
The default configuration file for Nominatim is /srv/nominatim/build/settings/settings.php. We can create a local.php file and add our modifications there.
sudo nano /srv/nominatim/build/settings/local.php
Add the following lines in the file.
<?php
@define('CONST_Website_BaseURL', '/nominatim/');
@define('CONST_Default_Lat', 55.0);
@define('CONST_Default_Lon', 1.0);
@define('CONST_Default_Zoom', 6);
@define('CONST_Map_Tile_URL', 'https://tile.linuxguru.com/osm/{z}/{x}/{y}.png');
The above configuration defines
- The path of the Nominatim instance relative to your tile server.
- Default latitude, longitude, and zoom level.
- URL of your OSM tile server. By default, Nominatim uses the public
https://tile.openstreetmap.orgtile server. Here I use my own tile server.
You can also take a look at the /srv/nominatim/build/settings/settings.php file and add your own customizations if the need arises. For example, if you are going to import a large dataset (Europe, North America, planet, etc.), it’s a good practice to enable flat node storage of node locations, so node coordinates will be stored in a simple file instead of the database, saving you import time and disk storage.
@define('CONST_Osm2pgsql_Flatnode_File', '/srv/nominatim/flatnode.file');
Save and close the file.
Step 3: Install and Configure PostgreSQL
Note: If OSM tile server and Nominatim are installed on the same server, then you can skip this step, because you have already done this when setting up the OSM tile server.
We will use PostgreSQL to store map data. PostGIS is a geospatial extension to PostgreSQL. Run the following commands to install them.
sudo apt install postgresql postgresql-contrib postgis postgresql-12-postgis-3
Then we need to tune PostgreSQL for maximal performance. Edit the main configuration file.
sudo nano /etc/postgresql/12/main/postgresql.conf
Finf the the following parameters in this file and use the following values.
shared_buffers = 15GB work_mem = 1GB maintenance_work_mem = 10GB effective_cache_size = 24GB synchronous_commit = off max_wal_size = 1GB checkpoint_timeout = 10min checkpoint_completion_target = 0.9 fsync = off full_page_writes = off
Save and close the file. Restart PostgreSQL for the changes to take effect.
sudo systemctl restart postgresql
Note that you should turn on fsync and full_page_write after importing the OSM database, or you risk corrupting the database.
By default, PostgreSQL would try to use huge pages in RAM. However, Linux by default does not allocate huge pages. Check the process ID of PostgreSQL.
sudo head -1 /var/lib/postgresql/12/main/postmaster.pid
Sample output:
7031
Then check the VmPeak value of this process ID.
grep ^VmPeak /proc/7031/status
Sample output:
VmPeak: 16282784 kB
This is the peak memory size that will be used by PostgreSQL. Now check the size of huge page in Linux.
cat /proc/meminfo | grep -i huge
Sample output:
AnonHugePages: 0 kB ShmemHugePages: 0 kB HugePages_Total: 0 HugePages_Free: 0 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB Hugetlb: 0 kB
We can calculate how many huge pages we need. Divide the VmPeak value by the size of huge page: 16282784 kB / 2048 kB = 7950. Edit /etc/sysctl.conf file.
sudo nano /etc/sysctl.conf
Add the following line to allocate 7950 huge pages.
vm.nr_hugepages = 7950
Save and close the file. Then apply the changes.
sudo sysctl -p
If you check the meminfo again,
cat /proc/meminfo | grep -i huge
We can see there are 7950 huge pages available.
AnonHugePages: 0 kB ShmemHugePages: 0 kB HugePages_Total: 7950 HugePages_Free: 7950 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB
Restart PostgreSQL to use huge pages.
sudo systemctl restart postgresql
Step 4: Import OSM Database
Download Wikipedia importance dump file, which will improve the quality of the Nomiatim search results.
cd /srv/nominatim/Nominatim-3.5.1/data wget https://www.nominatim.org/data/wikimedia-importance.sql.gz
Download US and UK postcodes data.
wget https://www.nominatim.org/data/us_postcode_data.sql.gz wget https://www.nominatim.org/data/gb_postcode_data.sql.gz
Download country code data file.
wget -O country_osm_grid.sql.gz https://www.nominatim.org/data/country_grid.sql.gz
Then you need to download an OSM file and import it to PostgreSQL. You can go to http://download.geofabrik.de to download the extract you need. You can also use the PBF file during the tile server setup process.
Create the www-data user in PostgreSQL, so the web server will have read-only access to the database.
sudo -u postgres createuser www-data
Grant permission to the postgres user.
sudo setfacl -R -m u:postgres:rwx /srv/nominatim/
Switch to the postgres user.
sudo -u postgres -i
And run the following command to import OSM extracts to PostgreSQL.
cd /srv/nominatim/build/ /srv/nominatim/build/utils/setup.php --osm-file /home/osm/great-britain-latest.osm.pbf --all 2>&1 | tee setup.log
After importing the database, the indexing process will begin. There are 30 ranks in total.
Once it’s finished, run the following command to verify.
/srv/nominatim/build/utils/check_import_finished.php
Exit out of the postgres user.
exit
Step 5: Set Up Apache
If Nominatim is installed on the OSM tile server, then edit the tile server configuration file.
sudo nano /etc/apache2/sites-enabled/tileserver_site-le-ssl.conf
Add the following lines between the VirtualHost tags.
<Directory "/srv/nominatim/build/website"> Options FollowSymLinks MultiViews AddType application/json .php DirectoryIndex search.php Require all granted </Directory> alias /nominatim /srv/nominatim/build/website
Save and close the file. Then reload Apache.
sudo systemctl reload apache2
If you are setting up Nominatim on a separate server, then you need to install Apache and PHP.
sudo apt install apache2 php7.4 libapache2-mod-php7.4 php-common php7.4-cli php7.4-common php7.4-json php7.4-opcache php7.4-readline
Create a virtual host for Nominatim.
sudo nano /etc/apache2/sites-enabled/nominatim.conf
Add the following lines in this file.
<VirtualHost *:80>
ServerName nominatim.example.com
DocumentRoot /srv/nominatim/build/website
<Directory "/srv/nominatim/build/website">
Options FollowSymLinks MultiViews
AddType application/json .php
DirectoryIndex search.php
Require all granted
</Directory>
alias /nominatim /srv/nominatim/build/website
ErrorLog ${APACHE_LOG_DIR}/nominatim_error.log
LogLevel warn
CustomLog ${APACHE_LOG_DIR}/nominatim_access.log combined
</VirtualHost>
Save and close the file. Then restart Apache.
sudo systemctl restart apache2
Now visit https://tile.yourdomain.com/nominatim. You will see your Nomiatim instance.
The CSS file is located at /srv/nominatim/build/website/css/search.css, if you want to customize the looking.
Step 6: Update Nominatim Database
To keep the Nominatim database up to date, we need to install Pyosmium. It’s available from the default software repository, but it’s recommended to install the latest version using pip3.
sudo pip3 install osmium
This will install a binary /usr/local/bin/pyosmium-get-changes. Edit Nominatim configuration file.
sudo nano /srv/nominatim/build/settings/local.php
Add the following line to specify the location of pyosmium-get-changes.
@define('CONST_Pyosmium_Binary', '/usr/local/bin/pyosmium-get-changes');
Next, we need to tell Nominatim where to download updates. By default, it’s configured to download updates from https://planet.openstreetmap.org/replication/minute. If you downloaded the OSM PBF file from geofabrik.de, then it’s better to also download updates from there.
To find the update URL for your own map, go to https://download.geofabrik.de/ and locate your region. Then find the URL for the .osc.gz file.
This URL is the update URL.
Add the following line in /srv/nominatim/build/settings/local.php file. You need to use your own update URL.
// base URL of the replication service
@define('CONST_Replication_Url', 'http://download.geofabrik.de/europe/great-britain-updates');
// How often upstream publishes diffs
@define('CONST_Replication_Update_Interval', '86400');
// How long to sleep if no update found yet
@define('CONST_Replication_Recheck_Interval', '900');
Save and close the file. Grant permissions to the postgres user.
sudo setfacl -R -m "u:postgres:rwx" /srv/nominatim/build/
Then swith to the postgres user.
sudo -u postgres -i
Initialize the update process.
/srv/nominatim/build/utils/update.php --init-updates
Update Nominatim database.
/srv/nominatim/build/utils/update.php --import-osmosis-all
Step 7: Set Up Cron Job For Automatic Update
Edit root user’s Crontab file.
sudo crontab -e
Add the following line in this file.
@daily sudo -u postgres /srv/nominatim/build/utils/update.php --import-osmosis-all
Save and close the file. If you don’t want to automatically update Nominatim database, simply remove the above line from your Crontab file.
How to Add Search Functionality to a Slippy Map
I assume your slippy map is displayed using the Leaflet JavaScript library. To add search functionality to your map, you need to use a Leaflet geocoding plugin. I will show you how to use Leaflet Control Geocoder. It’s actually very simple.
Suppose you used the following HTML code to display your slippy map.
<html>
<head>
<meta charset="UTF-8">
<title>My first osm</title>
<link rel="stylesheet" type="text/css" href="leaflet.css"/>
<script type="text/javascript" src="leaflet.js"></script>
<style>
#map{width:100%;height:100%}
</style>
</head>
<body>
<div id="map"></div>
<script>
var map = L.map('map').setView([54,1],6);
L.tileLayer('https://tile.yourdomain.com/osm/{z}/{x}/{y}.png',{maxZoom:19}).addTo(map);
</script>
</body>
</html>
Now you need to add the following two lines in the HTML header to use the Leaflet Control Geocoder plugin.
<link rel="stylesheet" href="https://unpkg.com/leaflet-control-geocoder/dist/Control.Geocoder.css" /> <script src="https://unpkg.com/leaflet-control-geocoder/dist/Control.Geocoder.js"></script>
Then add the following function to the <script>...</script> code so the search functionality will be added to your map.
L.Control.geocoder().addTo(map);
The final HTML code looks like this:
<html>
<head>
<meta charset="UTF-8">
<title>My first osm</title>
<link rel="stylesheet" type="text/css" href="leaflet.css"/>
<link rel="stylesheet" href="https://unpkg.com/leaflet-control-geocoder/dist/Control.Geocoder.css" />
<script type="text/javascript" src="leaflet.js"></script>
<script src="https://unpkg.com/leaflet-control-geocoder/dist/Control.Geocoder.js"></script>
<style>
#map{width:100%;height:100%}
</style>
</head>
<body>
<div id="map"></div>
<script>
var map = L.map('map').setView([54,1],6);
L.tileLayer('https://tile.yourdomain.com/osm/{z}/{x}/{y}.png',{maxZoom:19}).addTo(map);
L.Control.geocoder().addTo(map);
</script>
</body>
</html>
Save and close the file. Then reload the map in your web browser, you should see a search button on the upper-right corner.
By default, Leaflet Control Geocoder uses the public https://nominatim.openstreetmap.org geocoding service. To make it use your own Nominatim geocoding service, delete the following line.
L.Control.geocoder().addTo(map);
Add the following lines instead. Replace the URL with the URL of your Nominatim geocoding service. Note that you should not leave out the trailing slash.
var geocoder = L.Control.Geocoder.nominatim({serviceUrl:'https://tile.yourdomain.com/nominatim/'});
if (URLSearchParams && location.search) {
// parse /?geocoder=nominatim from URL
var params = new URLSearchParams(location.search);
var geocoderString = params.get('geocoder');
if (geocoderString && L.Control.Geocoder[geocoderString]) {
console.log('Using geocoder', geocoderString);
geocoder = L.Control.Geocoder[geocoderString]();
} else if (geocoderString) {
console.warn('Unsupported geocoder', geocoderString);
}
}
var control = L.Control.geocoder({
query: 'Type Address here',
placeholder: 'Search here...',
geocoder: geocoder,
position: 'topright'
}).addTo(map);
var marker;
setTimeout(function() {
control.setQuery('Type Address here');
}, 12000);
The default position is topright. You can change it to topleft if you like to.
You can also add the following code for reverse geocoding. When a visitor clicks on a point on the map, the name of that address will appear.
map.on('click', function(e) {
geocoder.reverse(e.latlng, map.options.crs.scale(map.getZoom()), function(results) {
var r = results[0];
if (r) {
if (marker) {
marker
.setLatLng(r.center)
.setPopupContent(r.html || r.name)
.openPopup();
} else {
marker = L.marker(r.center)
.bindPopup(r.name)
.addTo(map)
.openPopup();
}
}
});
});
Save and close the file. Then reload the map in your web browser.
Improve Accuracy of Reverse Search
There are two types of search in Nominatim:
- forward search, aka geocoding, returns latitude and longitude for an address
- reverse search, aka reverse geocoding, returns an address for latitude and longitude, i.e when a visitor clicks on a point in the map.
If you do a reverse search, the marker pin and pop-up are not in the immediate proximity of the position on the map that you clicked, you need to increase the zoom level. The map.getZoom() function will get the current map view, which is set with the setView() function like so
var map = L.map('map').setView([54,1],6);
The zoom level is set to 6, which will give poor accuracy for reverse search. We can hard code the zoom level for reverse search like so:
geocoder.reverse(e.latlng, map.options.crs.scale(21), function(results)
i.e change map.getZoom() to 21. The maximum zoom level for reverse search is 21. You can choose another zoom level to suit your needs.
Troubleshooting
If the search functionality on your map doesn’t work, you can check the console of your web browser to find out what went wrong. Some folks may see the 406 not acceptable or a CORS not allowed error. Make sure you have set the correct MIME type for .php in the Apache configuration file. Some folks may have the following line, which can cause the above errors.
AddType text/html .php
It should be
AddType application/json .php
After changing the MIME type. Reload Apache for the changes to take effect.
sudo systemctl reload apache2