Note

Containers have the same major.minor number, such as 2.2.x. Bug fixes are applied to the containers that changed. For example, psql-appmay be 2.2.1 while postgres is 2.2.0.

v2.2.1 Fixes:

  • Fixed issue with global IP rib not updating new/first seen prefixes. This caused an issue for new peer RIB dumps and when there are new prefixes added over time.
  • Added new sync_global_ip_rib() function. This function can be used to refresh/sync the entire global RIB. THIS CAN TAKE A WHILE TO RUN

Containers Updated

  • psql-app - Container will upgrade schema from 2.2.0

v2.2.0 Changes:

  • Major performance increase for IP rib inserts/updates. A major factor in disk IOPS is related to updating attributes and RIB entries at the same time. This in effect duplicates the mount of updates for every NLRI. This was not efficient and caused twice as much load on the disk when it didn't need to. The psql-app consumer now caches the state of a base_attr hash ID so that it can skip updating a hash_id that was already processed. In memory cache will auto expire hash_id's processed if there are no updates within 20 minutes.

  • Changed the openbmp/postgres container to use TimescaleDB-HA in order to support HA and add the much needed PostGIS and PGRouting. Together these extensions enable better analytics and the ability to calcuate shortest paths (e.g., Dijkstra) on link-state, BGP as-paths, best paths based on quality, etc.

  • Renamed psql-app container script /usr/local/openbmp/dbip-to-psql.py to /usr/local/openbmp/geo-csv-to-psql.py. Added MaxMind City CSV import to the import script. DB-IP or MaxMind CSV files can be imported now.

  • Add DB-IP cron job to automatically load [DB-IP]

  • Reworked update_global_ip() to improve performance on global rip updates. Global RIB update will now detect last time it was run in order to resume based on that instead of fixed interval. Max length of time to go back is 4 hours by default. The first param now identifies this value.

  • Removed prefix_bits column from ip_rib. It is no longer needed with GIN prefix indexing. This field was wasting space and slowing down inserts.

  • Removed partitioning on base_attrs and ip_rib as it did not improve performance.

  • Adjusted base_attrs auto vacuum settings. Auto vacuum is something that will need to be tuned in order to avoid it consuming too much disk IOPS/throughput.

  • Tuned psql-app default config to use 4 consumers, max writers per thread type defaults to 4.

  • Switched HASH indexes to BTREE. Hash indexes were performing worse even with equal join/queries.

  • Added prefix BTREE index for better equal ip_rib queries

  • Fixed psql-app container ENABLE_RPKI and ENABLE_IRR to only enable if the value is 1. Previously defining the variable with any value would enable the feature.

  • Added ENABLE_DBIP to enable DB-IP geo data import once a month. DB-IP will be imported only once a month and whenever the container is restarted.

  • Changed IRR import script to import only RADb considering RADb mirrors the others


Geo Coding Import

The 2.2.0 update includes DB-IP auto import script. The import will run only if ENABLE_DBIP=1. When enabled, it will run each time the container is restarted and thereafter once a month. In order to prevent mass downloads, the cron job will randomize the hour and minute that the script is run.

Upgrade

Danger

If you haven’t upgraded to 2.1.x yet, you will need to follow the Fresh Upgrade instructions.

Upgrade Options

It is possible to upgrade from 2.1.x, but if data can be purged, the recommendation is to perform a fresh DB upgrade by following the Fresh Upgrade instructions.

Upgrade from 2.2.0

  1. Update docker-compose by either downloading the latest or edit it manually. If you edit it manually, all you have to do is update the version of the psql-app to image: openbmp/psql-app:2.2.1

  2. Run docker compose up again to recreate the container. The container will upgrade the schema.

OBMP_DATA_ROOT=${OBMP_DATA_ROOT} docker-compose -p obmp up -d

The global IP RIB could be missing prefixes when initial RIB dumps are out of sync with the global rib cron job. The sync_global_ip_rib() function can be used to sync the global rib. The function can take a while to run and will cause a lot of extra disk IOPS. It is recommended to run it after initial RIB dumps.

Run select sync_global_ip_rib(); to synchronize the global IP RIB table.

Upgrading from 2.1.x

Warning

Only supported for 2.1.0 or greater

  1. Set the data root
      export OBMP_DATA_ROOT=/var/openbmp
    
  2. Update postgres file owner/uid timescaledb-ha sets the postgres user to use uid 1000, which is different than the previous container. You MUST fix the permissions before starting the container. Run the below to fix permissions.
sudo chown -R 1000 ${OBMP_DATA_ROOT}/postgres/
sudo chown -R 1000 ${OBMP_DATA_ROOT}/postgres/data
sudo chown -R 1000 ${OBMP_DATA_ROOT}/postgres/ts
  1. update postgres.conf The new container has some minor changes. Run the below to update the existing configuration.
export PGDATA=/var/lib/postgresql/data

openssl req -x509 -newkey rsa:4096 -nodes -subj "/C=US/ST=CA/L=Seattle/O=OpenBMP/CN=localhost"  \
        -keyout ${OBMP_DATA_ROOT}/postgres/data/psql_server.key -out ${OBMP_DATA_ROOT}/postgres/data/psql_server.crt -days 2048 \

sed -i -e "s/^\#*ssl_cert_file.*=.*/ssl_cert_file =  '${PGDATA//\//\\\/}\/psql_server.crt'/" ${OBMP_DATA_ROOT}/postgres/data/postgresql.conf
sed -i -e "s/^\#*ssl_key_file.*=.*/ssl_key_file =  '${PGDATA//\//\\\/}\/psql_server.key'/" ${OBMP_DATA_ROOT}/postgres/data/postgresql.conf
sed -i -e "s/^shared_preload_libraries.*/shared_preload_libraries = 'timescaledb,pg_cron'/g" ${OBMP_DATA_ROOT}/postgres/data/postgresql.conf
  1. Upgrade Grafana by copying over the latest provisioning data

     git clone https://github.com/OpenBMP/obmp-grafana.git
     # OR git pull
    
     cp -r obmp-grafana/dashboards obmp-grafana/provisioning ${OBMP_DATA_ROOT}/grafana/
     sudo chmod o+rx -R ${OBMP_DATA_ROOT}/grafana/
       
     docker restart obmp-grafana
    

    Make sure the files copied are owned by the container user. If not, provisioning will not load.

  2. Upgrade docker compose file to include the latest versions

     # Get latest compose file  - Your previous file will be saved as docker-compose.yml.<N>
     wget --backups=3 https://raw.githubusercontent.com/OpenBMP/obmp-docker/main/docker-compose.yml  
    
  3. Update the docker-compose.yml file variables and volumes based on your previous compose file. You can use diff to see the differences that need to be merged/updated. diff -u docker-compose.yml.1 docker-compose.yml

  4. Run docker compose to recreate the containers

     OBMP_DATA_ROOT=${OBMP_DATA_ROOT} docker-compose -p obmp up -d
    
  5. Apply postgres run-time changes Run the below after postgres is up and running/accepting connections. If you get

    FATAL:  the database system is starting up
    

try again later.

docker exec -it obmp-psql-app bash

# Run the below once in the container
source /usr/local/openbmp/pg_profile

psql -U $POSTGRES_USER -c "CREATE EXTENSION IF NOT EXISTS postgis CASCADE;" $POSTGRES_DB
psql -U $POSTGRES_USER -c "CREATE EXTENSION IF NOT EXISTS pgrouting CASCADE;" $POSTGRES_DB
psql -U $POSTGRES_USER -c "ALTER EXTENSION timescaledb UPDATE;" $POSTGRES_DB
psql -U $POSTGRES_USER -c "ALTER EXTENSION pg_cron UPDATE;" $POSTGRES_DB