A powerful, easily deployable network traffic analysis tool suite for network security monitoring
At this time there is not an “official” version upgrade procedure for Malcolm, as such procedures vary from platform to platform. However, the process is fairly simple and can be done by following these steps:
Users may wish to apply official updates for the underlying system’s software packages before proceededing. Consult operating system documentation for instructions on applying system updates.
If upgrading a Malcolm instance installed from Malcolm installation ISO, follow Scenario 2 below. Due to the Malcolm base operating system’s hardened configuration, users updating the underlying system must temporarily set the umask value to Debian default (umask 0022
in the root shell in which updates are being performed) instead of the more restrictive Malcolm default. This will allow updates to be applied with the correct permissions.
Here are the basic steps to perform an upgrade if Malcolm was checked with a git clone
command:
./scripts/stop
docker-compose.yml
and other files
git stash save "pre-upgrade Malcolm configuration changes"
./config/
directorygit pull --rebase
docker compose --profile malcolm pull
git stash pop
Merge conflict
messages appear, resolve the conflicts with a text editor./scripts/configure
as described in Malcolm Configuration in case there are any new configuration parameters for Malcolm that need to be set up./scripts/start
If Malcolm was installed from pre-packaged installation files, here are the basic steps to perform an upgrade:
./scripts/stop
malcolm_YYYYMMDD_HHNNSS_xxxxxxx.tar.gz
as an example, the file and/or directory names will be different depending on the release)
tar xf malcolm_YYYYMMDD_HHNNSS_xxxxxxx.tar.gz
mkdir -p ./upgrade_backup_$(date +%Y-%m-%d)
cp -r filebeat/ htadmin/ logstash/ nginx/ config/ docker-compose*.yml ./scripts ./README.md ./upgrade_backup_$(date +%Y-%m-%d)/
rm -rf ./scripts ./README.md
cp -r ./malcolm_YYYYMMDD_HHNNSS_xxxxxxx/scripts ./malcolm_YYYYMMDD_HHNNSS_xxxxxxx/README.md ./
docker-compose*.yml
file with new versions
cp ./malcolm_YYYYMMDD_HHNNSS_xxxxxxx/docker-compose*.yml ./
./scripts/configure
as described in Malcolm Configuration
diff
, meld
, Beyond Compare
, etc.), compare docker-compose.yml
and the docker-compose.yml
files backed up in Step 3, and manually migrate over any customizations in file.env
file Malcolm’s ./config/
directory with its corresponding .env.example
file. the author uses this command which uses difftastic, bat, unbuffer, and cmp.
for FILE in *.env; do \
cmp -s ../config/"$FILE.example" "$FILE" || \
unbuffer difft --display side-by-side-show-both \
--tab-width 4 --strip-cr \
--syntax-highlight on --ignore-comments \
../config/"$FILE.example" "$FILE"; \
done | bat --color=always
docker compose --profile malcolm pull
to pull them from GitHub or docker compose load -i malcolm_YYYYMMDD_HHNNSS_xxxxxxx_images.tar.xz
if an offline tarball of the Malcolm images is available./scripts/start
Technically minded users may wish to follow the debug output provided by ./scripts/start
(use ./scripts/logs
to re-open the log stream after it’s been closed), although there is a lot there and it may be hard to distinguish whether or not something is okay.
Running docker compose ps -a
should provide a good indication that all Malcolm’s containers started up and, in some cases, may be able to indicate if the containers are “healthy” or not.
After upgrading following one of the previous outlines, give Malcolm several minutes to get started. Once things are up and running, open one of Malcolm’s web interfaces to verify that things are working.
Once the upgraded instance Malcolm has started up, users will want to import the new dashboards and visualizations for OpenSearch Dashboards. Users can signal Malcolm to load the new visualizations by opening OpenSearch Dashboards, clicking Management → Index Patterns, then selecting the arkime_sessions3-*
index pattern and clicking the delete 🗑 button near the upper-right of the window. Confirm the Delete index pattern? prompt by clicking Delete. Close the OpenSearch Dashboards browser window. After a few minutes the missing index pattern will be detected and OpenSearch Dashboards will be signalled to load its new dashboards and visualizations.
The Malcolm project uses semantic versioning when choosing version numbers. When moving between major releases (e.g., from v4.0.1 to v5.0.0), users are likely to find there are enough major backwards compatibility-breaking changes that upgrading may not be worth the time and trouble. A fresh install is strongly recommended between major releases.