Logo

A powerful, easily deployable network traffic analysis tool suite for network security monitoring

Quick Start

Documentation

Components

Supported Protocols

Configuring

Arkime

Dashboards

Hedgehog Linux

Contribution Guide

Configure authentication

Malcolm requires authentication to access the user interface. Nginx can authenticate users with either local TLS-encrypted HTTP basic authentication or a remote Lightweight Directory Access Protocol (LDAP) authentication server.

With the local basic authentication method, user accounts are managed by Malcolm and can be created, modified, and deleted using a user management web interface. This method is suitable in instances where accounts and credentials do not need to be synced across many Malcolm installations.

LDAP authentication is managed on a remote directory service, such as a Microsoft Active Directory Domain Services or OpenLDAP.

Kecloak authentication can either be managed by Malcolm’s embedded Keycloak instance, or it can be managed by a remote Keycloak instance.

Malcolm’s authentication method is defined in the auth-common.env configuration file file with the NGINX_AUTH_MODE environment variable: valid values are basic (or true for legacy compatibility), to use TLS-encrypted HTTP basic authentication (default); ldap (or false for legacy compatibility) to use Lightweight Directory Access Protocol (LDAP) authentication; keycloak to use authentication managed by Malcolm’s embedded Keycloak instance; keycloak_remote to use authentication managed by a remote Keycloak instance; or, no_authentication to disable authentication.

Whichever method is chosen, users must run ./scripts/auth_setup before starting Malcolm for the first time in order to:

See the the end-to-end Malcolm installation example for a step-by-step guide to the auth_setup script.

Local account management

auth_setup is used to define the username and password for the administrator account. Once Malcolm is running, the administrator account can be used to manage other user accounts via a Malcolm User Management page at https://localhost/auth if connecting locally.

Malcolm user accounts can be used to access the interfaces of all of its components, including Arkime. Arkime uses its own internal database of user accounts, so when a Malcolm user account logs in to Arkime for the first time, Malcolm creates a corresponding Arkime user account automatically. This being the case, it is not recommended to use the Arkime Users settings page or change the password via the Password form under the Arkime Settings page, as those settings would not be consistently used across Malcolm.

Users may change their passwords via the Malcolm User Management page by clicking User Self Service.

Lightweight Directory Access Protocol (LDAP) authentication

The nginx-auth-ldap module serves as the interface between Malcolm’s Nginx web server and a remote LDAP server. When you run auth_setup for the first time, a sample LDAP configuration file is created at nginx/nginx_ldap.conf.

# This is a sample configuration for the ldap_server section of nginx.conf.
# Yours will vary depending on how your Active Directory/LDAP server is configured.
# See https://github.com/kvspb/nginx-auth-ldap#available-config-parameters for options.

ldap_server ad_server {
  url "ldap://ds.example.com:3268/DC=ds,DC=example,DC=com?sAMAccountName?sub?(objectClass=person)";

  binddn "bind_dn";
  binddn_passwd "bind_dn_password";

  group_attribute member;
  group_attribute_is_dn on;
  require group "CN=Malcolm,CN=Users,DC=ds,DC=example,DC=com";
  require valid_user;
  satisfy all;
}

auth_ldap_cache_enabled on;
auth_ldap_cache_expiration_time 10000;
auth_ldap_cache_size 1000;

This file is mounted into the nginx container when Malcolm is started to provide connection information for the LDAP server.

The contents of nginx_ldap.conf will vary depending on how the LDAP server is configured. Some of the avaiable parameters in that file include:

Before starting Malcolm, edit nginx/nginx_ldap.conf according to the specifics of your LDAP server and directory tree structure. Using a LDAP search tool such as ldapsearch in Linux or dsquery in Windows may be of help as you formulate the configuration. Your changes should be made within the curly braces of the ldap_server ad_server { … } section. You can troubleshoot configuration file syntax errors and LDAP connection or credentials issues by running ./scripts/logs (or docker compose logs nginx) and examining the output of the nginx container.

Note that as an alternative to the nginx LDAP integration described in this section, Keycloak can also be configured to federate one or more LDAP servers.

The Malcolm User Management page described above is not available when using LDAP authentication.

LDAP connection security

Authentication over LDAP can be done using one of three methods, two of which offer data confidentiality protection:

In addition to the NGINX_AUTH_MODE environment variable being set to ldap in the auth-common.env configuration file file, the NGINX_LDAP_TLS_STUNNEL and NGINX_LDAP_TLS_STUNNEL environment variables are used in conjunction with the values in nginx/nginx_ldap.conf to define the LDAP connection security level. Use the following combinations of values to achieve the connection security methods above, respectively:

For encrypted connections (whether using StartTLS or LDAPS), Malcolm will require and verify certificates when one or more trusted CA certificate files are placed in the nginx/ca-trust/ directory. Otherwise, any certificate presented by the domain server will be accepted.

Keycloak

Malcolm can utilize Keycloak, an identity and access management (IAM) tool, to provide a more robust authentication and authorization experience, including single sign-on (SSO) functionality.

The guides in this section cover configuring Malcolm to use Keycloak for authentication, but do not go into the details of the many capabilities Keycloak provides, including identity providers, SSO protocols, federate one or more LDAP or Kerberos servers, and more. Refer to the Keycloak Server Administration Guide for information on these and other topics.

Known Limitation with Hedgehog Linux

Due to incompatibilities between Arkime capture on Hedgehog Linux, Malcolm’s nginx reverse proxy, and Keycloak, if Malcolm is using a local OpenSearch instance exposed to external hosts, that endpoint (typically port 9200/tcp) must be authenticated using HTTP basic authentication even when Keycloak is selected as Malcolm’s main authentication method.

When configuring forwarding for arkime-capture on Hedgehog Linux, the credentials entered should be those described above in the section on local account management, not Keycloak credentials. In this configuration, the administrator account can be used to manage other user accounts via a Malcolm User Management page as described above. These credentials are only valid for Malcolm’s local OpenSearch API endpoint.

This limitation does not apply when Malcolm is configured to use a remote OpenSearch or Elasticsearch instance.

The Malcolm development team is exploring solutions to this issue.

Using a remote Keycloak instance

This section outlines how to use an existing Keycloak instance managed separately from Malcolm. See Using the embedded Keycloak instance for how to configure Malcolm to use its own embedded Keycloak instance. In addition to the NGINX_AUTH_MODE variable in auth-common.env described above, Malcolm uses environment variables in keycloak.env to configure its use of Keycloak.

These first two steps are performed during configuration:

  1. Prior to starting Malcolm, during configuration, the user is asked Which IP version does the network support? (IPv4, IPv6, or both)? This question is used to configure the resolver directive for Malcolm’s nginx reverse proxy. Note that this selection does not affect Malcolm’s ability to capture or inspect IPv4/IPv6 traffic: it is only used if and when nginx itself needs to resolve hostnames in the network in which Malcolm resides (for example, to resolve the Keycloak URL if Malcolm is configured to be accessible via a host name rather than an IP address). See related settings for nginx in the nginx.env configuration file.

nginx IPv4/IPv6 responder configuration

  1. A few steps later, the user is prompted to Select authentication method. Selecting keycloak_remote will use a remote Keycloak instance.

Select authentication method

The next steps happen in the context of auth_setup.

  1. Run ./scripts/auth_setup and select Configure Keycloak.

Configure Keycloak in auth_setup

  1. Specify the name of the Keycloak realm to use.

Specify the Keycloak realm

  1. Provide the relative path which is the Malcolm URI to which Keycloak will redirect users after a successful authentication. The default is /index.html, which will redirect users to the Malcolm landing page.

Specify the redirect URI

  1. Provide the Keycloak URL, or the URL to which Malcolm should direct authentication requests for Keycloak. This should be the URL of the existing keycloak provider.

Specify the Keycloak URL

  1. Enter the client ID and client secret that should be used by Malcolm to authenticate.

Client ID in auth_setup

Client secret in auth_setup

  1. Enter group membership restrictions and user realm role restrictions to limit the set of users permitted to authenticate to Malcolm to those that meeting those requirements. Blank values mean that no restriction of that type will be enforced. Multiple values may be specified as a comma-separated list. See Requiring user groups and realm roles below for more information.

Required user groups

Required user roles

  1. The values for temporary bootstrap username and password may be left blank, as they are irrelevant when using a remote Keycloak instance.

With the remote Keycloak configuration complete, start Malcolm.

  1. Run ./scripts/start and wait a few minutes for Malcolm to start.

  2. Open a web browser and navigate to the hostname or IP address where Malcolm can be reached (e.g., https://malcolm.example.com), which should redirect to the Sign in to your account Keycloak page provided by the remote Keycloak instance. Enter valid credentials, then click Sign in.

Malcolm authentication via Keycloak

  1. After successfully signing in, Keycloak redirects back to the Malcolm landing page.

Redirected to the Malcolm landing page

Using the embedded Keycloak instance

This section outlines how to configure an embedded Keycloak instance managed by Malcolm. See Using a remote Keycloak instance for how to configure Malcolm to use a remote Keycloak instance already extant in the network infrastructure. In addition to the NGINX_AUTH_MODE variable in auth-common.env described above, Malcolm uses environment variables in keycloak.env to configure its use of Keycloak.

As Keycloak is a much more full-featured authentication provider than the local basic authentication method, configuring Malcolm’s embedded Keycloak instance is a bit more involved. The following steps outline how this is done.

These first two steps are performed during configuration:

  1. Prior to starting Malcolm, during configuration, the user is asked Which IP version does the network support? (IPv4, IPv6, or both)? This question is used to configure the resolver directive for Malcolm’s nginx reverse proxy. Note that this selection does not affect Malcolm’s ability to capture or inspect IPv4/IPv6 traffic: it is only used if and when nginx itself needs to resolve hostnames in the network in which Malcolm resides (for example, to resolve the Keycloak URL if Malcolm is configured to be accessible via a host name rather than an IP address). See related settings for nginx in the nginx.env configuration file.

nginx IPv4/IPv6 responder configuration

  1. A few steps later, the user is prompted to Select authentication method. Selecting keycloak will use Malcolm’s embedded Keycloak instance.

Select authentication method

The next steps happen in the context of ./scripts/auth_setup.

  1. If the internal passwords for Keycloak’s PostgreSQL database were not generated during the initial run of auth_setup, this should be done now. If this step has already been done, the user will be warned about regenerating passwords over the top of those that already exist; this is only a concern if the PostgreSQL database already contains data.

Generating Keycloak's internal PostgreSQL database passwords

  1. Next, running auth_setup again, select Configure Keycloak.

Configure Keycloak in auth_setup

  1. Specify the name of the Keycloak realm to use. The default, as defined by Keycloak itself, is master.

Specify the Keycloak realm

  1. Provide the relative path which is the Malcolm URI to which Keycloak will redirect users after a successful authentication. The default is /index.html, which will redirect users to the Malcolm landing page.

Specify the redirect URI

  1. Provide the Keycloak URL, or the URL to which Malcolm should direct authentication requests for Keycloak. Since this guide is to configure the embedded Keycloak instance, this host portion of the URL should be the hostname or IP address at which Malcolm is available, followed by /keycloak.

Specify the Keycloak URL

  1. When configuring Keycloak for the first time, a Keycloak client ID and secret have not yet been configured, so the next two values should be left blank for now.

An empty value for Keycloak client ID

An empty value for Keycloak client ID

  1. Enter group membership restrictions and user realm role restrictions to limit the set of users permitted to authenticate to Malcolm to those that meeting those requirements. Blank values mean that no restriction of that type will be enforced. Multiple values may be specified as a comma-separated list. See Requiring user groups and realm roles below for more information.

Required user groups

Required user roles

  1. When Keycloak starts up for the first time and no users exist, it will bootstrap a temporary admin account with the username and password provided to the next two questions. These values will only be used once, after which it is recommended to delete this temporary account in the Keycloak user management interface.

Bootstrap admin username

Bootstrap admin password

With the initial configuration complete, start Malcolm.

  1. Run ./scripts/start and wait a few minutes for Malcolm to start.

  2. Open a web browser and enter the Keycloak URL specified above (i.e., the hostname or IP address at which Malcolm is available followed by /keycloak), then log in with the bootstrap username and password. Navigating to any other Malcolm URI will result in a “client not found” error until the Keycloak configuration is finished.

Bootstrap admin login

  1. Once logged in, Keycloak warns “You are logged in as a temporary admin user. To harden security, create a permanent admin account and delete the temporary one.” Navigate to the Users page under Manage on the navigation sidebar, then click Add user.

Add a new Keycloak admin user

  1. The Create user page requires that a username for the new admin user be provided. Other details such as name and email may also be supplied. Click Create after providing these details.

Provide Keycloak admin user details

  1. Once the new user has been created, navigate to the Credentials tab in the user details screen for the new user, and click Set password. Set a password for the newly created user.

Provide Keycloak admin user details

  1. Navigate to the Role mapping tab and click the Assign role button. Select Filter by realm roles, then check the box next to the admin role, then click Assign, after which the User role mapping successfully updated confirmation will appear.

Assign admin user role

  1. In the upper-right hand corner of the page, click on the username for the temporary admin account and select Sign out.

  2. Sign in again with the username and password for the new admin user created in the previous steps.

  3. Once again, navigate to the Users page under Manage on the navigation sidebar. Select the temporary bootstrap admin user account and click Delete user, then confirm by clicking Delete.

Deleting the temporary bootstrap user

  1. Next, a Keycloak client must be created which will be used by Malcolm’s nginx reverse proxy to handle user authentication. Navigate to the Clients page under Manage on the navigation sidebar and click Create client.

Creating a new client

  1. Provide a Client ID (e.g., malcolm or myclient) to identify the new client, then click Next.

Client ID

  1. Enable Client authentication, then click Next.

Client authentication

  1. Under Login settings provide the following values, replacing malcolm.example.com with the hostname or IP address of the Malcolm instance. Then click Save.
    • Root URL: https://malcolm.example.com
    • Home URL: https://malcolm.example.com
    • Valid redirect URIs: /*
    • Valid post logout redirect URIs: https://malcolm.example.com/keycloak
    • Web origins: https://malcolm.example.com

Client login settings

  1. From the Client details page for the new client, click the Credentials tab. Copy the Client Secret to the clipboard or make note of it so that it can be used in the following step.

Client secret

  1. Return to a command prompt in the Malcolm installation directory and re-run ./scripts/auth_setup, then select Configure Keycloak.

  2. Leave the realm, redirect URI, and keycloak URL unchanged. Enter the client ID and client secret when prompted. The values for temporary bootstrap username and password can also be left unchanged, as they are irrelevant and will not be used again.

Client ID in auth_setup

Client secret in auth_setup

  1. The nginx-proxy container needs to be updated with these new values. Restart the nginx-proxy container by running ./scripts/restart -s nginx-proxy (pressing CTRL+C to interrupt the debug log output once the service has restarted), or restart Malcolm entirely.
/scripts/restart -s nginx-proxy
Stopped Malcolm's ['nginx-proxy'] services

Removed Malcolm's ['nginx-proxy'] services

NAME                    IMAGE                                          COMMAND                  SERVICE       CREATED        STATUS                                     PORTS
malcolm-nginx-proxy-1   ghcr.io/idaholab/malcolm/nginx-proxy:25.03.1   "/sbin/tini -- /usr/…"   nginx-proxy   1 second ago   Up Less than a second (health: starting)   

nginx-proxy-1  | root
nginx-proxy-1  | uid=0(root) gid=0(root) groups=0(root),0(root),1(bin),2(daemon),3(sys),4(adm),6(disk),10(wheel),11(floppy),20(dialout),26(tape),27(video)
nginx-proxy-1  | 2025-03-11 17:29:02,866 INFO Set uid to user 0 succeeded
nginx-proxy-1  | 2025-03-11 17:29:02,873 INFO RPC interface 'supervisor' initialized
nginx-proxy-1  | 2025-03-11 17:29:02,873 CRIT Server 'unix_http_server' running without any HTTP authentication checking
nginx-proxy-1  | 2025-03-11 17:29:02,873 INFO supervisord started with pid 37
nginx-proxy-1  | 2025-03-11 17:29:03,876 INFO spawned: 'logaccess' with pid 111
nginx-proxy-1  | 2025-03-11 17:29:03,878 INFO spawned: 'logerrors' with pid 112
nginx-proxy-1  | 2025-03-11 17:29:03,879 INFO spawned: 'nginx' with pid 113
nginx-proxy-1  | 2025-03-11 17:29:14,283 INFO success: logaccess entered RUNNING state, process has stayed up for > than 10 seconds (startsecs)
nginx-proxy-1  | 2025-03-11 17:29:14,283 INFO success: logerrors entered RUNNING state, process has stayed up for > than 10 seconds (startsecs)
nginx-proxy-1  | 2025-03-11 17:29:14,283 INFO success: nginx entered RUNNING state, process has stayed up for > than 10 seconds (startsecs)
^C
  1. Open a web browser and navigate to the hostname or IP address where Malcolm can be reached (e.g., https://malcolm.example.com), which should redirect to the Sign in to your account Keycloak page. Enter the username and password for the user created above, then click Sign in.

Malcolm authentication via Keycloak

  1. After successfully signing in, Keycloak redirects back to the Malcolm landing page. Additional users can now be added by navigating to Keycloak (by clicking the 👤 Keycloak Authentication link) and repeating steps 11 and 12 for each new user to be added.

Redirected to the Malcolm landing page

Requiring user groups and realm roles

Full role-based fine-grained access controls will be implemented in a future release of Malcolm. In the meantime, Malcolm can be configured to require Keycloak-authenticated users to belong to groups and assigned realm roles, respectively. The values for these groups and/or roles are specified when running ./scripts/auth_setup under Configure Keycloak and are saved as NGINX_REQUIRE_GROUP and NGINX_REQUIRE_ROLE in the auth-common.env configuration file. An empty value for either of these settings means no restriction of that type is applied. Multiple values may be specified with a comma-separated list. These requirements are cumulative: users must match all of the items specified. Note that LDAP authentication can also require group membership, but that is specified in nginx_ldap.conf by setting require group rather than in auth-common.env.

For a discussion of roles vs. groups, see Assigning permissions using roles and groups in the Keycloak Server Administration Guide.

Groups can be managed in Keycloak by selecting the appropriate realm from the drop down at the top of the navigation panel and selecting Groups under Manage.

Groups

Users can be joined to groups by clicking on a username on the Keycloak Users page, selecting the Groups tab, then clicking Join Group.

User group membership

Realm roles can be managed in Keycloak by selecting the appropriate realm from the drop down at the top of the navigation panel and selecting Realm roles under Manage.

Realm roles

Users can be assigned realm roles by clicking on a username on the Keycloak Users page, selecting the Role mapping tab, then clicking Assign role. Select Filter by realm roles, then check the box next to the desired role(s), then click Assign, after which the User role mapping successfully updated confirmation will appear.

User realm role assignment

Keycloak does not include group or realm role information in authentication tokens by default; clients must be configured to include this information in order for users to log in to Malcolm with group and/or role restrictions set. This can be done by navigating to the Keycloak Clients page, selecting the desired client, then clicking the Client scopes tab. Click on the name of the assigned client scope beginning with the client ID and ending in -dedicated, which will also have a description of “Dedicated scope and mappers for this client.” Once on this Clients > Client details > Dedicated scopes screen, click the down arrow on the Add mapper button and select By configuration.

To include group information in the Keycloak token for this client, select Group Membership from the Configure a new mapper list. The important information to provide for this Group Membership mapper before clicking Save is:

Client group membership mapper

To include user realm role information in the Keycloak token for this client, once again click the down arrow on the Add mapper button and select Byte configuration. Select User Realm Role from the Configure a new mapper list. The important information to provide for this User Realm Role mapper before clicking Save is:

Client realm role mapper

Once the mapper(s) have been created, the list of mappers on the Clients > Client details > Dedicated scopes page will look something like this:

Client dedicated scopes

TLS certificates

When users set up authentication for Malcolm a set of unique self-signed TLS certificates are created which are used to secure the connection between clients (e.g., your web browser) and Malcolm’s browser-based interface. This is adequate for most Malcolm instances as they are often run locally or on internal networks, although your browser will most likely require users to add a security exception for the certificate when first connecting to Malcolm.

Another option is for users to generate their own certificates (or have them issued directly) and have them placed in the nginx/certs/ directory. The certificate and key file should be named cert.pem and key.pem, respectively.

A third possibility is to use a third-party reverse proxy (e.g., Traefik or Caddy) to handle the issuance of the certificates and to broker the connections between clients and Malcolm. Reverse proxies such as these often implement the ACME protocol for domain name authentication and can be used to request certificates from certificate authorities such as Let’s Encrypt. In this configuration, the reverse proxy will be encrypting the connections instead of Malcolm, so users will need to set the NGINX_SSL environment variable to false in nginx.env (or answer no to the “Require encrypted HTTPS connections?” question posed by ./scripts/configure). If you are setting NGINX_SSL to false, make sure user must understand precisely what they are doing, ensuring that external connections cannot reach ports over which Malcolm will be communicating without encryption, including verifying local firewall configuration.

Command-line arguments

The ./scripts/auth_setup script can also be run noninteractively which can be useful for scripting Malcolm setup. This behavior can be selected by supplying the --auth-noninteractive option on the command line. Running with the --help option will list the arguments accepted by the script:

$ ./scripts/auth_setup --help
usage: auth_setup <arguments>

Malcolm control script

options:
  -v [DEBUG], --verbose [DEBUG]
                        Verbose output
  -f <string>, --file <string>
                        Docker compose or kubeconfig YML file
  -e <string>, --environment-dir <string>
                        Directory containing Malcolm's .env files

…

Authentication Setup:
  --auth [CMDAUTHSETUP]
                        Configure Malcolm authentication
  --auth-noninteractive [CMDAUTHSETUPNONINTERACTIVE]
                        Configure Malcolm authentication (noninteractive using arguments provided)
  --auth-admin-username <string>
                        Administrator username (for --auth-noninteractive)
  --auth-admin-password-openssl <string>
                        Administrator password hash from "openssl -passwd -1" (for --auth-noninteractive)
  --auth-admin-password-htpasswd <string>
                        Administrator password hash from "htpasswd -n -B username | cut -d: -f2" (for --auth-noninteractive)
  --auth-arkime-password <string>
                        Password hash secret for Arkime viewer cluster (for --auth-noninteractive)
  --auth-generate-webcerts [AUTHGENWEBCERTS]
                        (Re)generate self-signed certificates for HTTPS access (for --auth-noninteractive)
  --auth-generate-fwcerts [AUTHGENFWCERTS]
                        (Re)generate self-signed certificates for a remote log forwarder
  --auth-generate-netbox-passwords [AUTHGENNETBOXPASSWORDS]
                        (Re)generate internal passwords for NetBox
  --auth-generate-redis-password [AUTHGENREDISPASSWORD]
                        (Re)generate internal passwords for Redis
  --auth-generate-postgres-password [AUTHGENPOSTGRESPASSWORD]
                        (Re)generate internal superuser passwords for PostgreSQL
  --auth-generate-keycloak-db-password [AUTHGENKEYCLOAKDBPASSWORD]
                        (Re)generate internal passwords for Keycloak's PostgreSQL database
  --auth-keycloak-realm <string>
                        Keycloak realm
  --auth-keycloak-redirect-uri <string>
                        Keycloak redirect URI
  --auth-keycloak-url <string>
                        Keycloak URL
  --auth-keycloak-client-id <string>
                        Keycloak client ID
  --auth-keycloak-client-secret <string>
                        Keycloak client secret
  --auth-keycloak-bootstrap-user <string>
                        Temporary Keycloak admin bootstrap username
  --auth-keycloak-bootstrap-password <string>
                        Temporary Keycloak admin bootstrap password
  --auth-require-group <string>
                        Required group(s) to which users must belong (--auth-method is keycloak|keycloak_remote)
  --auth-require-role <string>
                        Required role(s) which users must be assigned (--auth-method is keycloak|keycloak_remote)
…

In order to avoid specifying passwords as command line arguments, when setting the administrator password noninteractively the user must instead provide hashes generated by openssl -passwd -1 and htpasswd -n -B username.

Log Out of Malcolm

When using Keycloak, Malcolm users can log out of Malcolm by navigating to the Keycloak user interface, opening the drop-down with their name or username in the upper-right corner of the page, and selecting Sign out.

For the other authentication methods which use HTTP basic authentication, users should exit their browser to log out of Malcom. On the next browser restart, Malcolm will prompt the user for credentials. Another way to log out of Malcolm is for a user to manually clear their browser’s active sessions.