A powerful, easily deployable network traffic analysis tool suite for network security monitoring
Malcolm requires authentication to access the user interface. Nginx can authenticate users with either local TLS-encrypted HTTP basic authentication or a remote Lightweight Directory Access Protocol (LDAP) authentication server.
With the local basic authentication method, user accounts are managed by Malcolm and can be created, modified, and deleted using a user management web interface. This method is suitable in instances where accounts and credentials do not need to be synced across many Malcolm installations.
LDAP authentication is managed on a remote directory service, such as a Microsoft Active Directory Domain Services or OpenLDAP.
Kecloak authentication can either be managed by Malcolm’s embedded Keycloak instance, or it can be managed by a remote Keycloak instance.
Malcolm’s authentication method is defined in the auth-common.env
configuration file file with the NGINX_AUTH_MODE
environment variable: valid values are basic
(or true
for legacy compatibility), to use TLS-encrypted HTTP basic authentication (default); ldap
(or false
for legacy compatibility) to use Lightweight Directory Access Protocol (LDAP) authentication; keycloak
to use authentication managed by Malcolm’s embedded Keycloak instance; keycloak_remote
to use authentication managed by a remote Keycloak instance; or, no_authentication
to disable authentication.
Whichever method is chosen, users must run ./scripts/auth_setup
before starting Malcolm for the first time in order to:
nginx/certs/
directoryBEATS_SSL
environment variable above)
logstash/certs/
directoryfilebeat/certs/
directory; if using Hedgehog Linux, these certificates should be copied to the /opt/sensor/sensor_ctl/logstash-client-certificates
directory on the sensoropensearch/opensearch.keystore
See the the end-to-end Malcolm installation example for a step-by-step guide to the auth_setup
script.
auth_setup
is used to define the username and password for the administrator account. Once Malcolm is running, the administrator account can be used to manage other user accounts via a Malcolm User Management page at https://localhost/auth if connecting locally.
Malcolm user accounts can be used to access the interfaces of all of its components, including Arkime. Arkime uses its own internal database of user accounts, so when a Malcolm user account logs in to Arkime for the first time, Malcolm creates a corresponding Arkime user account automatically. This being the case, it is not recommended to use the Arkime Users settings page or change the password via the Password form under the Arkime Settings page, as those settings would not be consistently used across Malcolm.
Users may change their passwords via the Malcolm User Management page by clicking User Self Service.
The nginx-auth-ldap module serves as the interface between Malcolm’s Nginx web server and a remote LDAP server. When you run auth_setup
for the first time, a sample LDAP configuration file is created at nginx/nginx_ldap.conf
.
# This is a sample configuration for the ldap_server section of nginx.conf.
# Yours will vary depending on how your Active Directory/LDAP server is configured.
# See https://github.com/kvspb/nginx-auth-ldap#available-config-parameters for options.
ldap_server ad_server {
url "ldap://ds.example.com:3268/DC=ds,DC=example,DC=com?sAMAccountName?sub?(objectClass=person)";
binddn "bind_dn";
binddn_passwd "bind_dn_password";
group_attribute member;
group_attribute_is_dn on;
require group "CN=Malcolm,CN=Users,DC=ds,DC=example,DC=com";
require valid_user;
satisfy all;
}
auth_ldap_cache_enabled on;
auth_ldap_cache_expiration_time 10000;
auth_ldap_cache_size 1000;
This file is mounted into the nginx
container when Malcolm is started to provide connection information for the LDAP server.
The contents of nginx_ldap.conf
will vary depending on how the LDAP server is configured. Some of the avaiable parameters in that file include:
url
- the ldap://
or ldaps://
connection URL for the remote LDAP server, which has the following syntax: ldap[s]://<hostname>:<port>/<base_dn>?<attributes>?<scope>?<filter>
binddn
and binddn_password
- the account credentials used to query the LDAP directorygroup_attribute
- the group attribute name that contains the member object (e.g., member
or memberUid
)group_attribute_is_dn
- whether or not to search for the user’s full distinguished name as the value in the group’s member attributerequire
and satisfy
- require user
, require group
and require valid_user
can be used in conjunction with satisfy any
or satisfy all
to limit the users that are allowed to access the Malcolm instanceBefore starting Malcolm, edit nginx/nginx_ldap.conf
according to the specifics of your LDAP server and directory tree structure. Using a LDAP search tool such as ldapsearch
in Linux or dsquery
in Windows may be of help as you formulate the configuration. Your changes should be made within the curly braces of the ldap_server ad_server { … }
section. You can troubleshoot configuration file syntax errors and LDAP connection or credentials issues by running ./scripts/logs
(or docker compose logs nginx
) and examining the output of the nginx
container.
Note that as an alternative to the nginx LDAP integration described in this section, Keycloak can also be configured to federate one or more LDAP servers.
The Malcolm User Management page described above is not available when using LDAP authentication.
Authentication over LDAP can be done using one of three methods, two of which offer data confidentiality protection:
In addition to the NGINX_AUTH_MODE
environment variable being set to ldap
in the auth-common.env
configuration file file, the NGINX_LDAP_TLS_STUNNEL
and NGINX_LDAP_TLS_STUNNEL
environment variables are used in conjunction with the values in nginx/nginx_ldap.conf
to define the LDAP connection security level. Use the following combinations of values to achieve the connection security methods above, respectively:
NGINX_LDAP_TLS_STUNNEL
set to true
in auth-common.env
url
should begin with ldap://
and its port should be either the default LDAP port (389) or the default Global Catalog port (3268) in nginx/nginx_ldap.conf
NGINX_LDAP_TLS_STUNNEL
set to false
in auth-common.env
url
should begin with ldaps://
and its port should be either the default LDAPS port (636) or the default LDAPS Global Catalog port (3269) in nginx/nginx_ldap.conf
NGINX_LDAP_TLS_STUNNEL
set to false
in auth-common.env
url
should begin with ldap://
and its port should be either the default LDAP port (389) or the default Global Catalog port (3268) in nginx/nginx_ldap.conf
For encrypted connections (whether using StartTLS or LDAPS), Malcolm will require and verify certificates when one or more trusted CA certificate files are placed in the nginx/ca-trust/
directory. Otherwise, any certificate presented by the domain server will be accepted.
Malcolm can utilize Keycloak, an identity and access management (IAM) tool, to provide a more robust authentication and authorization experience, including single sign-on (SSO) functionality.
The guides in this section cover configuring Malcolm to use Keycloak for authentication, but do not go into the details of the many capabilities Keycloak provides, including identity providers, SSO protocols, federate one or more LDAP or Kerberos servers, and more. Refer to the Keycloak Server Administration Guide for information on these and other topics.
Due to incompatibilities between Arkime capture on Hedgehog Linux, Malcolm’s nginx reverse proxy, and Keycloak, if Malcolm is using a local OpenSearch instance exposed to external hosts, that endpoint (typically port 9200/tcp) must be authenticated using HTTP basic authentication even when Keycloak is selected as Malcolm’s main authentication method.
When configuring forwarding for arkime-capture on Hedgehog Linux, the credentials entered should be those described above in the section on local account management, not Keycloak credentials. In this configuration, the administrator account can be used to manage other user accounts via a Malcolm User Management page as described above. These credentials are only valid for Malcolm’s local OpenSearch API endpoint.
This limitation does not apply when Malcolm is configured to use a remote OpenSearch or Elasticsearch instance.
The Malcolm development team is exploring solutions to this issue.
This section outlines how to use an existing Keycloak instance managed separately from Malcolm. See Using the embedded Keycloak instance for how to configure Malcolm to use its own embedded Keycloak instance. In addition to the NGINX_AUTH_MODE
variable in auth-common.env
described above, Malcolm uses environment variables in keycloak.env
to configure its use of Keycloak.
These first two steps are performed during configuration:
nginx.env
configuration file.keycloak_remote
will use a remote Keycloak instance.The next steps happen in the context of auth_setup
.
./scripts/auth_setup
and select Configure Keycloak./index.html
, which will redirect users to the Malcolm landing page.With the remote Keycloak configuration complete, start Malcolm.
Run ./scripts/start
and wait a few minutes for Malcolm to start.
Open a web browser and navigate to the hostname or IP address where Malcolm can be reached (e.g., https://malcolm.example.com
), which should redirect to the Sign in to your account Keycloak page provided by the remote Keycloak instance. Enter valid credentials, then click Sign in.
This section outlines how to configure an embedded Keycloak instance managed by Malcolm. See Using a remote Keycloak instance for how to configure Malcolm to use a remote Keycloak instance already extant in the network infrastructure. In addition to the NGINX_AUTH_MODE
variable in auth-common.env
described above, Malcolm uses environment variables in keycloak.env
to configure its use of Keycloak.
As Keycloak is a much more full-featured authentication provider than the local basic authentication method, configuring Malcolm’s embedded Keycloak instance is a bit more involved. The following steps outline how this is done.
These first two steps are performed during configuration:
nginx.env
configuration file.keycloak
will use Malcolm’s embedded Keycloak instance.The next steps happen in the context of ./scripts/auth_setup
.
auth_setup
, this should be done now. If this step has already been done, the user will be warned about regenerating passwords over the top of those that already exist; this is only a concern if the PostgreSQL database already contains data.auth_setup
again, select Configure Keycloak.master
./index.html
, which will redirect users to the Malcolm landing page.With the initial configuration complete, start Malcolm.
Run ./scripts/start
and wait a few minutes for Malcolm to start.
Open a web browser and enter the Keycloak URL specified above (i.e., the hostname or IP address at which Malcolm is available followed by /keycloak), then log in with the bootstrap username and password. Navigating to any other Malcolm URI will result in a “client not found” error until the Keycloak configuration is finished.
In the upper-right hand corner of the page, click on the username for the temporary admin account and select Sign out.
Sign in again with the username and password for the new admin user created in the previous steps.
Once again, navigate to the Users page under Manage on the navigation sidebar. Select the temporary bootstrap admin user account and click Delete user, then confirm by clicking Delete.
malcolm
or myclient
) to identify the new client, then click Next.malcolm.example.com
with the hostname or IP address of the Malcolm instance. Then click Save.
https://malcolm.example.com
https://malcolm.example.com
/*
https://malcolm.example.com/keycloak
https://malcolm.example.com
Return to a command prompt in the Malcolm installation directory and re-run ./scripts/auth_setup
, then select Configure Keycloak.
Leave the realm, redirect URI, and keycloak URL unchanged. Enter the client ID and client secret when prompted. The values for temporary bootstrap username and password can also be left unchanged, as they are irrelevant and will not be used again.
nginx-proxy
container needs to be updated with these new values. Restart the nginx-proxy
container by running ./scripts/restart -s nginx-proxy
(pressing CTRL+C to interrupt the debug log output once the service has restarted), or restart Malcolm entirely./scripts/restart -s nginx-proxy
Stopped Malcolm's ['nginx-proxy'] services
Removed Malcolm's ['nginx-proxy'] services
NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS
malcolm-nginx-proxy-1 ghcr.io/idaholab/malcolm/nginx-proxy:25.03.1 "/sbin/tini -- /usr/…" nginx-proxy 1 second ago Up Less than a second (health: starting)
nginx-proxy-1 | root
nginx-proxy-1 | uid=0(root) gid=0(root) groups=0(root),0(root),1(bin),2(daemon),3(sys),4(adm),6(disk),10(wheel),11(floppy),20(dialout),26(tape),27(video)
nginx-proxy-1 | 2025-03-11 17:29:02,866 INFO Set uid to user 0 succeeded
nginx-proxy-1 | 2025-03-11 17:29:02,873 INFO RPC interface 'supervisor' initialized
nginx-proxy-1 | 2025-03-11 17:29:02,873 CRIT Server 'unix_http_server' running without any HTTP authentication checking
nginx-proxy-1 | 2025-03-11 17:29:02,873 INFO supervisord started with pid 37
nginx-proxy-1 | 2025-03-11 17:29:03,876 INFO spawned: 'logaccess' with pid 111
nginx-proxy-1 | 2025-03-11 17:29:03,878 INFO spawned: 'logerrors' with pid 112
nginx-proxy-1 | 2025-03-11 17:29:03,879 INFO spawned: 'nginx' with pid 113
nginx-proxy-1 | 2025-03-11 17:29:14,283 INFO success: logaccess entered RUNNING state, process has stayed up for > than 10 seconds (startsecs)
nginx-proxy-1 | 2025-03-11 17:29:14,283 INFO success: logerrors entered RUNNING state, process has stayed up for > than 10 seconds (startsecs)
nginx-proxy-1 | 2025-03-11 17:29:14,283 INFO success: nginx entered RUNNING state, process has stayed up for > than 10 seconds (startsecs)
^C
https://malcolm.example.com
), which should redirect to the Sign in to your account Keycloak page. Enter the username and password for the user created above, then click Sign in.Full role-based fine-grained access controls will be implemented in a future release of Malcolm. In the meantime, Malcolm can be configured to require Keycloak-authenticated users to belong to groups and assigned realm roles, respectively. The values for these groups and/or roles are specified when running ./scripts/auth_setup
under Configure Keycloak and are saved as NGINX_REQUIRE_GROUP
and NGINX_REQUIRE_ROLE
in the auth-common.env
configuration file. An empty value for either of these settings means no restriction of that type is applied. Multiple values may be specified with a comma-separated list. These requirements are cumulative: users must match all of the items specified. Note that LDAP authentication can also require group membership, but that is specified in nginx_ldap.conf
by setting require group
rather than in auth-common.env
.
For a discussion of roles vs. groups, see Assigning permissions using roles and groups in the Keycloak Server Administration Guide.
Groups can be managed in Keycloak by selecting the appropriate realm from the drop down at the top of the navigation panel and selecting Groups under Manage.
Users can be joined to groups by clicking on a username on the Keycloak Users page, selecting the Groups tab, then clicking Join Group.
Realm roles can be managed in Keycloak by selecting the appropriate realm from the drop down at the top of the navigation panel and selecting Realm roles under Manage.
Users can be assigned realm roles by clicking on a username on the Keycloak Users page, selecting the Role mapping tab, then clicking Assign role. Select Filter by realm roles, then check the box next to the desired role(s), then click Assign, after which the User role mapping successfully updated confirmation will appear.
Keycloak does not include group or realm role information in authentication tokens by default; clients must be configured to include this information in order for users to log in to Malcolm with group and/or role restrictions set. This can be done by navigating to the Keycloak Clients page, selecting the desired client, then clicking the Client scopes tab. Click on the name of the assigned client scope beginning with the client ID and ending in -dedicated, which will also have a description of “Dedicated scope and mappers for this client.” Once on this Clients > Client details > Dedicated scopes screen, click the down arrow on the Add mapper button and select By configuration.
To include group information in the Keycloak token for this client, select Group Membership from the Configure a new mapper list. The important information to provide for this Group Membership mapper before clicking Save is:
groups
To include user realm role information in the Keycloak token for this client, once again click the down arrow on the Add mapper button and select Byte configuration. Select User Realm Role from the Configure a new mapper list. The important information to provide for this User Realm Role mapper before clicking Save is:
realm_access.roles
Once the mapper(s) have been created, the list of mappers on the Clients > Client details > Dedicated scopes page will look something like this:
When users set up authentication for Malcolm a set of unique self-signed TLS certificates are created which are used to secure the connection between clients (e.g., your web browser) and Malcolm’s browser-based interface. This is adequate for most Malcolm instances as they are often run locally or on internal networks, although your browser will most likely require users to add a security exception for the certificate when first connecting to Malcolm.
Another option is for users to generate their own certificates (or have them issued directly) and have them placed in the nginx/certs/
directory. The certificate and key file should be named cert.pem
and key.pem
, respectively.
A third possibility is to use a third-party reverse proxy (e.g., Traefik or Caddy) to handle the issuance of the certificates and to broker the connections between clients and Malcolm. Reverse proxies such as these often implement the ACME protocol for domain name authentication and can be used to request certificates from certificate authorities such as Let’s Encrypt. In this configuration, the reverse proxy will be encrypting the connections instead of Malcolm, so users will need to set the NGINX_SSL
environment variable to false
in nginx.env
(or answer no
to the “Require encrypted HTTPS connections?” question posed by ./scripts/configure
). If you are setting NGINX_SSL
to false
, make sure user must understand precisely what they are doing, ensuring that external connections cannot reach ports over which Malcolm will be communicating without encryption, including verifying local firewall configuration.
The ./scripts/auth_setup
script can also be run noninteractively which can be useful for scripting Malcolm setup. This behavior can be selected by supplying the --auth-noninteractive
option on the command line. Running with the --help
option will list the arguments accepted by the script:
$ ./scripts/auth_setup --help
usage: auth_setup <arguments>
Malcolm control script
options:
-v [DEBUG], --verbose [DEBUG]
Verbose output
-f <string>, --file <string>
Docker compose or kubeconfig YML file
-e <string>, --environment-dir <string>
Directory containing Malcolm's .env files
…
Authentication Setup:
--auth [CMDAUTHSETUP]
Configure Malcolm authentication
--auth-noninteractive [CMDAUTHSETUPNONINTERACTIVE]
Configure Malcolm authentication (noninteractive using arguments provided)
--auth-admin-username <string>
Administrator username (for --auth-noninteractive)
--auth-admin-password-openssl <string>
Administrator password hash from "openssl -passwd -1" (for --auth-noninteractive)
--auth-admin-password-htpasswd <string>
Administrator password hash from "htpasswd -n -B username | cut -d: -f2" (for --auth-noninteractive)
--auth-arkime-password <string>
Password hash secret for Arkime viewer cluster (for --auth-noninteractive)
--auth-generate-webcerts [AUTHGENWEBCERTS]
(Re)generate self-signed certificates for HTTPS access (for --auth-noninteractive)
--auth-generate-fwcerts [AUTHGENFWCERTS]
(Re)generate self-signed certificates for a remote log forwarder
--auth-generate-netbox-passwords [AUTHGENNETBOXPASSWORDS]
(Re)generate internal passwords for NetBox
--auth-generate-redis-password [AUTHGENREDISPASSWORD]
(Re)generate internal passwords for Redis
--auth-generate-postgres-password [AUTHGENPOSTGRESPASSWORD]
(Re)generate internal superuser passwords for PostgreSQL
--auth-generate-keycloak-db-password [AUTHGENKEYCLOAKDBPASSWORD]
(Re)generate internal passwords for Keycloak's PostgreSQL database
--auth-keycloak-realm <string>
Keycloak realm
--auth-keycloak-redirect-uri <string>
Keycloak redirect URI
--auth-keycloak-url <string>
Keycloak URL
--auth-keycloak-client-id <string>
Keycloak client ID
--auth-keycloak-client-secret <string>
Keycloak client secret
--auth-keycloak-bootstrap-user <string>
Temporary Keycloak admin bootstrap username
--auth-keycloak-bootstrap-password <string>
Temporary Keycloak admin bootstrap password
--auth-require-group <string>
Required group(s) to which users must belong (--auth-method is keycloak|keycloak_remote)
--auth-require-role <string>
Required role(s) which users must be assigned (--auth-method is keycloak|keycloak_remote)
…
In order to avoid specifying passwords as command line arguments, when setting the administrator password noninteractively the user must instead provide hashes generated by openssl -passwd -1
and htpasswd -n -B username
.
When using Keycloak, Malcolm users can log out of Malcolm by navigating to the Keycloak user interface, opening the drop-down with their name or username in the upper-right corner of the page, and selecting Sign out.
For the other authentication methods which use HTTP basic authentication, users should exit their browser to log out of Malcom. On the next browser restart, Malcolm will prompt the user for credentials. Another way to log out of Malcolm is for a user to manually clear their browser’s active sessions.