Advanced Config

There are many other things that can be configured and tweaked according to your needs or use case. Some of this config is mandatory and other configuration is optional. These properties can be configured in the property file located in ./storage/backend/conf/craftworks-navio.properties.

Mandatory Properties

Spring Profile

A spring profile needs to be set. For the application to run properly in a production environment use the following line:

# Active Profiles, use prod for production environment
spring.profiles.active=prod

File Mounts

# file mount from inside the container
files.main.directory=/shared_filesystem/main
files.data.directory=/shared_filesystem/data
# file mount paths on the host system
files.main.host-directory=/mnt/shared_filesystem/main

Logging

# log files will be stored here
logging.path.file=./app/local_storage/logs/navio_backend.log
# recommended default log levels
logging.level.org.apache.spark=WARN
logging.level.org.apache.parquet=WARN
logging.level.org.apache.hadoop=WARN
logging.level.org.spark_project=WARN
spark.submit.logging.level=INFO
logging.level.at.craftworks.dataanalytics.predictor=INFO
logging.pattern.level=%5p [u:%X{dapuser}]

Security

# Security relevant configuration
# please replace the secret keys with random strings (around 32-40 characters long)
jwt.token.validity.seconds=180
jwt.token.refresh.validity.minutes=2000
jwt.token.secret=${SECRET_KEY1}
jwt.token.refresh.secret=${SECRET_KEY2}
jwt.token.navio.api.secret=${SECRET_KEY3}
jwt.token.internal.api.secret=${SECRET_KEY4}

# you can enable this property to include detailed exception messages in REST responses
# only for testing purposes, not for production environments
dap.rest.exception-details=false

Default User

craftworks.users[0].name=${DEFAULT_USER_EMAIL}
craftworks.users[0].password=${DEFAULT_PASSWORD}
craftworks.users[0].workspaces[0]=${DEFAULT_WORKSPACE}

Contains the following placeholders:

${DEFAULT_PASSWORD} - password for default user

${DEFAULT_USER_EMAIL} - email address for the default user i.e. admin@craftworks.at

${DEFAULT_WORKSPACE} - default workspace i.e. name of your company

# the docker network you use for navio (as configured in your docker-compose file)
app.docker.network=navio

# This config file configures some default values, the config is prepackaged and there is no need to change it
dap.config.shared.path=file:/app/config/shared-config.json

Email

# the Sendgrid API Key
mail.sendgrid.api-key=${SENDGRID-API-KEY}
# Message sent by navio will have this sender/name
mail.no-reply.sender=noreply@xxx.io
mail.no-reply.sender-name=navio

# the following properties configure how the mails sent by navio look like
# you have to configure 3 sendgrid templates (
mail.reset-password.link=https://xxx/auth/password-reset?email=%s&token=%s
mail.reset-password.template=SENDGRID TEMPLATE
mail.user-activate.link=https://xxx/auth/user-activate?email=%s&token=%s
mail.user-activate.template=SENDGRID TEMPLATE
mail.workspace-access-granted-info.link=https://xxx/dashboard/workspaces/%s
mail.workspace-access-granted-info.template=SENDGRID TEMPLATE

Cleanup & Retention

# Cleanup
# extended config when and how cleanups happen. It is recommended not to change this
dap.pipeline-run.max-training-time-factor=2
dap.pipeline-run.max-training-retries=5
dap.pipeline-run.training-status-check-interval=0 0/15 * * * *

dap.docker-image.retention-period=30d
dap.docker-image.cleanup-interval = 0 0 1 * * *
dap.dataset-verify-interval=0 0 2 * * *
dap.dataset-cleanup-interval=0 0 3 * * *
dap.dataset-cleanup-retention-period=1d
dap.custom-pipeline-cleanup-interval=0 0 3 * * *
dap.custom-pipeline-cleanup-retention-period=2h

Monitoring

# Prometheus
# URL of prometheus (see extended setup for more info)
dap.monitoring.prometheus.base-url= http://prometheus:9090
# set this to a secret password, so prometheus can access the information
monitoring.actuator.password=ACTUATOR_PASSWORD
management.endpoints.web.exposure.include=*
management.endpoint.health.show-details=when_authorized

Explainer

# Prediction Explainer
# URL of explainer (see extended setup for more info)
dap.explainer.base.path=http://prediction-explainer:5000
dap.explainer.self.base-url=http://backend:8080

Optional Properties

Javamelody

Application monitoring using JavaMelody is supported. The properties below are required to enable this.

tip

Consult the JavaMelody documentation for a complete list of properties.

javamelody.enabled=true
javamelody.init-parameters.application-name=craftworks-navio
javamelody.init-parameters.storage-directory=/app/local_storage/melody
javamelody.init-parameters.url-exclude-pattern=(/webjars/.*|/css/.*|/images/.*|/fonts/.*|/js/.*)

All Properties

To simplify copying the properties, here you can see a list of all the above mentioned properties in one place. Keep in mind that it contains placeholders as described above.

Complete List of Properties

# Active Profiles, use prod for production environment
spring.profiles.active=prod

# file mount from inside the container
files.main.directory=/shared_filesystem/main
files.data.directory=/shared_filesystem/data
# file mount paths on the host system
files.main.host-directory=/mnt/shared_filesystem/main

# log files will be stored here
logging.path.file=./app/local_storage/logs/navio_backend.log
# recommended default log levels
logging.level.org.apache.spark=WARN
logging.level.org.apache.parquet=WARN
logging.level.org.apache.hadoop=WARN
logging.level.org.spark_project=WARN
spark.submit.logging.level=INFO
logging.level.at.craftworks.dataanalytics.predictor=INFO
logging.pattern.level=%5p [u:%X{dapuser}]

# Security relevant configuration
# please replace the secret keys with random strings (around 32-40 characters long)
jwt.token.validity.seconds=180
jwt.token.refresh.validity.minutes=2000
jwt.token.secret=SECRET KEY1
jwt.token.refresh.secret=SECRET KEY2
jwt.token.navio.api.secret=SECRET KEY3
jwt.token.internal.api.secret=SECRET KEY4

craftworks.users[0].name=admin@craftworks.at
craftworks.users[0].password=default pwd
craftworks.users[0].workspaces[0]=company

# the docker network you use for navio (as configured in your docker-compose file)
**app.docker.network=navio**

# This config file configures some default values, the config is prepackaged and there is no need to change it
dap.config.shared.path=file:/app/config/shared-config.json

# you can enable this property to include detailed exception messages in REST responses
# only for testing purposes, not for production environments
dap.rest.exception-details=false

# Mail
# the Sendgrid API Key
mail.sendgrid.api-key=SENDGRID-API-KEY
# Message sent by navio will have this sender/name
mail.no-reply.sender=noreply@xxx.io
mail.no-reply.sender-name=navio

# the following properties configure how the mails sent by navio look like
# you have to configure 3 sendgrid templates (
mail.reset-password.link=https://xxx/auth/password-reset?email=%s&token=%s
mail.reset-password.template=SENDGRID TEMPLATE
mail.user-activate.link=https://xxx/auth/user-activate?email=%s&token=%s
mail.user-activate.template=SENDGRID TEMPLATE
mail.workspace-access-granted-info.link=https://xxx/dashboard/workspaces/%s
mail.workspace-access-granted-info.template=SENDGRID TEMPLATE

# Cleanup
# extended config when and how cleanups happen. It is recommended not to change this
dap.pipeline-run.max-training-time-factor=2
dap.pipeline-run.max-training-retries=5
dap.pipeline-run.training-status-check-interval=0 0/15 * * * *

dap.docker-image.retention-period=30d
dap.docker-image.cleanup-interval = 0 0 1 * * *
dap.dataset-verify-interval=0 0 2 * * *
dap.dataset-cleanup-interval=0 0 3 * * *
dap.dataset-cleanup-retention-period=1d
dap.custom-pipeline-cleanup-interval=0 0 3 * * *
dap.custom-pipeline-cleanup-retention-period=2h

# Prometheus
# URL of prometheus (see extended setup for more info)
dap.monitoring.prometheus.base-url= http://prometheus:9090
# set this to a secret password, so prometheus can access the information
monitoring.actuator.password=ACTUATOR_PASSWORD
management.endpoints.web.exposure.include=*
management.endpoint.health.show-details=when_authorized

# Prediction Explainer
# URL of explainer (see extended setup for more info)
dap.explainer.base.path=http://prediction-explainer:5000
dap.explainer.self.base-url=http://backend:8080

javamelody.enabled=true
javamelody.init-parameters.application-name=craftworks-navio
javamelody.init-parameters.storage-directory=/app/local_storage/melody
javamelody.init-parameters.url-exclude-pattern=(/webjars/.*|/css/.*|/images/.*|/fonts/.*|/js/.*)

Advanced Config

Mandatory Properties​

Spring Profile​

File Mounts​

Logging​

Security​

Default User​

Email​

Cleanup & Retention​

Monitoring​

Explainer​

Optional Properties​

Javamelody​

tip

All Properties​

Mandatory Properties

Spring Profile

File Mounts

Logging

Security

Default User

Email

Cleanup & Retention

Monitoring

Explainer

Optional Properties

Javamelody

All Properties