8. Options¶
Nemesyst uses ConfigArgParse for argument handling. This means you may pass in arguments as (in order of highest priority first):
CLI arguments
Environment variables
ini format .conf config files
Hard-coded defaults
In code Nemesyst will look for config files in the following default locations, in order of priority and with expansion (highest first):
def default_config_files():
"""Default config file generator, for cleaner abstraction.
:return: ordered list of config file expansions
:rtype: list
"""
config_files = [
"./nemesyst.d/*.conf",
"/etc/nemesyst/nemesyst.d/*.conf",
]
return config_files
Using the –config argument you may specify more config files, which will be perpended to the default ones in the order supplied. Please note however config file locations are only followed once to avoid infinite loops where two configs point to each other, making Nemesyst read one then the other infinitely.
8.1. All Options by Category¶
usage: nemesyst [-h] [-U] [--prevent-update] [-c CONFIG [CONFIG ...]]
[--process-pool PROCESS_POOL] [-d DATA [DATA ...]]
[--data-clean]
[--data-cleaner DATA_CLEANER [DATA_CLEANER ...]]
[--data-cleaner-entry-point DATA_CLEANER_ENTRY_POINT [DATA_CLEANER_ENTRY_POINT ...]]
[--data-collection DATA_COLLECTION [DATA_COLLECTION ...]]
[--dl-batch-size DL_BATCH_SIZE [DL_BATCH_SIZE ...]]
[--dl-epochs DL_EPOCHS [DL_EPOCHS ...]] [--dl-learn]
[--dl-learner DL_LEARNER [DL_LEARNER ...]]
[--dl-learner-entry-point DL_LEARNER_ENTRY_POINT [DL_LEARNER_ENTRY_POINT ...]]
[--dl-data-collection DL_DATA_COLLECTION [DL_DATA_COLLECTION ...]]
[--dl-data-pipeline DL_DATA_PIPELINE [DL_DATA_PIPELINE ...]]
[--dl-input-model-collection DL_INPUT_MODEL_COLLECTION [DL_INPUT_MODEL_COLLECTION ...]]
[--dl-input-model-pipeline DL_INPUT_MODEL_PIPELINE [DL_INPUT_MODEL_PIPELINE ...]]
[--dl-output-model-collection DL_OUTPUT_MODEL_COLLECTION [DL_OUTPUT_MODEL_COLLECTION ...]]
[--dl-sequence-length DL_SEQUENCE_LENGTH [DL_SEQUENCE_LENGTH ...]]
[--i-predictor I_PREDICTOR [I_PREDICTOR ...]]
[--i-predictor-entry-point I_PREDICTOR_ENTRY_POINT [I_PREDICTOR_ENTRY_POINT ...]]
[--i-output-prediction-collection I_OUTPUT_PREDICTION_COLLECTION [I_OUTPUT_PREDICTION_COLLECTION ...]]
[--i-predict] [--db-replica-set-name DB_REPLICA_SET_NAME]
[--db-replica-read-preference DB_REPLICA_READ_PREFERENCE]
[--db-replica-max-staleness DB_REPLICA_MAX_STALENESS]
[--db-tls] [--db-tls-ca-file DB_TLS_CA_FILE]
[--db-tls-certificate-key-file DB_TLS_CERTIFICATE_KEY_FILE]
[--db-tls-certificate-key-file-password DB_TLS_CERTIFICATE_KEY_FILE_PASSWORD]
[--db-tls-crl-file DB_TLS_CRL_FILE] [-l] [-s] [-S] [-i]
[--db-user-name DB_USER_NAME] [--db-password DB_PASSWORD]
[--db-intervention] [--db-authentication DB_AUTHENTICATION]
[--db-authentication-database DB_AUTHENTICATION_DATABASE]
[--db-user-role DB_USER_ROLE] [--db-ip DB_IP]
[--db-bind-ip DB_BIND_IP [DB_BIND_IP ...]] [--db-port DB_PORT]
[--db-name DB_NAME] [--db-collection-name DB_COLLECTION_NAME]
[--db-config-path DB_CONFIG_PATH] [--db-path DB_PATH]
[--db-log-path DB_LOG_PATH] [--db-log-name DB_LOG_NAME]
[--db-cursor-timeout DB_CURSOR_TIMEOUT]
[--db-batch-size DB_BATCH_SIZE] [--db-pipeline DB_PIPELINE]
8.1.1. Nemesyst options¶
- -U, --update
Nemesyst update, and restart.
Default: False
- --prevent-update
Prevent nemesyst from updating.
Default: False
- -c, --config
List of all ini files to be used.
Default: []
- --process-pool
The maximum number of processes to allocate.
Default: 1
8.1.2. Data pre-processing options¶
- -d, --data
List of data file paths.
Default: []
- --data-clean
Clean specified data files.
Default: False
- --data-cleaner
Path to data cleaner(s).
Default: []
- --data-cleaner-entry-point
Specify the entry point of custom scripts to use.
Default: [‘main’]
- --data-collection
Specify data storage collection name(s).
Default: [‘debug_data’]
8.1.3. Deep learning options¶
- --dl-batch-size
Batch size of the data to use.
Default: [32]
- --dl-epochs
Number of epochs to train on data.
Default: [1]
- --dl-learn
Use learner scripts.
Default: False
- --dl-learner
Path to learner(s).
Default: []
- --dl-learner-entry-point
Specify the entry point of custom scripts to use.
Default: [‘main’]
- --dl-data-collection
Specify data collection name(s).
Default: [‘debug_data’]
- --dl-data-pipeline
Specify pipeline(s) for data retrieval.
Default: [{}]
- --dl-input-model-collection
Specify model storage collection to retrain from.
Default: [‘debug_models’]
- --dl-input-model-pipeline
Specify model storage collection to retrain from.
Default: [{}]
- --dl-output-model-collection
Specify model storage collection to post trained neural networks to.
Default: [‘debug_models’]
- --dl-sequence-length
List of ints for how long a sequence ofdata should be/ expected.
Default: [32]
8.1.4. Infering options¶
- --i-predictor
Path to predictor(s).
Default: []
- --i-predictor-entry-point
Specify the entry point of predictor custom scripts to use.
Default: [‘main’]
- --i-output-prediction-collection
Specify prediction storage collection to post trained neural network predictions to.
Default: [‘debug_predictions’]
- --i-predict
Use predictor/ inferer scripts.
Default: False
8.1.5. MongoDb replica options¶
- --db-replica-set-name
Set the name for the replica set to use.
- --db-replica-read-preference
Set the read preference of mongo client.
Default: “primary”
- --db-replica-max-staleness
Max seconds replica can be out of sync.
Default: -1
8.1.6. MongoDb TLS options¶
- --db-tls
Set connection to mongodb use TLS.
Default: False
- --db-tls-ca-file
Certificat-authority certificate path.
- --db-tls-certificate-key-file
Clients certificate and key pem path.
- --db-tls-certificate-key-file-password
Set pass if certkey file needs password.
- --db-tls-crl-file
Path to certificate revocation list file.
8.1.7. MongoDb options¶
- -l, --db-login
Nemesyst log into mongodb.
Default: False
- -s, --db-start
Nemesyst launch mongodb.
Default: False
- -S, --db-stop
Nemesyst stop mongodb.
Default: False
- -i, --db-init
Nemesyst initialise mongodb files.
Default: False
- --db-user-name
Set mongodb username.
- --db-password
Set mongodb password.
Default: False
- --db-intervention
Manual intervention during database setup.
Default: False
- --db-authentication
Set the mongodb authentication method.
Default: “SCRAM-SHA-1”
- --db-authentication-database
Override db_name as database to authenticate.
- --db-user-role
Set the users permissions in the database.
Default: “readWrite”
- --db-ip
The ip of the database to connect to.
Default: “localhost”
- --db-bind-ip
The ip the database should be accessible from.
Default: [‘localhost’]
- --db-port
The port both the unauth and auth db will use.
Default: “65535”
- --db-name
The name of the authenticated database.
Default: “nemesyst”
- --db-collection-name
The name of the collection to use in database.
Default: “test”
- --db-config-path
The path to the mongodb configuration file.
- --db-path
The parent directory to use for the database.
Default: /home/docs/db
- --db-log-path
The parent directory to use for the db log.
Default: /home/docs/db/log
- --db-log-name
The base name of the log file to maintain.
Default: “mongo_log”
- --db-cursor-timeout
The duration in seconds before an unused cursor will time out.
Default: 600000
- --db-batch-size
The number of documents to return from the db at once/ pre round.
Default: 32
- --db-pipeline
The file path of the pipeline to use on db.