Resolve "persistent MQTT messages"

Closes #79 (closed)

Suggestion for testing:

• start with docker-compose-dev(.example).yml to mount dispatcher into workers and extractor into basic-demo-scheduler
  • checkout this tsm-orchestration branch 79-persistent-mqtt-messages
  • checkout tsm-dispatcher branch 29-persistent-mqtt-message (or use DISPATCHER_IMAGE_TAG=29-persistent-mqtt-message)
  • checkout tsm-extractor branch 31-mqtt-message-persistence
• stop all workers

go through all the workers by generating the respective mqtt messages

thing_creation

• populate frontend (e.g. with ./tsm-workflow populate from tsm-dev-tools)
• restart thing_creation workers (...db_setup, ...object_storage_setup, ...frost_setup, ...mqtt_user_creation, ...grafana_dashboard)
• check whether they received a message

object_storage_notification

• upload a file to minio bucket (e.b. with ./tsm-workflow upload)
• restart worker-file-ingest
• check whether message was received

data_parsed

• if data was parsed successfully, the extractor should send a notification to topic data_parsed
  • check basic-demo-scheduler logs whether data was parsed successfully
• restart worker-run-qaqc
• check whether message was received

user_login

• log in to frontend via helmholtz aai
  • might need to to remove all data, otherwise there could be a conflict with user groups in the frontenddb
• restart worker-grafana-user-orgs and check for message

I hope all works as expected

assigned to @joost.hemmen1

added 1 commit

• 314b5bfa - make MQTT messages persistent with MQTT_QOS=2 et al.

Compare with previous version

changed the description

added 1 commit

• 38427a9f - pass QOS and CLEAN_SESSION from .env

Compare with previous version

marked this merge request as ready

requested review from @florian.gransee

see also tsm-dispatcher!80 (merged) and tsm-extractor!43 (merged)

mentioned in merge request tsm-dispatcher!80 (merged)

mentioned in merge request tsm-extractor!43 (merged)

added 1 commit

• b78446f9 - adopting switch back to single docker-compose.yml

Compare with previous version

added 34 commits

• b78446f9...2efeff76 - 33 commits from branch main
• 227cff5f - Merge branch 'main' into '79-persistent-mqtt-messages'

Compare with previous version

changed the description

merged

approved this merge request

mentioned in commit f3997b98