Changes

Add creator of shard regions with a custom hand-off message.

Signed-off-by: Yufei Cai <yufei.cai@bosch.io>

Make ThingUpdater participate in shard hand-off; test self restart of sharded actors.

Signed-off-by: Yufei Cai <yufei.cai@bosch.io>

Make StopShardedActor java-serializable for tests; restructure ShardRegionCreatorTest.

Signed-off-by: Yufei Cai <yufei.cai@bosch.io>

Integrate SearchActor into coordinated shutdown.

Signed-off-by: Yufei Cai <yufei.cai@bosch.io>

[wip] send thing events directly to search updater shard region.

Signed-off-by: Yufei Cai <yufei.cai@bosch.io>

Replace Ditto pubsub by direct shard messaging for search update.

Reason: Ditto pubsub reacts too slowly for coordinated shutdown.

Signed-off-by: Yufei Cai <yufei.cai@bosch.io>

Add signals to things registry tests.

Signed-off-by: Yufei Cai <yufei.cai@bosch.io>

Integrate policies into Akka coordinated shutdown.

Signed-off-by: Yufei Cai <yufei.cai@bosch.io>

Add PolicySupervisorActorTest; fix unit tests.

Signed-off-by: Yufei Cai <yufei.cai@bosch.io>

add private constructor for ShardRegionCreator;

Signed-off-by: Stefan Maute <stefan.maute@bosch.io>

minor formatting changes;

Signed-off-by: Stefan Maute <stefan.maute@bosch.io>

Review: minor code formatting change

Signed-off-by: Stefan Maute <stefan.maute@bosch.io>

Integrate Things into Akka coordinated shutdown.

Signed-off-by: Yufei Cai <yufei.cai@bosch.io>

Also terminate twin operations on coordinated shutdown after timeout.

Reason: Twin operations can include acknowledgement requests.

Signed-off-by: Yufei Cai <yufei.cai@bosch.io>

remove thing-event-aware from test config;

Signed-off-by: Stefan Maute <stefan.maute@bosch.io>

add remove method to DistPubSubAccess in order to remove an actor from pubSubMediator;

Signed-off-by: Stefan Maute <stefan.maute@bosch.io>

made second start method of ShardRegionCreator also public;

Signed-off-by: Stefan Maute <stefan.maute@bosch.io>

extend logging for unsubscribing from pubsub;

Signed-off-by: Stefan Maute <stefan.maute@bosch.io>

adjust method name and logging;
fixed javadoc and typo;

Signed-off-by: Stefan Maute <stefan.maute@bosch.io>

increased coordinated-shutdown-timeout to 65s to give the requests enough time to complete;
removed match for DistributedPubSubMediator.SubscribeAck from GatewayProxyActor and EdgeCommandForwarderActor because no subscribe message is send via pudSubMediator;
wait with stopping the AbstractHttpRequestActor until the request is completed;
add test to HttpRequestActorTest;

Signed-off-by: Stefan Maute <stefan.maute@bosch.io>

made coordinated shutdown task in AbstractHttpRequestActor cancellable and remove task when actor is shutdown;
wait for requests in QueryThingsPerRequestActor;
pass httpConfig to GatewayProxyActor and QueryThingsPerRequestActor;
add test to QueryThingsPerRequestActorTest,

Signed-off-by: Stefan Maute <stefan.maute@bosch.io>

add new exception GatewayWebsocketSessionAbortedException which is used for terminating the websocket session;
extend Connect with an optional killSwitch;
shutdown websocket streams in service requests done phase of CoordinatedShutdown;

Signed-off-by: Stefan Maute <stefan.maute@bosch.io>

cleanup unused variable;
rename method;

Signed-off-by: Stefan Maute <stefan.maute@bosch.io>

Extract group-index-key from Ditto PubSub so that it is not restricted to signals with entity ID.

Signed-off-by: Yufei Cai <yufei.cai@bosch.io>

provide announcement blogpost for Ditto 3.0 release
* also includes some fixes to OpenAPI

Signed-off-by: Thomas Jaeckle <thomas.jaeckle@bosch.io>

made StopShardedActor an implementation of AkkaJacksonCborSerializable because java serialization is disabled;

Signed-off-by: Stefan Maute <stefan.maute@bosch.io>

Replace ClientActorRefs by ConnectionPubSub for consistency during coordinated shutdown.

Signed-off-by: Yufei Cai <yufei.cai@bosch.io>

extend AbstractPersistenceOperationsActor with getActorName method;
add graceful shutdown behaviour to AbstractPersistenceOperationsActor;
add test for shutdown behaviour to PolicyPersistenceOperationsActorIT;

Signed-off-by: Stefan Maute <stefan.maute@bosch.io>

add ditto-internal-utils-akka as test dependency to pom.xml

Signed-off-by: Stefan Maute <stefan.maute@bosch.io>

Restore distinction between inbound and outbound signals after going through ConnectionPubSub.

Signed-off-by: Yufei Cai <yufei.cai@bosch.io>

save purgeNamespace and purgeEntities commands in a map while processing and re-publish them in case the actor gets shutdown;
adapt PolicyPersistenceOperationsActorIT;

Signed-off-by: Stefan Maute <stefan.maute@bosch.io>

terminate persistence cleanup stream when graceful shutdown is triggered;
add test to PersistenceCleanupActor;

Signed-off-by: Stefan Maute <stefan.maute@bosch.io>

changed signature of Policy.isSemanticallySameAs to check for another Policy

Signed-off-by: Thomas Jaeckle <thomas.jaeckle@bosch.io>

add shutdown behaviour to SnapshotStreamingActor and terminate stream in PhaseServiceRequestsDone;
add test for shutdown behaviour to SnapshotStreamingActorTest;
change PubSub mechanism from send to publish for SudoStreamSnapshots commands in ThingsMetadataSource and SnapshotStreamingActor;

Signed-off-by: Stefan Maute <stefan.maute@bosch.io>

removed Ditto version 1.x from docs dropdown
* updated SECURITY.md to support 3.0 and 2.4

Signed-off-by: Thomas Jaeckle <thomas.jaeckle@bosch.io>

fix BackgroundSyncActorTest;

Signed-off-by: Stefan Maute <stefan.maute@bosch.io>

moved actor name of SnapshotStreamingActor into the actor itself;
add ditto-internal-utils-persistence dependency to ditto-things-api;

Signed-off-by: Stefan Maute <stefan.maute@bosch.io>

made Cancellable nullable and initialize it with null to fix sonar issue;

Signed-off-by: Stefan Maute <stefan.maute@bosch.io>

subscribe via PubSub for RetrieveThings and SudoRetrieveThings in ThingsAggregatorActor and publish commands in ThingsAggregatorProxyActor;
add unsubscribe task for graceful shutdown;
add shutdown behaviour to ThingsAggregatorProxyActor;
add test;

Signed-off-by: Stefan Maute <stefan.maute@bosch.io>

set logger akka.actor.CoordinatedShutdown to INFO instead DEBUG;

Signed-off-by: Stefan Maute <stefan.maute@bosch.io>

updated swagger-ui to 4.14.1

Signed-off-by: Thomas Jaeckle <thomas.jaeckle@bosch.io>

adjusted latest additions to Ditto 3.0 and updated the potential release date

Signed-off-by: Thomas Jaeckle <thomas.jaeckle@bosch.io>

remove HealthSignal from BaseClientActor because it isn't needed anymore;

Signed-off-by: Stefan Maute <stefan.maute@bosch.io>

remove putting searchactor ref to pubSubMediator because searchActor registeres itself via pubSub;

Signed-off-by: Stefan Maute <stefan.maute@bosch.io>

increased ditto.http.coordinated-shutdown-timeout to 65s in ditto-http.conf;

Signed-off-by: Stefan Maute <stefan.maute@bosch.io>

fixed info log for successfully unsubscribing from pubSub;

Signed-off-by: Stefan Maute <stefan.maute@bosch.io>

add searchActor to pubSub in SearchRootActor;

Signed-off-by: Stefan Maute <stefan.maute@bosch.io>

handle SERVICE_REQUESTS_DONE inAnyState() of PersistenceCleanupActor;
unsubscribe from pubSub only if the subscription was done in AbstractPersistenceOperationsActor;

Signed-off-by: Stefan Maute <stefan.maute@bosch.io>

use Control.SERVICE_REQUESTS_DONE of PersistenceCleanupActor;

Signed-off-by: Stefan Maute <stefan.maute@bosch.io>

re-enable Debug logs for akka.actor.CoordinatedShutdown in things;

Signed-off-by: Stefan Maute <stefan.maute@bosch.io>

extend logging;

Signed-off-by: Stefan Maute <stefan.maute@bosch.io>

extracted an abstract class AbstractActorWithShutdownBehavior which handles the shutdown behaviour of Actors;
refactored several Actors to use the new abstract class;

Signed-off-by: Stefan Maute <stefan.maute@bosch.io>

also persist "revision" in journal entries to make it possible to read when replaying events

Signed-off-by: Thomas Jaeckle <thomas.jaeckle@bosch.io>

post-release 3.0.0 commit:
* updated binary compatibility check; removed exclusions

Signed-off-by: Thomas Jaeckle <thomas.jaeckle@bosch.io>

add parameter to include loggers set to "off"

Signed-off-by: Johannes Schneider <johannes.schneider@bosch.io>

Stop consuming and unsubscribe from PubSub on Service-Unbind.

Signed-off-by: Yufei Cai <yufei.cai@bosch.io>

fixed remaining mentions of "java -jar" usage to start containers

Signed-off-by: Thomas Jaeckle <thomas.jaeckle@bosch.io>

Stop client actors at the end of tests; do not wait for graceful shutdown in client actor tests.

Signed-off-by: Yufei Cai <yufei.cai@bosch.io>

Test stopping consumers during graceful shutdown.

Signed-off-by: Yufei Cai <yufei.cai@bosch.io>

Use correct key for setting correlation ID to tags.

Furthermore, added null-checks and removed some redundancy in code.

Signed-off-by: Juergen Fickel <juergen.fickel@bosch.io>

Use correct key for setting correlation ID to tags.

Furthermore, added null-checks and removed some redundancy in code.

Signed-off-by: Juergen Fickel <juergen.fickel@bosch.io>

Fix review finding

Signed-off-by: David Schwilk <david.schwilk@bosch.io>

Wait for active acknowledgement aggregator actors for a short time during coordinated shutdown.

Signed-off-by: Yufei Cai <yufei.cai@bosch.io>

Bump scala-library from 2.13.8 to 2.13.9 in /bom

Bumps [scala-library](https://github.com/scala/scala) from 2.13.8 to 2.13.9.
- [Release notes](https://github.com/scala/scala/releases)
- [Commits](https://github.com/scala/scala/compare/v2.13.8...v2.13.9)

---
updated-dependencies:
- dependency-name: org.scala-lang:scala-library
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

Omit initial WebSocket request "/ws/2" from tracing in `RequestTracingDirective`.

Furthermore,
* introduced `TraceOperationName` to be used for trace operation names instead of a plain string.
* Removed unused methods from `DittoTracing` to reduce dependencies which are not proximately related to tracing, e.g. to akka-http.

Signed-off-by: Juergen Fickel <juergen.fickel@bosch.io>

Explorer UI: URL parameters for environments
Additional improvements:
* Stacking for (multiple) error toasts
* Avoid error thrown twice in ditto rest call
* Optimized layout for long thingIds in tables (hidden)
* fixed some eslint issues in things.js

Signed-off-by: thfries <thomas.fries0@gmail.com>

add new class AbstractActorWithShutdownBehaviorAndRequestCounting which implements shutdown and request counting behavior;
use new abstract class for ThingsAggregatorActor, ThingsAggregatorProxyActor, SnapshotStreamingActor, AbstractPersistenceOperationsActor and SearchActor;

Signed-off-by: Stefan Maute <stefan.maute@bosch.io>

ignored flaky unit test again

Signed-off-by: Thomas Jaeckle <thomas.jaeckle@bosch.io>

Move client actors to their own shard region.

Signed-off-by: Yufei Cai <yufei.cai@bosch.io>

Remove headers as a part of client actor props to prevent accidental restarts.

Signed-off-by: Yufei Cai <yufei.cai@bosch.io>

Use inbound AMQP 1.0 message annotations

If the annotation equals a well known AMQP header it gets prefixed, in the same way this is done for application properties

Co-authored-by: Stanchev Aleksandar <aleksandar.stanchev@bosch.io>
Signed-off-by: David Schwilk <david.schwilk@bosch.io>

Set message annotations for outbound AMQP messages

Co-authored-by: Stanchev Aleksandar <aleksandar.stanchev@bosch.io>
Signed-off-by: David Schwilk <david.schwilk@bosch.io>

Add documentation about AMQP message annotation header mapping

Co-authored-by: Stanchev Aleksandar <aleksandar.stanchev@bosch.io>
Signed-off-by: David Schwilk <david.schwilk@bosch.io>

Prefix message annotations

Co-authored-by: Stanchev Aleksandar <aleksandar.stanchev@bosch.io>
Signed-off-by: David Schwilk <david.schwilk@bosch.io>

Fix client shard region name; reintroduce CLOSE_CONNECTION stage to restore connection connection announcements.

Signed-off-by: Yufei Cai <yufei.cai@bosch.io>

ignored flaky unit test again

Co-authored-by: Stanchev Aleksandar <aleksandar.stanchev@bosch.io>
Signed-off-by: Thomas Jaeckle <thomas.jaeckle@bosch.io>

fix that connection "remember-entities" based automatic start of "known" connections did no longer work
* the "initialization" was broken as no configuration was retrieved when being restored without initially processing a message

Co-authored-by: Stanchev Aleksandar <aleksandar.stanchev@bosch.io>
Signed-off-by: Thomas Jaeckle <thomas.jaeckle@bosch.io>

configure remember-entities-store = "eventsourced"
* enable "remember-entities" by default as "file access" is not required any longer with that config
* adjusted ping config to wait for the remember-entities being recovered first

Co-authored-by: Stanchev Aleksandar <aleksandar.stanchev@bosch.io>
Signed-off-by: Thomas Jaeckle <thomas.jaeckle@bosch.io>

Accept and test TextMessage with null payload

Signed-off-by: Tobias Månsson <tobias@zept.io>

Fix close-connection behavior of ClientSupervisor; fix ConnectionPersistenceActorTest.

Signed-off-by: Yufei Cai <yufei.cai@bosch.io>

Add hand-off messages to connection and client shard regions.

Signed-off-by: Yufei Cai <yufei.cai@bosch.io>

Replaced absolute css units in explorer UI

Signed-off-by: thfries <thomas.fries0@gmail.com>

Move client actor shutdown to shard region shutdown phase to prevent restarts during coordinated shutdown.

Signed-off-by: Yufei Cai <yufei.cai@bosch.io>

Tolerate lower case connection status.

Signed-off-by: Yufei Cai <yufei.cai@bosch.io>

Explorer UI: Set URL param for environment

Signed-off-by: thfries <thomas.fries0@gmail.com>

New documentation page for ui

Signed-off-by: thfries <thomas.fries0@gmail.com>

Changed Ditto version in UI environment to enum

Signed-off-by: thfries <thomas.fries0@gmail.com>

Added screenshot to UI documentation

Signed-off-by: thfries <thomas.fries0@gmail.com>

fixed adding additional tags for ditto-ui docker image

fixed docker-ui rc publishing

added workflow for pushing docker images on demand

adjusted push-dockerhub-on-demand.yml script to include choice option for which image to push

Signed-off-by: Thomas Jaeckle <thomas.jaeckle@bosch.io>

Replace consistent hashable envelope by a dedicated binary message envelope for communication between the connection persistence actor and its client actors.

Reason: Consistent hashable envelope is not serializable.

Still to do: Make all messages wrapped in ShardedBinaryEnvelope serializable, i. e., ClientActorPropsArgs.

Signed-off-by: Yufei Cai <yufei.cai@bosch.io>

OpenAPI docs: fixed "connectionStatus" type of "Connection" in OpenAPI definition

Signed-off-by: Thomas Jaeckle <thomas.jaeckle@bosch.io>

update swagger-ui fixing security issues
* configured links to Ditto Explorer UI to directly point to the "primaryEnvironmentName=ditto_sandbox"

Signed-off-by: Thomas Jaeckle <thomas.jaeckle@bosch.io>

Add serializer for ClientActorPropsArgs.

Signed-off-by: Yufei Cai <yufei.cai@bosch.io>

Update documentation/src/main/resources/pages/ditto/user-interface.md

Co-authored-by: Thomas Jaeckle <thomas.jaeckle@bosch.io>

Update documentation/src/main/resources/pages/ditto/user-interface.md

Co-authored-by: Thomas Jaeckle <thomas.jaeckle@bosch.io>

Update documentation/src/main/resources/pages/ditto/user-interface.md

Co-authored-by: Thomas Jaeckle <thomas.jaeckle@bosch.io>

UI Documentation: feedback from review
- remove trade mark
- change order in sidebar
- make links to online sandbox work

Signed-off-by: thfries <thomas.fries0@gmail.com>

nullLiteral() and @Nullable

added slides about Ditto + WoT
to present at WoT CommunityGroup meeting in 10/2022

Signed-off-by: Thomas Jaeckle <thomas.jaeckle@bosch.io>

bump scala.full.version to 2.13.10, kafka-client.version to 3.0.2 and netty-bom.version to 4.1.82.Final;

Signed-off-by: Stefan Maute <stefan.maute@bosch.io>

made presentation better printable

Signed-off-by: Thomas Jaeckle <thomas.jaeckle@bosch.io>

added Ditto/WoT presentation to presentations.md

Signed-off-by: Thomas Jaeckle <thomas.jaeckle@bosch.io>

updates to the "What, why and how" of the WoT integration

Signed-off-by: Thomas Jaeckle <thomas.jaeckle@bosch.io>

extend timeout for test to 5 sec;

Signed-off-by: Stefan Maute <stefan.maute@bosch.io>

last polishing of WoT presentation
* fixed links to github in all previous presentations

Signed-off-by: Thomas Jaeckle <thomas.jaeckle@bosch.io>

increase timeout for test to 15 sec;

Signed-off-by: Stefan Maute <stefan.maute@bosch.io>

Add vaildation of condition header for live messages

This should enable the possibility to prevent live message publishing based on the condition header not matching.

Co-authored-by: Stanchev Aleksandar <aleksandar.stanchev@bosch.io>
Signed-off-by: David Schwilk <david.schwilk@bosch.io>

Adjust documentation for supported live message conditions

Co-authored-by: Stanchev Aleksandar <aleksandar.stanchev@bosch.io>
Signed-off-by: David Schwilk <david.schwilk@bosch.io>

Only increase timeout for flaky test

Signed-off-by: David Schwilk <david.schwilk@bosch.io>

Make ShardedBinaryEnvelope self-delimiting; log failed serialization.

Signed-off-by: Yufei Cai <yufei.cai@bosch.io>

Fix endian-ness of serializer; fix serialization of SudoRetrieveConnectionStatusResponse.

Signed-off-by: Yufei Cai <yufei.cai@bosch.io>

Refactored `DittoTracing` to slim down its API and get rid of direct dependencies to Kamon.

* `DittoTracing` now also reflects its actual state.
* Wrapped Kamon HTTP context propagation in `KamonHttpContextPropagation` for passing it around.
* When creating a prepared span, some well-known tags like for example correlation ID are automatically put if they can be found in the specified headers.
* Introduced new simple wrapper types to leverage the benefits of strong and static typing.
* Empty traces now have an operation name.

Signed-off-by: Juergen Fickel <juergen.fickel@bosch.io>