Changes

Summary

1. Add creator of shard regions with a custom hand-off message. (details)
2. Make ThingUpdater participate in shard hand-off; test self restart of sharded actors. (details)
3. Make StopShardedActor java-serializable for tests; restructure ShardRegionCreatorTest. (details)
4. Logging and validation (details)
5. Payload Validation Test (details)
6. Mapping incoming CloudEvents with Ditto Protocol in Data (details)
7. re-committing CloudEvents mapper (details)
8. reformat (details)
9. remove main function from mapper (details)
10. Remove logging statements (details)
11. change license header year (details)
12. remove Tests (details)
13. Add mapping Tests (details)
14. More Tests (details)
15. fix build (details)
16. validate headers (details)
17. Integrate SearchActor into coordinated shutdown. (details)
18. Initial Binary CE support (details)
19. formatting and minor changes (details)
20. fix build (details)
21. [wip] send thing events directly to search updater shard region. (details)
22. Replace Ditto pubsub by direct shard messaging for search update. (details)
23. Add signals to things registry tests. (details)
24. Integrate policies into Akka coordinated shutdown. (details)
25. Add PolicySupervisorActorTest; fix unit tests. (details)
26. add private constructor for ShardRegionCreator; (details)
27. Placeholder Outbound Mapping (details)
28. minor formatting changes; (details)
29. build exception and content-type check (details)
30. change content-type for tests (details)
31. ExternalMessaqge building with Json (details)
32. Review: minor code formatting change (details)
33. binary CE test (details)
34. Integrate Things into Akka coordinated shutdown. (details)
35. Documentation (details)
36. Also terminate twin operations on coordinated shutdown after timeout. (details)
37. add documentation plus remove blocklist (details)
38. add Test plus minor housekeeping (details)
39. remove thing-event-aware from test config; (details)
40. add remove method to DistPubSubAccess in order to remove an actor from pubSubMediator; (details)
41. reformatting (details)
42. remove InboundDispatchSink from PR (details)
43. reformat InboundDispatchSink (details)
44. remove comments and other necessary changes (details)
45. minor reformatting (details)
46. exception handling (details)
47. reformat docs (details)
48. allow filtering a single background sync iteration to a set of namespaces (e.g. to forcefully re-index all things within a given namespace) (details)
49. exception handling (details)
50. made second start method of ShardRegionCreator also public; (details)
51. Update documentation/src/main/resources/pages/ditto/connectivity-mapping.md (details)
52. Update documentation/src/main/resources/pages/ditto/connectivity-mapping.md (details)
53. extend logging for unsubscribing from pubsub; (details)
54. Binary CE supports content type application/vnd.eclipse.ditto+json (details)
55. fix build (details)
56. Filter MQTT messages where the message topic does not match the source topic (details)
57. move creation of list of topic filters out of message stream (details)
58. adjust method name and logging; (details)
59. Outbound spec changes and extract Id from correlation-id (details)
60. Adjust the command response enforcement exception handling (details)
61. increased coordinated-shutdown-timeout to 65s to give the requests enough time to complete; (details)
62. applied Ditto code formatter to CloudEventsMapper (details)
63. made coordinated shutdown task in AbstractHttpRequestActor cancellable and remove task when actor is shutdown; (details)
64. add new exception GatewayWebsocketSessionAbortedException which is used for terminating the websocket session; (details)
65. cleanup unused variable; (details)
66. Throw ThingNotAccessibleException when thing is deleted and policy not available (details)
67. Extract group-index-key from Ditto PubSub so that it is not restricted to signals with entity ID. (details)
68. fix unittest (details)
69. adjusted Ditto explorer UI to use new HTTP routes for /connections (details)
70. fix intendation in connections.js (details)
71. added input field for selecting "Ditto version" in environments tab (details)
72. provide announcement blogpost for Ditto 3.0 release (details)
73. fixed UnsupportedOperationException in DefaultWoTThingDescriptionGenerator (details)
74. some small fixes for usability and performance improvements: (details)
75. adjusted input type from "integer" to "number" (details)
76. made StopShardedActor an implementation of AkkaJacksonCborSerializable because java serialization is disabled; (details)
77. fixed check for timeout using the timeout from the dom (details)
78. adjusted definition label (details)
79. add unit-test that shows wrong fn:delete() behavior (details)
80. Fix fn:delete() behavior in placeholder expressions. An expression that resolves to [deleted] does not result in the whole expression to be [deleted] (e.g. {{ header:name }}:{{ fn:delete() }} resolves to "value:" instead of [deleted] (with header name:value)). (details)
81. fix failing tests (details)
82. Fix in documentation about placeholder functions. (details)
83. update used MongoDB vesion to 4.4 (details)
84. Replace ClientActorRefs by ConnectionPubSub for consistency during coordinated shutdown. (details)
85. extend AbstractPersistenceOperationsActor with getActorName method; (details)
86. add ditto-internal-utils-akka as test dependency to pom.xml (details)
87. Restore distinction between inbound and outbound signals after going through ConnectionPubSub. (details)
88. save purgeNamespace and purgeEntities commands in a map while processing and re-publish them in case the actor gets shutdown; (details)
89. terminate persistence cleanup stream when graceful shutdown is triggered; (details)
90. Clone bytebuffer in raw message mapper to prevent ReadOnlyBufferException (details)
91. Filter MQTT messages where the message topic does not match the source topic (details)
92. Adjust the default sessionExpiryInterval, so retained and in flight QOS messages get delivered after reconnect (details)
93. Introduced package 'mqtt' for connectivity service model. (details)
94. Removed unused `ReceiveMaximum` class. (details)
95. Introduced dedicated type for session expiry interval. (details)
96. Fixed error in unit test. (details)
97. Removed unused static factory methods from `SessionExpiryInterval`. (details)
98. Add connections Route HTTP API (details)
99. Connections route devOps authentication added (details)
100. changed signature of Policy.isSemanticallySameAs to check for another Policy (details)
101. Retrieve all connections Command impl (details)
102. Retrieve all connections support (details)
103. Retrieve only connectionIds support with optional parameter (details)
104. Review connections API (details)
105. Fix tests (details)
106. Fix ConnectionIdRetrievalActor init (details)
107. Send  connection commands to EdgeCommandForwarder (details)
108. Add exception for invalid amount of connection-ids for RetrieveConnections (details)
109. Test ConnectionsRoute (details)
110. Review (details)
111. Decoupe ConnectionsRoute from authorization (details)
112. Test RootRoute devOps Connections Route (details)
113. Ditto connections Swager API (details)
114. Connections retrieval actor timeout (details)
115. Copyright headers added and minor fixes (details)
116. Connections Retrieval actor refactor. Fix bug in connections route path (details)
117. review: simplified ConnectionsRoute usage in RootRoute (details)
118. configured nginx to pass through /api/2/connections requests to Ditto without applying nginx authentication (same as /devops route) (details)
119. Removed timeout from RetrieveConnections and use from dittoHeaders and minor code review fixes. (details)
120. ConnectionsRetrievalActor-s common logic to abstract class (details)
121. ids-only param to ConnectionsParameter and openapi (details)
122. Update documentation with /connections endpoint (details)
123. DevOps auth to connections route only (details)
124. Apply suggestions from code review (details)
125. Update documentation review fixes (details)
126. review of docs: (details)
127. fixed RootRoute dispatching of "/connections" route and other routes (details)
128. Fix copyright headers (details)
129. Fix rebase conflicts (details)
130. add shutdown behaviour to SnapshotStreamingActor and terminate stream in PhaseServiceRequestsDone; (details)
131. removed Ditto version 1.x from docs dropdown (details)
132. fix BackgroundSyncActorTest; (details)
133. moved actor name of SnapshotStreamingActor into the actor itself; (details)
134. made Cancellable nullable and initialize it with null to fix sonar issue; (details)
135. subscribe via PubSub for RetrieveThings and SudoRetrieveThings in ThingsAggregatorActor and publish commands in ThingsAggregatorProxyActor; (details)
136. set logger akka.actor.CoordinatedShutdown to INFO instead DEBUG; (details)
137. updated swagger-ui to 4.14.1 (details)
138. adjusted latest additions to Ditto 3.0 and updated the potential release date (details)
139. adjust log levels of BaseClientActor and AmqpConsumerActor (details)
140. remove HealthSignal from BaseClientActor because it isn't needed anymore; (details)
141. added "Direction" column to Ditto Explorer UI - Connection Metrics (details)
142. remove putting searchactor ref to pubSubMediator because searchActor registeres itself via pubSub; (details)
143. increased ditto.http.coordinated-shutdown-timeout to 65s in ditto-http.conf; (details)
144. fixed info log for successfully unsubscribing from pubSub; (details)
145. add searchActor to pubSub in SearchRootActor; (details)
146. handle SERVICE_REQUESTS_DONE inAnyState() of PersistenceCleanupActor; (details)
147. use Control.SERVICE_REQUESTS_DONE of PersistenceCleanupActor; (details)
148. make it possible to put "extension.conf" files in Ditto docker container file-system in order to apply configuration overwrites (details)
149. added path /opt/ditto/extensions/* to CLASSPATH of Ditto docker images (details)
150. first draft of payload mapper extension (details)
151. adjusted MessageMapper loading mechanism via DittoExtensionPoint to: (details)
152. fixed issues of "getOrCreateInstance()" approach and always create a new instance, copying all fields of the existing instance (via copy constructor approach) (details)
153. fixed unit test using wrong Mapper Alias (details)
154. simplified ImplicitThingCreationMessageMapper by not using BiFunction resolver construct, but by just creating a new ExpressionResolver for each message (details)
155. adjust CloudEventsMapper to new MessageMapper DittoExtensionPoint adjustment (details)
156. configure "add-opens" JVM command args when building Ditto docker connectivity images (details)
157. enhanced/adjusted documentation on how to provide a custom, Java based payload mapper into Ditto's connectivity service (details)
158. fixed /api/2/connections devOps basic authentication in RootRoute (details)
159. added missing "ditto-sudo" header for devops connections route commands (details)
160. fixed nginx.conf files which did reverse proxying wrongly for /api/2/connections and others (details)
161. re-enable Debug logs for akka.actor.CoordinatedShutdown in things; (details)
162. extend logging; (details)
163. extracted an abstract class AbstractActorWithShutdownBehavior which handles the shutdown behaviour of Actors; (details)
164. also persist "revision" in journal entries to make it possible to read when replaying events (details)
165. post-release 3.0.0 commit: (details)
166. add parameter to include loggers set to "off" (details)
167. Stop consuming and unsubscribe from PubSub on Service-Unbind. (details)
168. fixed remaining mentions of "java -jar" usage to start containers (details)
169. Stop client actors at the end of tests; do not wait for graceful shutdown in client actor tests. (details)
170. Test stopping consumers during graceful shutdown. (details)
171. Use correct key for setting correlation ID to tags. (details)
172. Use correct key for setting correlation ID to tags. (details)
173. Fix review finding (details)
174. Wait for active acknowledgement aggregator actors for a short time during coordinated shutdown. (details)
175. Bump scala-library from 2.13.8 to 2.13.9 in /bom (details)
176. Omit initial WebSocket request "/ws/2" from tracing in `RequestTracingDirective`. (details)
177. Explorer UI: URL parameters for environments (details)
178. add new class AbstractActorWithShutdownBehaviorAndRequestCounting which implements shutdown and request counting behavior; (details)
179. ignored flaky unit test again (details)
180. Move client actors to their own shard region. (details)
181. Remove headers as a part of client actor props to prevent accidental restarts. (details)
182. Use inbound AMQP 1.0 message annotations (details)
183. Set message annotations for outbound AMQP messages (details)
184. Add documentation about AMQP message annotation header mapping (details)
185. Prefix message annotations (details)
186. Fix client shard region name; reintroduce CLOSE_CONNECTION stage to restore connection connection announcements. (details)
187. ignored flaky unit test again (details)
188. fix that connection "remember-entities" based automatic start of "known" connections did no longer work (details)
189. configure remember-entities-store = "eventsourced" (details)
190. Accept and test TextMessage with null payload (details)
191. Fix close-connection behavior of ClientSupervisor; fix ConnectionPersistenceActorTest. (details)
192. Add hand-off messages to connection and client shard regions. (details)
193. Replaced absolute css units in explorer UI (details)
194. Move client actor shutdown to shard region shutdown phase to prevent restarts during coordinated shutdown. (details)
195. Tolerate lower case connection status. (details)
196. Explorer UI: Set URL param for environment (details)
197. New documentation page for ui (details)
198. Changed Ditto version in UI environment to enum (details)
199. Added screenshot to UI documentation (details)
200. fixed adding additional tags for ditto-ui docker image (details)
201. fixed docker-ui rc publishing (details)
202. added workflow for pushing docker images on demand (details)
203. adjusted push-dockerhub-on-demand.yml script to include choice option for which image to push (details)
204. Replace consistent hashable envelope by a dedicated binary message envelope for communication between the connection persistence actor and its client actors. (details)
205. OpenAPI docs: fixed "connectionStatus" type of "Connection" in OpenAPI definition (details)
206. update swagger-ui fixing security issues (details)
207. Add serializer for ClientActorPropsArgs. (details)
208. Update documentation/src/main/resources/pages/ditto/user-interface.md (details)
209. Update documentation/src/main/resources/pages/ditto/user-interface.md (details)
210. Update documentation/src/main/resources/pages/ditto/user-interface.md (details)
211. UI Documentation: feedback from review (details)
212. nullLiteral() and @Nullable (details)
213. added slides about Ditto + WoT (details)
214. bump scala.full.version to 2.13.10, kafka-client.version to 3.0.2 and netty-bom.version to 4.1.82.Final; (details)
215. made presentation better printable (details)
216. added Ditto/WoT presentation to presentations.md (details)
217. updates to the "What, why and how" of the WoT integration (details)
218. extend timeout for test to 5 sec; (details)
219. last polishing of WoT presentation (details)
220. increase timeout for test to 15 sec; (details)
221. Add vaildation of condition header for live messages (details)
222. Adjust documentation for supported live message conditions (details)
223. Only increase timeout for flaky test (details)
224. Make ShardedBinaryEnvelope self-delimiting; log failed serialization. (details)
225. Fix endian-ness of serializer; fix serialization of SudoRetrieveConnectionStatusResponse. (details)
226. Refactored `DittoTracing` to slim down its API and get rid of direct dependencies to Kamon. (details)
227. Disable automatic passivation for shard region creator; start blocked namespaces replicator in connectivity. (details)
228. Delete SearchActorTest#terminateStreams; log blocked namespaces replicator. (details)
229. Add logger for connection validation exception (details)
230. `ActorSystemResource` now can receive an actor system name. (details)
231. feat: add possibility to configure audience in oauth client credentials (details)
232. Fixed but in `UnmodifiableExternalMessage`. (details)
233. Fixed failing unit tests because of uninitialized DittoTracing. (details)
234. Fixed handling of possible duplicate header keys to default behaviour of `Map`. (details)
235. Added builder for `TracingConfig` to `DittoTracingInitResource` to create a config directly at instantiation. (details)
236. Added unit tests for `RequestTracingDirective`. (details)
237. Ensured that `DittoTracing#init` could not be called multiple times. (details)
238. Renamed types from '*Trace' to '*Span' where appropriate. (details)
239. Fixed wrong expectation in unit test. (details)
240. Wait for pending issued acknowledgements during graceful shutdown. (details)
241. Fix incorrect envelope in ClientSupervisor. (details)
242. Make acknowledgement forwarder actor notify outbound dispatching actor when acknowledgements are forwarded. (details)
243. Revert "Make acknowledgement forwarder actor notify outbound dispatching actor when acknowledgements are forwarded." (details)
244. Revert "Wait for pending issued acknowledgements during graceful shutdown." (details)
245. Reworked tagging to be more correct. (details)
246. Fixed issue which was uncovered by unit tests. (details)
247. Extended trace context propagation to all instances of `DittoHeadersSettable`. (details)
248. added slides for EclipseCon 2022 Community Day (details)
249. fixed link (details)
250. Updated slides (details)
251. Allow to use namespaces in search in Explorer UI (details)
252. Use different library for subnet validation (Apache didn't work with IPv6) (details)
253. fix year in copyright header; (details)
254. Fixed handling of already existing W3C tracing headers within RequestTracingDirective. (details)
255. Include `KamonTracingInitResource` in test-jar to make it available for other module's unit tests. (details)
256. Invoke get segment tag only if logger has tracing enabled. (details)
257. Introduced configuration option to include or exclude operation names to or from tracing. (details)
258. Fixed test failure and added unit test for filtering in `DittoTracing`. (details)
259. Rely on configured filter to exclude WebSocket initialisation from tracing. (details)
260. Revert "Add logger for connection validation exception" (details)
261. Review changes (details)
262. review: fixed some minor findings and adjusted code style; (details)
263. Fix failing test (details)
264. Add type name to serialize and deserialize trace span operation name if possible. (details)
265. Fixed glob pattern of includes to accept all operation names regardless of any slashes. (details)
266. Ensure that W3C tracing headers are always set to request. (details)
267. adjusted URL of github pages after Ditto GitHub organization move (details)
268. Renamed `fail` methods of `StartedSpan` to `tagAsFailed`. (details)
269. Added test resource for Kamon test span reporters. (details)
270. Made static factory method of `KamonHttpContextPropagation` less error-prone. (details)
271. Add blog post for live message conditions (details)
272. Minor code review fixes (details)
273. Explorer UI json payload for messages (details)
274. Introduced result type for Java. (details)
275. Replaced Scala `Try` with Ditto's new `Result`. (details)
276. Made instance creation of `KamonHttpContextPropagation` even more fluent. (details)
277. Fixed year of license header. (details)
278. Adjusted source version of Javadoc plugin. (details)
279. Blog post redacted (details)
280. fix blogpost code-snippets to render correctly in Jekyll (details)
281. Improved search filter in explorer ui (details)
282. Explorer UI: fixed bug caused by filter dropdown (details)
283. Implement SubnetValidator for blocked subnet matching (details)
284. Add ClassRule for DittoTracingInitResource to disable tracing in the test; (details)
285. Fix PolicySupervisorActorTest. (details)
286. Fix SearchActorTest. (details)
287. fix installation-operating.md typo (details)
288. fixed wrong span key string for SpanTagKey.REQUEST_URI (details)
289. Turn on debug log for PolicySupervisorActorTest. (details)
290. Fix license header; fix PolicySupervisorActorTest. (details)
291. removed mention of Bosch IoT Things as "free plan" sandbox from docs (details)
292. update star chart in Readme (details)
293. cancelReceiveTimeout moved after exception creating (details)
294. removed deprecated TLS versions from sandbox nginx config (details)
295. added security enhancing headers to nginx.conf of sandbox (details)
296. fix k8s descritor file, removing "java", "-jar", "starter.jar" (details)
297. Fix unserializable restart message in ClientSupervisor. (details)
298. Connections API doc fixes (details)
299. credentials in connections are encrypted when written to the db (details)
300. dependency updates: (details)
301. configured content-security-policy correctly for sandbox (details)
302. fixed content-security-policy for sandbox (details)
303. ClientSupervisor: Retrieve connection status on startup to deal with shard rebalancing. (details)
304. Fix metrics "connection_client" and "connecting_client" underreporting due to multiple client actors of a connection starting on the same instance. (details)
305. Increase relative rebalance limit to coincide with absolute limit when all shards are active. (details)
306. directly close&release OSS nexus repositories on Ditto release (details)
307. Code review fixes (details)
308. extend documentation for credentials encryption (details)
309. connectivity-manage-connections-piggyback added to ditto_sidebar (details)
310. Documentation links minor fixes. (details)
311. prepare Ditto 3.1.0 (details)
312. added doc version 3.1 to _config.yml (details)
313. fixes wrong asString call instead of formatAsString (details)
314. prepare 3.1.0 release, continued (details)

Add creator of shard regions with a custom hand-off message.

Signed-off-by: Yufei Cai <yufei.cai@bosch.io>

Make ThingUpdater participate in shard hand-off; test self restart of sharded actors.

Signed-off-by: Yufei Cai <yufei.cai@bosch.io>

Make StopShardedActor java-serializable for tests; restructure ShardRegionCreatorTest.

Signed-off-by: Yufei Cai <yufei.cai@bosch.io>

Logging and validation

Signed-off-by: Pranshu-G <pranshu.grover18@gmail.com>

Payload Validation Test

Signed-off-by: Pranshu-G <pranshu.grover18@gmail.com>

Mapping incoming CloudEvents with Ditto Protocol in Data

Signed-off-by: Pranshu-G <pranshu.grover18@gmail.com>

re-committing CloudEvents mapper

Signed-off-by: Pranshu-G <pranshu.grover18@gmail.com>

reformat

Signed-off-by: Pranshu-G <pranshu.grover18@gmail.com>

remove main function from mapper

Signed-off-by: Pranshu-G <pranshu.grover18@gmail.com>

Remove logging statements

Signed-off-by: Pranshu-G <pranshu.grover18@gmail.com>

change license header year

Signed-off-by: Pranshu-G <pranshu.grover18@gmail.com>

remove Tests

Signed-off-by: Pranshu-G <pranshu.grover18@gmail.com>

Add mapping Tests

Signed-off-by: Pranshu-G <pranshu.grover18@gmail.com>

More Tests

Signed-off-by: Pranshu-G <pranshu.grover18@gmail.com>

fix build

Signed-off-by: Pranshu-G <pranshu.grover18@gmail.com>

validate headers

Signed-off-by: Pranshu-G <pranshu.grover18@gmail.com>

Integrate SearchActor into coordinated shutdown.

Signed-off-by: Yufei Cai <yufei.cai@bosch.io>

Initial Binary CE support

Signed-off-by: Pranshu-G <pranshu.grover18@gmail.com>

formatting and minor changes

Signed-off-by: Pranshu-G <pranshu.grover18@gmail.com>

fix build

Signed-off-by: Pranshu-G <pranshu.grover18@gmail.com>

[wip] send thing events directly to search updater shard region.

Signed-off-by: Yufei Cai <yufei.cai@bosch.io>

Replace Ditto pubsub by direct shard messaging for search update.

Reason: Ditto pubsub reacts too slowly for coordinated shutdown.

Signed-off-by: Yufei Cai <yufei.cai@bosch.io>

Add signals to things registry tests.

Signed-off-by: Yufei Cai <yufei.cai@bosch.io>

Integrate policies into Akka coordinated shutdown.

Signed-off-by: Yufei Cai <yufei.cai@bosch.io>

Add PolicySupervisorActorTest; fix unit tests.

Signed-off-by: Yufei Cai <yufei.cai@bosch.io>

add private constructor for ShardRegionCreator;

Signed-off-by: Stefan Maute <stefan.maute@bosch.io>

Placeholder Outbound Mapping

Signed-off-by: Pranshu-G <pranshu.grover18@gmail.com>

minor formatting changes;

Signed-off-by: Stefan Maute <stefan.maute@bosch.io>

build exception and content-type check

Signed-off-by: Pranshu-G <pranshu.grover18@gmail.com>

change content-type for tests

Signed-off-by: Pranshu-G <pranshu.grover18@gmail.com>

ExternalMessaqge building with Json

Signed-off-by: Pranshu-G <pranshu.grover18@gmail.com>

Review: minor code formatting change

Signed-off-by: Stefan Maute <stefan.maute@bosch.io>

binary CE test

Signed-off-by: Pranshu-G <pranshu.grover18@gmail.com>

Integrate Things into Akka coordinated shutdown.

Signed-off-by: Yufei Cai <yufei.cai@bosch.io>

Documentation

Signed-off-by: Pranshu-G <pranshu.grover18@gmail.com>

Also terminate twin operations on coordinated shutdown after timeout.

Reason: Twin operations can include acknowledgement requests.

Signed-off-by: Yufei Cai <yufei.cai@bosch.io>

add documentation plus remove blocklist

Signed-off-by: Pranshu-G <pranshu.grover18@gmail.com>

add Test plus minor housekeeping

Signed-off-by: Pranshu-G <pranshu.grover18@gmail.com>

remove thing-event-aware from test config;

Signed-off-by: Stefan Maute <stefan.maute@bosch.io>

add remove method to DistPubSubAccess in order to remove an actor from pubSubMediator;

Signed-off-by: Stefan Maute <stefan.maute@bosch.io>

reformatting

Signed-off-by: Pranshu-G <pranshu.grover18@gmail.com>

remove InboundDispatchSink from PR

Signed-off-by: Pranshu-G <pranshu.grover18@gmail.com>

reformat InboundDispatchSink

Signed-off-by: Pranshu-G <pranshu.grover18@gmail.com>

remove comments and other necessary changes

Signed-off-by: Pranshu-G <pranshu.grover18@gmail.com>

minor reformatting

Signed-off-by: Pranshu-G <pranshu.grover18@gmail.com>

exception handling

Signed-off-by: Pranshu-G <pranshu.grover18@gmail.com>

reformat docs

Signed-off-by: Pranshu-G <pranshu.grover18@gmail.com>

allow filtering a single background sync iteration to a set of namespaces (e.g. to forcefully re-index all things within a given namespace)

Signed-off-by: Dominik Guggemos <dominik.guggemos@bosch.io>

exception handling

Signed-off-by: Pranshu-G <pranshu.grover18@gmail.com>

made second start method of ShardRegionCreator also public;

Signed-off-by: Stefan Maute <stefan.maute@bosch.io>

Update documentation/src/main/resources/pages/ditto/connectivity-mapping.md

Co-authored-by: Thomas Jaeckle <thomas.jaeckle@bosch.io>

Update documentation/src/main/resources/pages/ditto/connectivity-mapping.md

Co-authored-by: Thomas Jaeckle <thomas.jaeckle@bosch.io>

extend logging for unsubscribing from pubsub;

Signed-off-by: Stefan Maute <stefan.maute@bosch.io>

Binary CE supports content type application/vnd.eclipse.ditto+json

Signed-off-by: Pranshu-G <pranshu.grover18@gmail.com>

fix build

Signed-off-by: Pranshu-G <pranshu.grover18@gmail.com>

Filter MQTT messages where the message topic does not match the source topic

Since now we consumed all messages that a client is subscribed to on all sources (not respecting the sources adresses)
The HiveMq client however makes it pretty hard to only consume messages which match a single subscribes topic (subscribePublished)
Since this would require major refactorings of the MQTT client connectivity, the messages not matching the sources adresses will now just be filtered out in the consumer actor.

Signed-off-by: David Schwilk <david.schwilk@bosch.io>

move creation of list of topic filters out of message stream

Signed-off-by: Dominik Guggemos <dominik.guggemos@bosch.io>

adjust method name and logging;
fixed javadoc and typo;

Signed-off-by: Stefan Maute <stefan.maute@bosch.io>

Outbound spec changes and extract Id from correlation-id

Signed-off-by: Pranshu-G <pranshu.grover18@gmail.com>

Adjust the command response enforcement exception handling

Currently, failed policy enforcer loading leads to error logging and 500 exceptions. This case happens when the corresponding policy is deleted before command response enforcement. Thus adjusted the error to 404 and removed error logging.

Additionally, the command response exceptions always get wrapped in an internal exception. Adjusted that exceptions which are already DRE are thrown as is.

Signed-off-by: David Schwilk <david.schwilk@bosch.io>

increased coordinated-shutdown-timeout to 65s to give the requests enough time to complete;
removed match for DistributedPubSubMediator.SubscribeAck from GatewayProxyActor and EdgeCommandForwarderActor because no subscribe message is send via pudSubMediator;
wait with stopping the AbstractHttpRequestActor until the request is completed;
add test to HttpRequestActorTest;

Signed-off-by: Stefan Maute <stefan.maute@bosch.io>

applied Ditto code formatter to CloudEventsMapper

Signed-off-by: Thomas Jaeckle <thomas.jaeckle@bosch.io>

made coordinated shutdown task in AbstractHttpRequestActor cancellable and remove task when actor is shutdown;
wait for requests in QueryThingsPerRequestActor;
pass httpConfig to GatewayProxyActor and QueryThingsPerRequestActor;
add test to QueryThingsPerRequestActorTest,

Signed-off-by: Stefan Maute <stefan.maute@bosch.io>

add new exception GatewayWebsocketSessionAbortedException which is used for terminating the websocket session;
extend Connect with an optional killSwitch;
shutdown websocket streams in service requests done phase of CoordinatedShutdown;

Signed-off-by: Stefan Maute <stefan.maute@bosch.io>

cleanup unused variable;
rename method;

Signed-off-by: Stefan Maute <stefan.maute@bosch.io>

Throw ThingNotAccessibleException when thing is deleted and policy not available

Signed-off-by: Yannic Klem <yannic.klem@bosch.io>

Extract group-index-key from Ditto PubSub so that it is not restricted to signals with entity ID.

Signed-off-by: Yufei Cai <yufei.cai@bosch.io>

fix unittest

Signed-off-by: Yannic Klem <yannic.klem@bosch.io>

adjusted Ditto explorer UI to use new HTTP routes for /connections
* also added an example template using a JS payload mapper

Signed-off-by: Thomas Jaeckle <thomas.jaeckle@bosch.io>

fix intendation in connections.js

Co-authored-by: Thomas Fries <thomas.fries0@gmail.com>

added input field for selecting "Ditto version" in environments tab

Signed-off-by: Thomas Jaeckle <thomas.jaeckle@bosch.io>

provide announcement blogpost for Ditto 3.0 release
* also includes some fixes to OpenAPI

Signed-off-by: Thomas Jaeckle <thomas.jaeckle@bosch.io>

fixed UnsupportedOperationException in DefaultWoTThingDescriptionGenerator

Signed-off-by: Thomas Jaeckle <thomas.jaeckle@bosch.io>

some small fixes for usability and performance improvements:
* replaced requesting "_policy" with requesting "policyId"
* made it possible to define an "own" thingId when creating things, using "If-Match" and "If-None-Match" headers
* added new definition template for an example WoT ThingModel

Signed-off-by: Thomas Jaeckle <thomas.jaeckle@bosch.io>

adjusted input type from "integer" to "number"

made StopShardedActor an implementation of AkkaJacksonCborSerializable because java serialization is disabled;

Signed-off-by: Stefan Maute <stefan.maute@bosch.io>

fixed check for timeout using the timeout from the dom

Signed-off-by: Thomas Jaeckle <thomas.jaeckle@bosch.io>

adjusted definition label

add unit-test that shows wrong fn:delete() behavior

Signed-off-by: Dominik Guggemos <dominik.guggemos@bosch.io>
Co-authored-by: Andrey Balarev <andrey.balarev@bosch.io>

Fix fn:delete() behavior in placeholder expressions. An expression that resolves to [deleted] does not result in the whole expression to be [deleted] (e.g. {{ header:name }}:{{ fn:delete() }} resolves to "value:" instead of [deleted] (with header name:value)).

Signed-off-by: Dominik Guggemos <dominik.guggemos@bosch.io>
Co-authored-by: Andrey Balarev <andrey.balarev@bosch.io>

fix failing tests

Signed-off-by: Dominik Guggemos <dominik.guggemos@bosch.io>
Co-authored-by: Andrey Balarev <andrey.balarev@bosch.io>

Fix in documentation about placeholder functions.

Signed-off-by: Andrey Balarev <andrey.balarev@bosch.io>

update used MongoDB vesion to 4.4

Signed-off-by: Thomas Jaeckle <thomas.jaeckle@bosch.io>

Replace ClientActorRefs by ConnectionPubSub for consistency during coordinated shutdown.

Signed-off-by: Yufei Cai <yufei.cai@bosch.io>

extend AbstractPersistenceOperationsActor with getActorName method;
add graceful shutdown behaviour to AbstractPersistenceOperationsActor;
add test for shutdown behaviour to PolicyPersistenceOperationsActorIT;

Signed-off-by: Stefan Maute <stefan.maute@bosch.io>

add ditto-internal-utils-akka as test dependency to pom.xml

Signed-off-by: Stefan Maute <stefan.maute@bosch.io>

Restore distinction between inbound and outbound signals after going through ConnectionPubSub.

Signed-off-by: Yufei Cai <yufei.cai@bosch.io>

save purgeNamespace and purgeEntities commands in a map while processing and re-publish them in case the actor gets shutdown;
adapt PolicyPersistenceOperationsActorIT;

Signed-off-by: Stefan Maute <stefan.maute@bosch.io>

terminate persistence cleanup stream when graceful shutdown is triggered;
add test to PersistenceCleanupActor;

Signed-off-by: Stefan Maute <stefan.maute@bosch.io>

Clone bytebuffer in raw message mapper to prevent ReadOnlyBufferException

Encountered while testing raw message mapping for MQTT 5, where we get read only buffers

Signed-off-by: David Schwilk <david.schwilk@bosch.io>

Filter MQTT messages where the message topic does not match the source topic

Since now we consumed all messages that a client is subscribed to on all sources (not respecting the sources adresses)
The HiveMq client however makes it pretty hard to only consume messages which match a single subscribes topic (subscribePublished)
Since this would require major refactorings of the MQTT client connectivity, the messages not matching the sources adresses will now just be filtered out in the consumer actor.

Signed-off-by: David Schwilk <david.schwilk@bosch.io>

Adjust the default sessionExpiryInterval, so retained and in flight QOS messages get delivered after reconnect

Signed-off-by: David Schwilk <david.schwilk@bosch.io>

Introduced package 'mqtt' for connectivity service model.

Moved `ReceiveMaximum` and `IllegalReceiveMaximumValueException` to this new package.
More types will follow.

Signed-off-by: Juergen Fickel <juergen.fickel@bosch.io>

Removed unused `ReceiveMaximum` class.

This fixes a compilation error, too.

Signed-off-by: Juergen Fickel <juergen.fickel@bosch.io>

Introduced dedicated type for session expiry interval.

This makes sure that only a valid session expiry interval can be used in the service.

Signed-off-by: Juergen Fickel <juergen.fickel@bosch.io>

Fixed error in unit test.

Signed-off-by: Juergen Fickel <juergen.fickel@bosch.io>

Removed unused static factory methods from `SessionExpiryInterval`.

Signed-off-by: Juergen Fickel <juergen.fickel@bosch.io>

Add connections Route HTTP API

set binary-compatibility-check.version to release version

Signed-off-by: Stanchev Aleksandar <aleksandar.stanchev@bosch.io>

Connections route devOps authentication added

Signed-off-by: Stanchev Aleksandar <Aleksandar.Stanchev@bosch.io>

changed signature of Policy.isSemanticallySameAs to check for another Policy

Signed-off-by: Thomas Jaeckle <thomas.jaeckle@bosch.io>

Retrieve all connections Command impl

Signed-off-by: Stanchev Aleksandar <Aleksandar.Stanchev@bosch.io>

Retrieve all connections support

Signed-off-by: Stanchev Aleksandar <Aleksandar.Stanchev@bosch.io>

Retrieve only connectionIds support with optional parameter

Signed-off-by: Stanchev Aleksandar <Aleksandar.Stanchev@bosch.io>

Review connections API

Signed-off-by: David Schwilk <david.schwilk@bosch.io>

Fix tests

Signed-off-by: David Schwilk <david.schwilk@bosch.io>

Fix ConnectionIdRetrievalActor init

Signed-off-by: David Schwilk <david.schwilk@bosch.io>

Send  connection commands to EdgeCommandForwarder

Signed-off-by: Stanchev Aleksandar <Aleksandar.Stanchev@bosch.io>

Add exception for invalid amount of connection-ids for RetrieveConnections

Signed-off-by: David Schwilk <david.schwilk@bosch.io>

Test ConnectionsRoute

Signed-off-by: Stanchev Aleksandar <Aleksandar.Stanchev@bosch.io>

Review

Signed-off-by: David Schwilk <david.schwilk@bosch.io>

Decoupe ConnectionsRoute from authorization

Signed-off-by: Stanchev Aleksandar <aleksandar.stanchev@bosch.io>

Test RootRoute devOps Connections Route

Signed-off-by: Stanchev Aleksandar <aleksandar.stanchev@bosch.io>

Ditto connections Swager API

Signed-off-by: Stanchev Aleksandar <Aleksandar.Stanchev@bosch.io>

Connections retrieval actor timeout

Signed-off-by: Stanchev Aleksandar <aleksandar.stanchev@bosch.io>

Copyright headers added and minor fixes

Signed-off-by: Stanchev Aleksandar <Aleksandar.Stanchev@bosch.io>

Connections Retrieval actor refactor. Fix bug in connections route path

Signed-off-by: Stanchev Aleksandar <aleksandar.stanchev@bosch.io>

review: simplified ConnectionsRoute usage in RootRoute
* moved DevOpsAuthenticationDirective to apply to DevOpsRoute

Signed-off-by: Thomas Jaeckle <thomas.jaeckle@bosch.io>

configured nginx to pass through /api/2/connections requests to Ditto without applying nginx authentication (same as /devops route)

Signed-off-by: Thomas Jaeckle <thomas.jaeckle@bosch.io>

Removed timeout from RetrieveConnections and use from dittoHeaders and minor code review fixes.

Signed-off-by: Stanchev Aleksandar <Aleksandar.Stanchev@bosch.io>

ConnectionsRetrievalActor-s common logic to abstract class

Signed-off-by: Stanchev Aleksandar <Aleksandar.Stanchev@bosch.io>

ids-only param to ConnectionsParameter and openapi

Signed-off-by: Stanchev Aleksandar <Aleksandar.Stanchev@bosch.io>

Update documentation with /connections endpoint

Signed-off-by: Stanchev Aleksandar <aleksandar.stanchev@bosch.io>

DevOps auth to connections route only

Signed-off-by: Stanchev Aleksandar <Aleksandar.Stanchev@bosch.io>

Apply suggestions from code review

Co-authored-by: Jaeckle Thomas <jat1imb@bosch.com>

Signed-off-by: Stanchev Aleksandar <aleksandar.stanchev@bosch.io>

Update documentation review fixes

Signed-off-by: Stanchev Aleksandar <aleksandar.stanchev@bosch.io>

review of docs:
* re-generated ditto-api-2.yml file
* fixed typo "/devps" to "/devops"
* applied some formatting

Signed-off-by: Thomas Jaeckle <thomas.jaeckle@bosch.io>

fixed RootRoute dispatching of "/connections" route and other routes

* the "rootRouteHeadersStepBuilder" was not applied correctly any longer
* the DittoHeaders were extracted wrongly
* use "apiAuthentication()" also for connectionsRoute, by exclude the path from being actually authenticated with the "apiAuthenticationDirective"

Signed-off-by: Thomas Jaeckle <thomas.jaeckle@bosch.io>

Fix copyright headers

Signed-off-by: Stanchev Aleksandar <Aleksandar.Stanchev@bosch.io>

Fix rebase conflicts

Signed-off-by: Stanchev Aleksandar <aleksandar.stanchev@bosch.io>

add shutdown behaviour to SnapshotStreamingActor and terminate stream in PhaseServiceRequestsDone;
add test for shutdown behaviour to SnapshotStreamingActorTest;
change PubSub mechanism from send to publish for SudoStreamSnapshots commands in ThingsMetadataSource and SnapshotStreamingActor;

Signed-off-by: Stefan Maute <stefan.maute@bosch.io>