Commit Graph

  • 9f76fabaed give tokio threads a more convenient name for monitoring purposes James Wilson 2021-08-09 11:36:46 +01:00
  • b22efc804a Fix comment typo James Wilson 2021-08-09 10:56:01 +01:00
  • 626fe95d89 1 aggregator loop by default for now James Wilson 2021-08-09 10:13:10 +01:00
  • c469ef8dfe make AggregatorSet close to zero cost when only 1 aggregator asked for James Wilson 2021-08-09 10:09:17 +01:00
  • c99cbee1e9 Allow multiple aggregator loops in the core to try and spread feed subscription/send cost out James Wilson 2021-08-07 17:08:58 +01:00
  • 9c001bdcfd Add a comment on core worker threads James Wilson 2021-08-06 17:50:50 +01:00
  • 8017d8b00f Cargo fmt James Wilson 2021-08-06 17:49:19 +01:00
  • 78ad7115e5 set sensible worker thread limits again but allow full control to caller James Wilson 2021-08-06 17:49:04 +01:00
  • 88c3db3562 cargo fmt James Wilson 2021-08-06 17:44:26 +01:00
  • 74cf55174e Use rayon to speed up subscribe message serializing James Wilson 2021-08-06 17:42:03 +01:00
  • 0788270756 Add 'subscribe' benchmark James Wilson 2021-08-06 14:22:01 +01:00
  • 2dae6c90ce Keep file handle limit comment in sync James Wilson 2021-08-05 12:22:22 +01:00
  • dbabd453e0 Remove feed message cap; it wasn't pulling its weight James Wilson 2021-08-05 12:21:16 +01:00
  • bc75ebb068 default core/shard worker threads to 8/4 respectively James Wilson 2021-08-05 12:18:44 +01:00
  • 4da299bf76 Tweak logging around connections opening and messages coming in James Wilson 2021-08-05 12:11:46 +01:00
  • acfe3d45d1 num-cpus to worker-threads; more accurate name James Wilson 2021-08-05 11:31:47 +01:00
  • 5c5c7e592a allow configuration of cpus in soak tests James Wilson 2021-08-05 11:16:10 +01:00
  • 759d28b1a2 add --num-cpus option James Wilson 2021-08-05 11:03:53 +01:00
  • 6221cbfd17 Dev tweaks for testing James Wilson 2021-08-04 17:05:00 +01:00
  • 7d00d72baa Small tweak to make it more obvious when feeds are disconnected James Wilson 2021-08-04 10:52:53 +01:00
  • db5284a3db Add a comment explaining realistic_soak_test and --release on all James Wilson 2021-08-03 15:42:42 +01:00
  • 5d81128e74 create more realistic soak test so that we can see a more representative UI in action James Wilson 2021-08-03 15:40:20 +01:00
  • a85e13e0ec Merge pull request #360 from arshamparity/helm arshamparity 2021-08-03 15:29:28 +03:00
  • cf3f323c2e exposing the shard service to the internet Arsham Teymourı 2021-08-03 15:22:09 +03:00
  • f5df1ef68b Merge pull request #356 from arshamparity/helm James Wilson 2021-08-02 14:18:33 +01:00
  • 4b00ac79bb Point sahrd to correct port for core James Wilson 2021-08-02 14:16:20 +01:00
  • 55537af6f5 extending the helm chart Arsham Teymourı 2021-08-02 12:00:42 +03:00
  • c2892286a6 Add a comment James Wilson 2021-08-02 09:54:11 +01:00
  • b79759a412 Tidy up close handling and allow either side to close James Wilson 2021-08-02 09:50:31 +01:00
  • 963ef576f3 add on_close to abstract and simplify ws_client socket closing James Wilson 2021-08-02 09:30:29 +01:00
  • 13afb96f93 Be more fussy with Soketto channel wrapper; Soketto might do otherwise, but we only explicitly close the connection when both ends of the channel handed out are closed (or we encounter a connection error) James Wilson 2021-07-30 21:26:52 +01:00
  • 18918e6d3e undo a previous change; close the connection if either side of the channel handed back is dropped/fails James Wilson 2021-07-30 19:43:26 +01:00
  • a5b04cb4f0 improve socket channel close handling, and test the node banning (roughly) James Wilson 2021-07-30 18:58:10 +01:00
  • 3f523178be Fix broken test (node being banned) and allow test config of new max-bytes option on shard James Wilson 2021-07-30 16:55:06 +01:00
  • a368553085 Add note and reduce max data per second default as we expect much less traffic than existing avg James Wilson 2021-07-30 15:39:30 +01:00
  • 0b1a6f45fa readme tweaks James Wilson 2021-07-30 12:36:23 +01:00
  • 3fa04f6a81 Update readme to bring into line with new shard/core split James Wilson 2021-07-30 12:33:34 +01:00
  • b70ab9ed3d Add license to the top of each source file James Wilson 2021-07-29 17:34:40 +01:00
  • 51e2330777 tweak a log message James Wilson 2021-07-29 08:56:34 +01:00
  • 8a23f342a4 add another test for the rolling total James Wilson 2021-07-29 08:53:13 +01:00
  • aa63e2960c block addresses from reconnecting if they send too much data James Wilson 2021-07-28 16:43:51 +01:00
  • 5f022069db cargo fmt James Wilson 2021-07-28 16:15:34 +01:00
  • 83d31ef0b3 Add rolling total and allow control over bytes per second allowed from node connections James Wilson 2021-07-28 16:08:02 +01:00
  • 4e5815ced9 adding liveness and rediness probes arshamparity 2021-07-28 17:56:24 +03:00
  • a0c434f238 set pipeline to manual arshamparity 2021-07-28 16:37:00 +03:00
  • d46e28e580 change the image tag arshamparity 2021-07-28 16:22:48 +03:00
  • cbca6a0974 change the ci arshamparity 2021-07-28 16:17:39 +03:00
  • fff38c1f38 change affinity rules arshamparity 2021-07-28 15:23:04 +03:00
  • 10647fe033 Merge remote-tracking branch 'origin/jsdw-sharding' into helm arshamparity 2021-07-28 13:22:41 +03:00
  • 99a3c474b9 deploying the shard alongside the core arshamparity 2021-07-28 13:18:28 +03:00
  • 9ec48adcaa Wee tidy up of test code James Wilson 2021-07-27 14:45:45 +01:00
  • 61fd903f55 Remove benchmark things for now and no need for release mode on a test James Wilson 2021-07-27 14:28:27 +01:00
  • 2ca5f6a1fb cargo fmt James Wilson 2021-07-27 12:38:05 +01:00
  • a44f39cfaf Add/test max_nodes_per_connection feature James Wilson 2021-07-27 12:37:32 +01:00
  • 3802c75830 fixing CI arshamparity 2021-07-27 11:39:26 +03:00
  • e7a7a7b237 fixing ci and replicas arshamparity 2021-07-27 10:43:23 +03:00
  • ecf5fccaab make possible to test, test, and fix feed cutoff James Wilson 2021-07-26 16:38:24 +01:00
  • 11972e3dc4 adding helm chart and configuring CI arshamparity 2021-07-26 15:21:27 +03:00
  • 50935b29fc put a deadline on message sending to feeds James Wilson 2021-07-26 11:52:46 +01:00
  • e043de8ddb Expose raw Soketto interface via ws_client for more control where needed (eg in tests) James Wilson 2021-07-26 11:31:40 +01:00
  • cf0e424803 Adapt shard to use hyper+soketto instead James Wilson 2021-07-23 17:07:29 +01:00
  • 480abd1e49 remove println James Wilson 2021-07-23 16:21:30 +01:00
  • 0eff32d10e Migrate telemetry_core to Hyper+Soketto James Wilson 2021-07-23 15:57:50 +01:00
  • 649fb966d2 Improve comment and bias select loop James Wilson 2021-07-21 17:48:18 +01:00
  • cedc2f9dbe tweak a comment James Wilson 2021-07-21 13:59:44 +01:00
  • 999d8ff0bd debounce feed polling a little to reduce CPU load when lots of messages are being sent out James Wilson 2021-07-21 12:06:22 +01:00
  • cca1df2e21 use ws_client in shard, too, and make it better at handling core disconnecting James Wilson 2021-07-20 13:04:10 +01:00
  • 36c4e7b0ef Remove flamegraph bits James Wilson 2021-07-19 17:11:31 +01:00
  • 73a4dbb095 Allow tests to hook up to running process to help with profiling James Wilson 2021-07-19 15:57:39 +01:00
  • f77afe0b75 avoid overquota in soak tests James Wilson 2021-07-16 18:07:10 +01:00
  • 582c49413b Make server able to run old or new binary so that we can compare the actix version with this one James Wilson 2021-07-16 17:58:26 +01:00
  • 81a0140c3b bump wait-time for port to be ready; build-host is slower to compile than my laptop.. James Wilson 2021-07-16 10:03:38 +01:00
  • fd79b3e85b Add some tests, and add a configurable soak test James Wilson 2021-07-15 17:18:58 +01:00
  • db8ea9a8f3 Add more tests, fix (and test for) a deadlock re overquota messages, more unbounded channels and less .awaiting James Wilson 2021-07-15 10:53:02 +01:00
  • 86a3edf053 increase wait-for-port timeout to accomodate for --release mode and slower machines James Wilson 2021-07-14 15:14:47 +01:00
  • e34da5761f move tests around and add (non working) benchmark James Wilson 2021-07-14 15:05:09 +01:00
  • c1208b9e81 cargo fmt James Wilson 2021-07-13 16:51:24 +01:00
  • 9ac5ea7624 Various bits and pieces to address PR comments James Wilson 2021-07-13 16:51:14 +01:00
  • faedba87d4 no e2e feature, and don't assume 'cargo run' is a good default James Wilson 2021-07-13 11:12:23 +01:00
  • 66bf24dc4d re-add e2e (again..) James Wilson 2021-07-13 09:25:54 +01:00
  • 74d5890ea0 Wee tidy up James Wilson 2021-07-13 09:21:43 +01:00
  • 61d6b620b6 Add small comment to make diff more obvious James Wilson 2021-07-13 08:57:29 +01:00
  • cfe7ff39bb contains_matches macro and more tests James Wilson 2021-07-12 17:21:01 +01:00
  • c6c262c9c5 tidyup, add more e2e tests, fix (and test a little) feed msg decoding, supporting bits James Wilson 2021-07-12 13:17:24 +01:00
  • f2f122285e make it obvious that unbounded channels don't need to await, and tidy up some bits James Wilson 2021-07-12 10:01:23 +01:00
  • 4046edc47d Merge branch 'jsdw-sharding-test-runner' into jsdw-sharding James Wilson 2021-07-09 21:23:38 +01:00
  • 65178de82f Just one dockerfile to simplify things a little and reduce repetition James Wilson 2021-07-09 20:57:39 +01:00
  • 28be68e65f Get a basic first test working, and lots of changes to supporting code to facilitate this James Wilson 2021-07-09 17:27:49 +01:00
  • c043393e28 Write some basic test utils to get going; time to start trying them James Wilson 2021-07-07 17:25:59 +01:00
  • f2adead2e9 rename binaries for clarity, and first pass of connect_to_servers test util James Wilson 2021-07-07 12:49:03 +01:00
  • 8bf412cad9 Begin writing the core utils that we'll use for testing things James Wilson 2021-07-06 16:18:21 +01:00
  • 11c0db389a Get docker-compose working again James Wilson 2021-07-09 19:40:52 +01:00
  • ea52d36999 make sure to gracefully handle 'close' messages James Wilson 2021-07-05 15:11:20 +01:00
  • 6910b4dca4 typo James Wilson 2021-07-05 10:45:04 +01:00
  • 750aaa06dc Tweak docker(compose) files (can't test them yet) and add a little backend documentation James Wilson 2021-07-05 10:26:32 +01:00
  • ea98b15def No need for LogLevel James Wilson 2021-07-01 15:59:41 +01:00
  • 6e4756ecc0 also add back telemetry license James Wilson 2021-07-01 15:13:59 +01:00
  • 3c371ef9a8 add license back to Cargo.toml files James Wilson 2021-07-01 15:12:39 +01:00
  • 4bb1f19085 add some doc comments James Wilson 2021-07-01 09:58:18 +01:00
  • 5d00445324 Make syntax of id_type identical to normal tuple struct James Wilson 2021-07-01 09:38:53 +01:00