 James Wilson
|
759d28b1a2
|
add --num-cpus option
|
2021-08-05 11:11:23 +01:00 |
|
|
James Wilson
|
6221cbfd17
|
Dev tweaks for testing
|
2021-08-05 11:11:05 +01:00 |
|
|
James Wilson
|
7d00d72baa
|
Small tweak to make it more obvious when feeds are disconnected
|
2021-08-04 10:52:53 +01:00 |
|
|
James Wilson
|
db5284a3db
|
Add a comment explaining realistic_soak_test and --release on all
|
2021-08-03 15:42:42 +01:00 |
|
|
James Wilson
|
5d81128e74
|
create more realistic soak test so that we can see a more representative UI in action
|
2021-08-03 15:40:20 +01:00 |
|
|
James Wilson
|
f5df1ef68b
|
Merge pull request #356 from arshamparity/helm
Helm
|
2021-08-02 14:18:33 +01:00 |
|
|
James Wilson
|
4b00ac79bb
|
Point sahrd to correct port for core
|
2021-08-02 14:16:20 +01:00 |
|
|
James Wilson
|
c2892286a6
|
Add a comment
|
2021-08-02 09:54:11 +01:00 |
|
|
James Wilson
|
b79759a412
|
Tidy up close handling and allow either side to close
|
2021-08-02 09:50:31 +01:00 |
|
|
James Wilson
|
963ef576f3
|
add on_close to abstract and simplify ws_client socket closing
|
2021-08-02 09:30:29 +01:00 |
|
|
James Wilson
|
13afb96f93
|
Be more fussy with Soketto channel wrapper; Soketto might do otherwise, but we only explicitly close the connection when both ends of the channel handed out are closed (or we encounter a connection error)
|
2021-07-30 21:26:52 +01:00 |
|
|
James Wilson
|
18918e6d3e
|
undo a previous change; close the connection if either side of the channel handed back is dropped/fails
|
2021-07-30 19:43:26 +01:00 |
|
|
James Wilson
|
a5b04cb4f0
|
improve socket channel close handling, and test the node banning (roughly)
|
2021-07-30 18:58:10 +01:00 |
|
|
James Wilson
|
3f523178be
|
Fix broken test (node being banned) and allow test config of new max-bytes option on shard
|
2021-07-30 16:55:06 +01:00 |
|
|
James Wilson
|
a368553085
|
Add note and reduce max data per second default as we expect much less traffic than existing avg
|
2021-07-30 15:39:30 +01:00 |
|
|
James Wilson
|
0b1a6f45fa
|
readme tweaks
|
2021-07-30 12:36:23 +01:00 |
|
|
James Wilson
|
3fa04f6a81
|
Update readme to bring into line with new shard/core split
|
2021-07-30 12:33:34 +01:00 |
|
|
James Wilson
|
b70ab9ed3d
|
Add license to the top of each source file
|
2021-07-29 17:34:40 +01:00 |
|
|
James Wilson
|
51e2330777
|
tweak a log message
|
2021-07-29 08:56:34 +01:00 |
|
|
James Wilson
|
8a23f342a4
|
add another test for the rolling total
|
2021-07-29 08:53:13 +01:00 |
|
|
James Wilson
|
aa63e2960c
|
block addresses from reconnecting if they send too much data
|
2021-07-28 16:43:51 +01:00 |
|
|
James Wilson
|
5f022069db
|
cargo fmt
|
2021-07-28 16:15:34 +01:00 |
|
|
James Wilson
|
83d31ef0b3
|
Add rolling total and allow control over bytes per second allowed from node connections
|
2021-07-28 16:08:02 +01:00 |
|
|
James Wilson
|
9ec48adcaa
|
Wee tidy up of test code
|
2021-07-27 14:45:45 +01:00 |
|
|
James Wilson
|
61fd903f55
|
Remove benchmark things for now and no need for release mode on a test
|
2021-07-27 14:28:27 +01:00 |
|
|
James Wilson
|
2ca5f6a1fb
|
cargo fmt
|
2021-07-27 12:38:05 +01:00 |
|
|
James Wilson
|
a44f39cfaf
|
Add/test max_nodes_per_connection feature
|
2021-07-27 12:37:32 +01:00 |
|
|
James Wilson
|
ecf5fccaab
|
make possible to test, test, and fix feed cutoff
|
2021-07-26 16:38:24 +01:00 |
|
|
James Wilson
|
50935b29fc
|
put a deadline on message sending to feeds
|
2021-07-26 11:52:46 +01:00 |
|
|
James Wilson
|
e043de8ddb
|
Expose raw Soketto interface via ws_client for more control where needed (eg in tests)
|
2021-07-26 11:31:40 +01:00 |
|
|
James Wilson
|
cf0e424803
|
Adapt shard to use hyper+soketto instead
|
2021-07-23 17:07:29 +01:00 |
|
|
James Wilson
|
480abd1e49
|
remove println
|
2021-07-23 16:21:30 +01:00 |
|
|
James Wilson
|
0eff32d10e
|
Migrate telemetry_core to Hyper+Soketto
|
2021-07-23 15:57:50 +01:00 |
|
|
James Wilson
|
649fb966d2
|
Improve comment and bias select loop
|
2021-07-21 17:48:18 +01:00 |
|
|
James Wilson
|
cedc2f9dbe
|
tweak a comment
|
2021-07-21 13:59:44 +01:00 |
|
|
James Wilson
|
999d8ff0bd
|
debounce feed polling a little to reduce CPU load when lots of messages are being sent out
|
2021-07-21 12:06:22 +01:00 |
|
|
James Wilson
|
cca1df2e21
|
use ws_client in shard, too, and make it better at handling core disconnecting
|
2021-07-20 13:04:10 +01:00 |
|
|
James Wilson
|
36c4e7b0ef
|
Remove flamegraph bits
|
2021-07-19 17:11:31 +01:00 |
|
|
James Wilson
|
73a4dbb095
|
Allow tests to hook up to running process to help with profiling
|
2021-07-19 15:57:39 +01:00 |
|
|
James Wilson
|
f77afe0b75
|
avoid overquota in soak tests
|
2021-07-16 18:07:10 +01:00 |
|
|
James Wilson
|
582c49413b
|
Make server able to run old or new binary so that we can compare the actix version with this one
|
2021-07-16 17:58:26 +01:00 |
|
|
James Wilson
|
81a0140c3b
|
bump wait-time for port to be ready; build-host is slower to compile than my laptop..
|
2021-07-16 10:03:38 +01:00 |
|
|
James Wilson
|
fd79b3e85b
|
Add some tests, and add a configurable soak test
|
2021-07-15 17:18:58 +01:00 |
|
|
James Wilson
|
db8ea9a8f3
|
Add more tests, fix (and test for) a deadlock re overquota messages, more unbounded channels and less .awaiting
|
2021-07-15 10:53:02 +01:00 |
|
|
James Wilson
|
86a3edf053
|
increase wait-for-port timeout to accomodate for --release mode and slower machines
|
2021-07-14 15:14:47 +01:00 |
|
|
James Wilson
|
e34da5761f
|
move tests around and add (non working) benchmark
|
2021-07-14 15:05:09 +01:00 |
|
|
James Wilson
|
c1208b9e81
|
cargo fmt
|
2021-07-13 16:51:24 +01:00 |
|
|
James Wilson
|
9ac5ea7624
|
Various bits and pieces to address PR comments
|
2021-07-13 16:51:14 +01:00 |
|
|
James Wilson
|
faedba87d4
|
no e2e feature, and don't assume 'cargo run' is a good default
|
2021-07-13 11:12:23 +01:00 |
|
|
James Wilson
|
66bf24dc4d
|
re-add e2e (again..)
|
2021-07-13 09:25:54 +01:00 |
|
|
James Wilson
|
74d5890ea0
|
Wee tidy up
|
2021-07-13 09:21:43 +01:00 |
|
|
James Wilson
|
61d6b620b6
|
Add small comment to make diff more obvious
|
2021-07-13 08:57:29 +01:00 |
|
|
James Wilson
|
cfe7ff39bb
|
contains_matches macro and more tests
|
2021-07-12 17:21:01 +01:00 |
|
|
James Wilson
|
c6c262c9c5
|
tidyup, add more e2e tests, fix (and test a little) feed msg decoding, supporting bits
|
2021-07-12 13:17:24 +01:00 |
|
|
James Wilson
|
f2f122285e
|
make it obvious that unbounded channels don't need to await, and tidy up some bits
|
2021-07-12 10:01:23 +01:00 |
|
|
James Wilson
|
4046edc47d
|
Merge branch 'jsdw-sharding-test-runner' into jsdw-sharding
|
2021-07-09 21:23:38 +01:00 |
|
|
James Wilson
|
65178de82f
|
Just one dockerfile to simplify things a little and reduce repetition
|
2021-07-09 20:57:39 +01:00 |
|
|
James Wilson
|
28be68e65f
|
Get a basic first test working, and lots of changes to supporting code to facilitate this
|
2021-07-09 19:41:53 +01:00 |
|
|
James Wilson
|
c043393e28
|
Write some basic test utils to get going; time to start trying them
|
2021-07-09 19:41:51 +01:00 |
|
|
James Wilson
|
f2adead2e9
|
rename binaries for clarity, and first pass of connect_to_servers test util
|
2021-07-09 19:41:47 +01:00 |
|
|
James Wilson
|
8bf412cad9
|
Begin writing the core utils that we'll use for testing things
|
2021-07-09 19:41:36 +01:00 |
|
|
James Wilson
|
11c0db389a
|
Get docker-compose working again
|
2021-07-09 19:40:52 +01:00 |
|
|
James Wilson
|
ea52d36999
|
make sure to gracefully handle 'close' messages
|
2021-07-05 15:11:20 +01:00 |
|
|
James Wilson
|
6910b4dca4
|
typo
|
2021-07-05 10:45:04 +01:00 |
|
|
James Wilson
|
750aaa06dc
|
Tweak docker(compose) files (can't test them yet) and add a little backend documentation
|
2021-07-05 10:26:32 +01:00 |
|
|
James Wilson
|
ea98b15def
|
No need for LogLevel
|
2021-07-01 15:59:41 +01:00 |
|
|
James Wilson
|
6e4756ecc0
|
also add back telemetry license
|
2021-07-01 15:13:59 +01:00 |
|
|
James Wilson
|
3c371ef9a8
|
add license back to Cargo.toml files
|
2021-07-01 15:12:39 +01:00 |
|
|
James Wilson
|
4bb1f19085
|
add some doc comments
|
2021-07-01 09:58:18 +01:00 |
|
|
James Wilson
|
5d00445324
|
Make syntax of id_type identical to normal tuple struct
|
2021-07-01 09:38:53 +01:00 |
|
|
James Wilson
|
509542e460
|
fmt
|
2021-07-01 09:38:26 +01:00 |
|
|
James Wilson
|
16747dd66c
|
remove a few unneeded structopt bits
|
2021-06-30 17:28:51 +01:00 |
|
|
James Wilson
|
f7ab3292c2
|
Move a bunch of things around and flatten common crate
|
2021-06-30 17:27:10 +01:00 |
|
|
James Wilson
|
4308359feb
|
Give things unique ID types, not aliases, to prevent mixups
|
2021-06-30 16:59:03 +01:00 |
|
|
James Wilson
|
06bd660599
|
Passing a callback isn't worth the extra code; just pass a feed thing
|
2021-06-30 14:20:58 +01:00 |
|
|
James Wilson
|
770739c7c8
|
Finish first pass update_node impl
|
2021-06-30 12:14:17 +01:00 |
|
|
James Wilson
|
c5ca84ee9a
|
Add some State tests, and use genesis_hash, not label, where possible
|
2021-06-28 11:20:48 +01:00 |
|
|
James Wilson
|
8a0eb14aca
|
test and fix most_seen
|
2021-06-28 10:29:40 +01:00 |
|
|
James Wilson
|
00c6e4f4c5
|
simplify feed sending a little
|
2021-06-25 17:39:22 +01:00 |
|
|
James Wilson
|
89dfad5bbe
|
Handle removing a node, and a shard disconnecting (bulk remove)
|
2021-06-25 17:21:24 +01:00 |
|
|
James Wilson
|
4f60453689
|
handle port more properly in real_ip filter and a little refactor
|
2021-06-24 20:28:30 +01:00 |
|
|
James Wilson
|
fb80edba47
|
Update node locations when they come in, and get the real IP addr of nodes
|
2021-06-24 18:34:32 +01:00 |
|
|
James Wilson
|
e383866322
|
locator shuffling around
|
2021-06-24 09:54:00 +01:00 |
|
|
James Wilson
|
47c12ce210
|
Lots more refactoring, finish add node (and almost the location updating)
|
2021-06-23 18:03:11 +01:00 |
|
|
James Wilson
|
2db2677217
|
bimap to store global ID mappings: we'll assign them in node state
|
2021-06-23 10:12:50 +01:00 |
|
|
James Wilson
|
7dfc582a20
|
feed/shard disconnects can be handled, and unbounded output to feeds
|
2021-06-23 09:55:28 +01:00 |
|
|
James Wilson
|
63283195ad
|
WIP filling in core aggregator match arms and various other tweaks
|
2021-06-23 09:34:56 +01:00 |
|
|
James Wilson
|
83e2cee295
|
Remove 'remove' logging
|
2021-06-21 16:28:04 +01:00 |
|
|
James Wilson
|
19ef458e5b
|
Allow multiple SystemConnects to be handled from a single node in the shard
|
2021-06-21 16:27:42 +01:00 |
|
|
James Wilson
|
06d131bf3f
|
Add a note about closing ws with statuscode+reason
|
2021-06-21 14:23:29 +01:00 |
|
|
James Wilson
|
20524ac8ae
|
wrap assigning local/global IDs into struct to avoid things getting out of sync
|
2021-06-21 14:18:44 +01:00 |
|
|
James Wilson
|
486418e5e9
|
Fix CI
|
2021-06-21 10:54:46 +01:00 |
|
|
James Wilson
|
dfe016597e
|
Rework: Shard working, Telemetry Core needs logic filling in
|
2021-06-21 10:45:31 +01:00 |
|
|
James Wilson
|
9741b0f910
|
Remove NodeConnector from core for now; only messages from shards until refactor
|
2021-06-16 14:58:17 +01:00 |
|
|
James Wilson
|
3a527e69c8
|
Get chatter between shard and core working
|
2021-06-16 11:57:58 +01:00 |
|
|
James Wilson
|
2b0accbbb6
|
Ci tidy
|
2021-06-16 11:34:33 +01:00 |
|
|
James Wilson
|
5b01179ca2
|
tweak CI to work with shards+core split
|
2021-06-16 11:31:19 +01:00 |
|
|
James Wilson
|
588f1ea027
|
remove a few unnecessary structs
|
2021-06-16 11:24:00 +01:00 |
|
|
James Wilson
|
8e25b4fbdf
|
Split msg into JSON and internal variant, and other bits
|
2021-06-16 11:08:33 +01:00 |
|
|
James Wilson
|
c276c2065a
|
rename shared to common to disambiguate from 'shard'
|
2021-06-14 10:57:06 +01:00 |
|