LBRYFoundation/reflector.go
1
0
Fork 0
mirror of https://github.com/LBRYFoundation/reflector.go.git synced 2025-09-21 02:19:46 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
454 commits 6 branches 14 tags 3.4 MiB
Commit graph

160 commits

Author SHA1 Message Date
Niko Storni
b098aa2dc8 stupid viper... 2025-05-14 00:58:37 +02:00
Niko Storni
4928e430c0 fix bugs 2025-05-13 22:52:40 +02:00
Niko Storni
d21b0867dd remove unused cmds 
change conf param
fix upload cmd
 2025-05-13 22:30:40 +02:00
Niko Storni
32a85a815e adjust s3 configuration 
hardcode multiwriter for now
 2025-05-13 18:01:28 +02:00
Niko Storni
77836f9c83 prepare stores for reflector upload 
create a new reflector cmd that will replace old reflector cmd
 2025-05-13 05:54:46 +02:00
Niko Storni
9716e41d63 replace hub servers with valid ones 
change ITTT metrics
fix tracing for http store
remove unused launch params
 2025-04-25 20:04:50 +02:00
Niko Storni
7260388255 make servers modular 
fix bugs
 2025-04-25 17:39:52 +02:00
Niko Storni
071a7907c1 Refactors store implementations for config loading 
Updates store implementations to load configurations from YAML files.

This change introduces a configuration loader that reads store parameters from a YAML file, allowing for more flexible and manageable store configurations.
 2025-04-24 05:11:52 +02:00
Niko Storni
185900582d add support for sharding in http stores 2025-04-19 04:21:46 +02:00
Niko Storni
03445476ec rename stores to better reflect what they do 
move cleanup code in db store
fix bugs
refactor code
update go
 2025-04-19 03:49:54 +02:00
Niko Storni
e5a9590cb6 make s3 endpoint configurable 
update deps
 2024-12-16 22:35:50 +01:00
Niko Storni
e93c097fd9 upgrade dependencies 
replace deprecated function calls
refactor build process
 2023-03-09 18:41:41 +01:00
Niko Storni
5693529216 protect protected content 2022-07-29 04:59:15 +02:00
Niko Storni
29d1ccf68c add singleflight to web requests 2022-05-02 23:07:22 +02:00
Niko Storni
4af5c2f4c6 make slack channel configurable 2021-10-30 00:21:58 +02:00
Niko Storni
86f3e62aa8 fix a panic error 
update gin-go
 2021-08-05 17:47:20 +02:00
Niko Storni
b4913ecedf cleanup 2021-07-24 01:03:51 +02:00
Niko Storni
63a574ec2f unify caches 
fix tests
 2021-07-24 01:03:51 +02:00
Niko Storni
b8af3408e0 move server packages 2021-07-24 01:03:51 +02:00
Niko Storni
2b458a6bd0 fix params 
more cleanups
 2021-07-24 01:03:51 +02:00
Niko Storni
febfc51cb0 refactor refactor refactor 2021-07-24 01:03:51 +02:00
Niko Storni
63aacd8a69 use O_DIRECT to write to disk (fixes everything) 
add queue back to serving blobs
improve a lot of things
upgrade modules
 2021-07-24 01:03:51 +02:00
Niko Storni
2651a64dbb add http server/client 2021-07-24 01:03:51 +02:00
Niko Storni
2be913b077 request queue size param 2021-07-24 01:03:51 +02:00
Niko Storni
64acdc29c3 improve disk cleanup 
add index to is_stored
fix test
replace LRU cache
 2021-07-24 01:03:51 +02:00
Niko Storni
766238fd7e add if this than that store 
switch to wasabi for uploads
 2021-07-24 01:03:47 +02:00
Niko Storni
ac5242f173 add integrity check cmd 
throttle live integrity checks
bug fixes
 2021-07-24 01:03:47 +02:00
Niko Storni
36d4156e2a add tracing to blobs 2021-07-24 01:03:47 +02:00
Alex Grintsvayg
6f95b3395f avoid heavy interpolateparams call 2021-07-24 01:03:47 +02:00
Niko Storni
5794c57898 save uploaded blobs and work around the blocklist issue 2021-07-24 01:03:47 +02:00
Niko Storni
35c713a26e add cmd to populate db 
fix store init
try fixing unreasonable db bottleneck
 2021-07-24 01:03:47 +02:00
Alex Grintsvayg
6fb0620091 something like this 2021-07-24 01:03:38 +02:00
Niko Storni
03df751bc7 add PoC for litedb to avoid all the overhead 2021-07-24 01:03:16 +02:00
Niko Storni
84fabdd5f4 add option to run with RO-CF only as upstream 
increase idle timeout to avoid errors downstream
add option to delete blobs from DB if storage doesn't have it (for future local tracking)
 2021-07-24 01:03:16 +02:00
Niko Storni
ff13d7b2f7 fix cache size mess 2021-07-24 01:03:10 +02:00
Niko Storni
7f5a89fa5a fix buffer cache running out of space 2021-07-24 01:03:10 +02:00
Niko Storni
704e15f8c1 use LFUDA store 
swap size to bytes
 2021-07-24 01:03:10 +02:00
Alex Grintsvayg
176e05714e rename cahces 2021-07-24 01:03:10 +02:00
Niko Storni
eefd84b02d add buffer cache for nvme drive 2021-07-24 01:03:10 +02:00
Alex Grintsvayg
2cf4acdb59
add 'send' command to efficiently send a file to reflector 2021-04-02 14:30:36 -04:00
Alex Grintsvayg
560e180e36
separate singleflight cache wrapper, component names for cache metrics 2020-11-04 16:03:46 -05:00
Alex Grintsvayg
124d4065c2
split cloudfront into RO and RW stores 2020-11-04 16:03:45 -05:00
Alex Grintsvayg
3608971f0b
add lru cache eviction metric 2020-11-04 16:03:44 -05:00
Alex Grintsvayg
c9fa04043c
rename the stores, add caching to reflector cmd 2020-11-04 16:03:44 -05:00
Alex Grintsvayg
69fa06420b
LRU cache for disk store, abstract fs in disk store for testing 2020-11-04 16:03:43 -05:00
Niko Storni
f043516a14 revert upload changes 2020-10-15 03:08:27 +02:00
Niko Storni
f1875454cc store uploaded blobs 2020-10-15 00:12:18 +02:00
Alex Grintsvayg
8364d3fc54
small changes for easier testing 2020-10-05 18:08:54 -04:00
Alex Grintsvayg
de0ccd4da7
track approximate access time for blobs 2020-10-05 18:08:53 -04:00
Alex Grintsvayg
5658fe4607
add publish command 2020-10-05 16:30:57 -04:00