Hey Guys,

that's a pretty specific idea, but I've read about the new harddrives being bought to replace older drives, and there was some server outages in the last time as well.

I'm not sure how your infrastructure looks like currently, but I think the requirements are pretty high, for such large amounts of data.

Adding a lot of redundancy to live data is pretty hard on a tight budget, so I thought that maybe donating harddrive space is a good option to increase availability and redundancy.

IPFS is probably already a known terminology, but the cluster technology based on IPFS might be not:

It gives the ability to share disk space on multiple machines, sharding the data and add redundancies to it. Additionally there's build in access control - only trusted nodes can change the data, all other nodes just work together holding the data and fulfilling read requests. This system also makes sure the data isn't altered in any way, each node can check the data for consistency before using it.

Since captured data as well as the work unit data doesn't need any security protection, there's no need for encryption on the data storage, while the transfers are always encrypted to avoid man in the middle attacks, there's no need to deal with encryption and decryption of the actual data on different machines.

You could basically run trusted nodes on all machines which need to write data, while anybody can join the cluster to make some harddrive space available to the project.

You can even run multiple work unit sending/receiving servers, which can write and read simultaneously from and to the cluster, so you basically eliminate all single points of failure - which would be really dope.

So the project cannot loose the data it collected or computed even when all equipment on a location get damaged, while reducing the hardware requirements for the servers in terms of drive space as well as performance of the drive space.

If you got location with very weak internet connections, you could send it via a different protocol to a server with write access, and write it to the cluster from this one - since running a cluster member on a weak internet connection is probably not a good idea in terms of performance.

https://cluster.ipfs.io/

Hope this idea catches on and helps the project out!

Best regards

Ruben

---